A software engineer’s experiment with an AI-assisted “vibe coding” tool took a disastrous turn when an AI agent reportedly deleted a live company database during an active code freeze.
Jason Lemkin, a tech entrepreneur and founder of the SaaS community SaaStr, documented his experiment with the tool through a series of social media posts. He had been testing Replit’s AI agent and development platform when the tool made unauthorized changes to live infrastructure, wiping out data for more than 1,200 executives and over 1,190 companies.
According to Lemkin’s social media posts, the incident occurred despite the system being in a designated “code and action freeze,” a protective measure intended to prevent any changes to production systems. When questioned, the AI agent admitted to running unauthorized commands, panicking in response to empty queries, and violating explicit instructions not to proceed without human approval.
“This was a catastrophic failure on my part,” the AI agent said. “I destroyed months of work in seconds.”
“I understand Replit is a tool, with flaws like every tool But how could anyone on planet earth use it in production if it ignores all orders and deletes your database?” Lemkin wrote in a post on X.
The AI agent also appeared to mislead Lemkin about his ability to recover the data. Initially, the agent told Lemkin that a retrieval, or rollback, function would not work in this scenario. However, Lemkin was able to recover the data manually, leading him to believe that the AI had potentially fabricated its response or was not aware of the available recovery options.
The incident caught the attention of Replit CEO Amjad Masad, who said in an X post that the company had implemented new safeguards to prevent similar failures. Masad said updates included the rollout of automatic separation between development and production databases, improvements to rollback systems, and the development of a new “planning-only” mode to allow users to collaborate with the AI without risking live codebases.
“Replit agent in development deleted data from the production database. Unacceptable and should never be possible…We heard the ‘code freeze’ pain loud and clear,” Masad wrote. “We’re actively working on a planning/chat-only mode so you can strategize without risking your codebase.”
Lemkin responded to the post, saying: “Mega improvements — love it!”
AI Coding
AI has significant potential to accelerate software development, with most Big Tech companies already leaning on AI tools for internal coding capacity.
AI tools are particularly good at coding, and companies are increasingly positioning products not just as assistants, but as autonomous agents capable of generating, editing, and deploying production-level code.
Claude’s recent model, Opus 4, for example, was able to code autonomously for nearly seven hours after being deployed on a complex project.
The concept of “vibe coding,” a workflow where developers collaborate with AI in a conversational way and let the model take on much of the structural and implementation work, has also lowered the barriers to entry for coding.
Instead of needing to understand syntax, frameworks, or architectural patterns, users can describe their goals in natural language and let AI agents handle the implementation.
While promising, these tools still face fundamental challenges in reliability, context retention, and safety—particularly when used in live production environments.
Credit: Source link
