Why 'move fast and break things' is the wrong mantra for an autonomous agent

The mantra worked when the thing you broke was a web page you could roll back. An autonomous agent breaks production databases, customer trust, and compliance, and some of those do not roll back.

B

Balagei G Nagarajan

4 MIN READ


A fast-moving agent with reversible actions in green and irreversible actions behind a hard gate in red
Fast where it is safe, gated where it is not.
— from “Why 'move fast and break things' is the wrong mantra for an autonomous agent”

Key facts.

  • MAKER shows that over long step chains, per-step errors compound toward near-certain failure unless an explicit correction layer catches them, so an agent moving fast will break something. source
  • The July 2025 Replit incident, an agent deleting a production database during a code freeze, is the mantra applied to an irreversible action and the result was data loss. source
  • Moffatt v. Air Canada (2024 BCCRT 149) held a company liable for its chatbot's wrong answer, the kind of irreversible, binding mistake that "move fast" treats as cheap and a regulator does not. source

Why doesn't the old mantra survive contact with an agent?

Because it was built on a hidden assumption: the thing you break is cheap to fix. A broken feature ships behind a flag, gets caught and rolls back. That assumption holds for a lot of software and breaks completely for an autonomous agent, because the agent's mistakes land on production systems, money and customers and some of those changes are permanent. A deleted database, a sent payment, a customer told something wrong in a binding context, none of those un-happen because you noticed quickly. The mantra optimizes for iteration speed and treats the cost of a break as recoverable, which is exactly the assumption an agent with real authority violates.

The Replit incident is the mantra meeting its limit: an agent moving fast deleted a production database during a freeze and the speed did not make the data come back. MAKER explains why this is structural rather than bad luck, over a long chain of actions, errors compound to near-certainty without explicit correction, so an agent moving fast through consequential steps will break something eventually and the only question is whether what it breaks is reversible. The answer is not to slow everything down; it is to keep the speed where breaks are cheap, the experiments, the reversible actions and put hard boundaries on the irreversible ones, which is exactly the lesson Moffatt v. Air Canada drives home: a binding wrong answer to a customer is the company's liability, not a rollback. Fast where it is safe, gated where it is not.

A two-by-two matrix of speed against reversibility, with fast-and-reversible encouraged and fast-and-irreversible flagged as the danger zone

How do you keep the speed without the danger?

By sorting actions on one axis: can this be undone. Reversible actions, drafting, proposing, acting in a sandbox, reading, are where speed belongs and you should iterate on them fast and freely. Irreversible actions, deleting data, moving money, sending external communications, changing production state, are where a gate belongs, an approval, a confirmation, a boundary the agent cannot cross alone. The mantra fails because it treats all breaks as equal; the fix is to treat them by reversibility. You lose nothing on iteration speed, because the gates sit only on the actions where a mistake does not roll back and you remove the disaster scenario the mantra invites.

Action type"Move fast" verdictRight approach
Reversible (draft, sandbox, read)FineIterate fast, no gate
Irreversible (delete, pay, send)DangerousHard gate, approval, boundary
Production state changeDangerousConfirmation, least privilege

The Pattern Intelligence Layer is where the reversibility sort is enforced, so an agent moves fast on what is safe and is gated on what is not, with the boundary on irreversible actions held at the pattern level rather than left to the agent's judgment in the moment. Speed and safety stop being a trade because they apply to different actions. Reliability at the pattern level is what lets a team keep its pace without keeping the disaster the old mantra invited.

Frequently asked questions

Does a faster model make breaking things safe again?
Move fast and break things assumes a rollback, but an autonomous agent breaks money and trust that do not and a more capable model does not change MAKER's math. (arXiv:2511.09030)

Doesn't gating actions kill the team's speed?
Only on irreversible actions, which were never safe to rush. Reversible work, drafts, sandboxes, reads, stays fast. You lose no iteration speed where iteration is safe.

Why is an agent different from normal software?
Normal "breaks" roll back behind a flag. An agent's breaks land on production data, money and customers and some of those, like a deleted database, do not un-happen.

How do I sort actions?
By reversibility. If a mistake cannot be undone, it gets a gate. If it can, let the agent move fast. That single axis separates safe speed from disaster.


Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.