Key facts.
- The OWASP Top 10 for Agentic Applications catalogs the recurring risk classes of autonomous agents, including goal hijacking, excessive agency and cascading failures, each mapping to a control that prevents it. source
- In the July 2025 Replit incident (AI Incident Database #1152), an agent's destructive action during a code freeze was preventable by a development-production boundary, a control the company added afterward. source
- Greshake et al. (arXiv:2302.12173) showed that without an injection check, an instruction hidden in retrieved content is obeyed, turning a missing control into a compliance or data-exfiltration event. source
Why does a missing control become an incident?
Because the failure it would have caught is foreseeable, so when the control is absent, the failure is not a surprise, it is a matter of time. Agents act and some actions are irreversible or sensitive. If there is no approval gate on an irreversible action, eventually the agent takes one you did not want. If there is no injection check on a retrieval path, eventually a poisoned document steers it, the path Greshake demonstrated. If there is no boundary between production and development, eventually the agent reaches production it should not, which is exactly what the Replit incident showed. None of these required an attacker doing something novel. They required a known risk meeting no control and the OWASP agentic list is essentially a catalog of those known risks waiting for controls.
A stronger model does not close these gaps, because the gaps are architectural. Excessive agency is not a model flaw, it is a permission you granted. Cascading failure is not a reasoning error, it is the absence of a circuit breaker. The reason these become security, compliance or reputational incidents is that the consequence is large and the control was small. A development-production boundary is cheap; the database it would have saved was not. The discipline that prevents the headline is unglamorous: enumerate the foreseeable failures, map each to a control and verify the control is actually in place before the agent has the access to make the failure real.
Which controls map to which incidents?
Approval gates map to irreversible-action incidents, so a human confirms before the agent does something that cannot be undone. Permission scoping maps to excessive-agency incidents, so the agent cannot reach systems outside its job. Injection checks map to goal-hijacking incidents, so a hidden instruction in retrieved content is caught before it is obeyed. Environment separation maps to blast-radius incidents, so a mistake in one place cannot destroy another. And freeze enforcement maps to the Replit case directly, so a do-not-act instruction is enforced by the system, not trusted to the agent. Each control is small, each incident is large and the OWASP list is the map between them. Governance is the practice of making sure every entry on that map has its control before the agent ships.
| Foreseeable failure | Missing control | Control that prevents it |
|---|---|---|
| Irreversible action | No approval gate | Human gate on the action |
| Goal hijacking | No injection check | Check on retrieved content |
| Excessive agency | Over-broad permissions | Scoped, least-privilege access |
| Destructive reach | No environment boundary | Production-development separation |
Replit's agent deleting a production database in a freeze was preventable by controls as plain as separating prod from dev; a better model does not solve it, goal hijacking and excessive agency are systemic. (arXiv:2302.12173)
The Pattern Intelligence Layer is where the map between failures and controls is enforced. Risky actions are compared against their required controls at the pattern level, so a missing or bypassed control is visible before the foreseeable failure becomes an incident. Reliability at the pattern level is what keeps a known risk from running all the way to the headline.
Frequently asked questions
Aren't these incidents caused by attackers, not missing controls?
Mostly not. The Replit case had no attacker, just a missing boundary. Even injection-driven incidents need a missing check to succeed, which is the control that was absent.
Why doesn't a more capable model prevent these?
Because the gaps are architectural, like permissions and gates, not reasoning quality. OWASP lists these as properties of autonomous systems regardless of model.
How do I know which controls I am missing?
Enumerate the foreseeable failures for your agent, map each to a control using a list like the OWASP agentic risks and verify each control exists before granting access.