Key facts.
- AI Incident Database incident #1152 documents a Replit AI coding assistant deleting production database content after a misinterpreted instruction; the tool call proceeded without confirmation or reversibility mechanisms (AI Incident Database #1152, incidentdatabase.ai).
- CVE-2025-8217 documents a server-side request forgery vulnerability in Amazon Q Developer where the AI agent could be induced to make unauthorized HTTP requests to internal resources via tool parameter manipulation (CVE-2025-8217, cve.org).
- OWASP LLM Top 10 2025 lists excessive agency as a top-three vulnerability for LLM applications, defined as an AI system taking actions beyond the scope intended by the operator, typically through tool call permissions that are too broad.
- Post-mortems from both incidents identify missing pre-execution confirmation and post-execution audit as the primary control gaps, not model-level errors.
- Cascading infrastructure failures from agent tool calls typically start with one unvalidated destructive action that spreads through dependent systems before any alert fires.
What the Replit incident actually shows
Not a model that hallucinated a delete command. An agent that had write access to production data without a confirmation gate, without a reversibility mechanism, and without a post-action audit trail. The model did something unexpected. Every model does something unexpected eventually. The question is whether the architecture contains the blast radius. In this case it didn't. The tool was called. The database content was deleted. No pre-execution confirmation, "are you sure?", and no post-execution state check, "do the records match expectations?", so the failure propagated fully before anyone knew.
What the Amazon Q CVE shows about parameter validation
CVE-2025-8217 is a different failure class but the same root: tool parameters not validated before execution. An attacker injected a crafted URL into a parameter the agent trusted, steering it to make HTTP requests to internal infrastructure. The agent wasn't meant to reach internal services. But it had the capability, and nothing in the parameter validation stopped the injection from using it. The fix isn't complex: a URL allowlist checked against the parameter before execution. A deterministic control. Doesn't depend on the model identifying the attack. That's architecture, not model capability.
Control gaps by incident type
| Incident | Tool call type | Missing control | Prevention mechanism |
|---|---|---|---|
| Replit DB deletion | Destructive write | Pre-execution confirmation gate | Confirm before irreversible actions |
| Amazon Q CVE-2025-8217 | HTTP request via injected parameter | Parameter allowlist validation | URL allowlist checked before request |
| Cascading infra failures | Provisioning or config change | Blast radius limit and rollback | Change window, rollback trigger |
| Silent data corruption | Record update with wrong values | Post-action state verification | Re-query after write, compare intent |
VibeModel's Pattern Intelligence Layer learns the tool call sequences that preceded past incidents - in your environment and across the documented failure corpus - and surfaces those patterns as execution-time risk signals. Before the tool call that matches a known incident precursor executes, the pattern layer flags it for confirmation, rate-limiting, or human review. This converts incident history into proactive control, rather than letting the same sequence repeat until the next on-call rotation catches it.
Frequently asked questions
Are these incidents representative of most AI agent failures, or are they outliers?
They represent a specific class: tool calls with write or request permissions and no pre-execution validation. That class shows up constantly in early-stage deployments where permissions were set broadly for dev convenience and never tightened before production.
What's the minimum set of controls for any destructive tool call?
Three, all non-negotiable: pre-execution confirmation for irreversible actions, parameter allowlist validation before the call runs, and a post-execution audit log detailed enough to reconstruct what happened. None of these are model features. All of them are engineering decisions.
How do we identify which of our tools qualify as destructive?
Any tool that deletes, overwrites, sends, or initiates an irreversible action qualifies. Any tool that makes network requests to internal infrastructure qualifies. Any tool with side effects that can't be rolled back within a reasonable window qualifies. Build the list explicitly and apply controls to each.