Key facts.
- PALADIN benchmarked tool-calling agents on ToolBench across systematic failure injection: vanilla agents (no recovery) achieved only 23.75% recovery rate when tools fail mid-task, they generate a response but do not complete the workflow (PALADIN, arXiv 2509.25238, ICLR 2026).
- GhostCite found that 13 leading LLMs fabricate citations at rates ranging from 14% to 95% of the time when asked to produce sourced answers, the model generated a complete-looking response, but the citations do not exist (GhostCite, arXiv 2602.06718, 2026).
- Barnett's seven failure points in RAG systems include "incorrect response generation" as a distinct failure from retrieval failure, the retrieval can succeed while the answer fails, because generation and completion are measured separately (Barnett et al., arXiv 2401.05856).
Where does the confusion come from?
Treating generation as completion is a class of silent failure; an upgrade inherits it as production rework. (arXiv:2509.25238)
In traditional software, if a function returned without throwing an exception, it completed. That model trained everyone who now builds agents. The problem is that an agent function can return a coherent, well-formatted, non-null answer and still have done nothing. The tool call timed out but the agent moved on. The API returned an error in the response body that the agent parsed as success. The summary is grammatically correct but refers to the wrong document.
Every one of these cases looks like a completion. The token stream ended. The response is there. Nothing threw. Completion requires a different check: did the world state change in the way we intended? That is a semantic question, and it cannot be answered by observing the output alone.
The four stages of completion
| Stage | What fails here |
|---|---|
| Generation | Hallucinated plan, wrong reasoning, fabricated tool parameter |
| Tool execution | Silent tool failure, timeout, wrong API endpoint |
| State change | Tool reported success but state did not update |
| Verified completion | State updated but not as intended (wrong record, wrong amount) |
The Pattern Intelligence Layer works at the stage boundary. VibeModel checks that each stage actually produced the output required by the next one, and that the final verified completion is what the task specified. Generation is the first stage of a four-stage process. Treating it as the last is how agents fail in production without anyone noticing.
Frequently asked questions
How do I know which stage my agent is failing at?
Instrument each stage separately. Log the tool call intent, the tool response, the state before and after, and a semantic check on the final outcome. When you see a gap between what was intended and what happened, you know exactly which stage introduced the failure.
Does this mean every agent needs a verifier?
Every agent that touches real state does. For pure information retrieval you have more flexibility. For anything that writes, files, orders, tickets, emails, a completion check is not optional, it is the minimum viable safety net.