Key facts.
- PALADIN reports agents fail frequently on tool malfunctions (timeouts, exceptions, inconsistent outputs), with recovery rate rising from 23.75% to 89.86% only after training specifically on failure trajectories. source
- Standard agent training optimizes for success trajectories, so the model never learns the failure cases that dominate real deployment. source
- OSWorld's low completion on real computer tasks reflects the same gap: a plan that reads well meets a real environment it did not account for. source
Why does execution expose what planning hid?
PALADIN found tool agents recover 23.75% from timeouts; a more capable model plans well, hits an unmodeled world and retries climb. (arXiv:2509.25238)
Because planning happens in the agent's head and execution happens in the world. When the agent plans, it reasons over an idealized model: the tool returns what its schema promises, the API is up, the data is current. Execution is where those assumptions get tested and the real world fails them constantly. A tool times out. An API returns an empty list instead of an error. A downstream system is briefly inconsistent. The plan had no slot for any of this, so when it happens, the agent does not treat it as a failure. It treats the broken result as the truth and keeps going, building the next step on a foundation that already collapsed.
The reason this is so common is structural. Agents are trained mostly on runs that worked, so the model has seen far more clean executions than broken ones. It is fluent at the happy path and naive about the failure path, which is exactly backwards from what production demands, where the failure path is where the money is lost.
What makes execution reliable?
Treat every tool result as something that might have failed and check it before it flows on. Detect the timeout, the empty response, the malformed output and route to a recovery action instead of pretending the step succeeded. PALADIN's gain came entirely from teaching the agent to recognize and recover from failure, not from a better plan. The plan was never the problem. The agent's blindness to its own broken execution was.
| Agent posture | On a tool failure | Result |
|---|---|---|
| Trained on success only | Accepts the broken result | Builds the next step on a collapse |
| Trained to recover | Detects and recovers or escalates | Execution survives the real world |
This is what VibeModel handles as the Pattern Intelligence Layer. We model the patterns of an execution that has quietly failed, the timeout, the empty payload, the inconsistent state, so the agent acts on what actually happened rather than on what the plan assumed would happen.
Frequently asked questions
Isn't this just error handling?
It is error handling the agent was never trained to do. Recognizing a silent tool failure and recovering is a learned competence, not a default.
Why not just make the plan more detailed?
You cannot enumerate every way the world breaks. Detection and recovery at execution time generalize where an exhaustive plan cannot.
Does a better model fix this?
Only partly. PALADIN's gain came from training on failure cases, which a more capable model does not get for free.