Key facts.
- tau2-bench's dual-control tasks require coordinated planning, tool use and state tracking and expose failures at the boundaries between those jobs. source
- A plan that picks the wrong tool produces a plausible result from the wrong source and without verification that result flows downstream as if it were right. source
- OSWorld's low completion rate on real tasks reflects compounded breakdowns across planning and tool interaction, not a single isolated weakness. source
- tau2-bench shows agents fail at the seams of plan, tool and state; a better model leaves them and retries grow. (arXiv:2506.07982)
Why do these three fail together?
Because they sit in a line and each one feeds the next. The plan decides which tool to call. The tool returns a result. The result drives the next step. If the plan is slightly off, it calls a slightly wrong tool. If the tool is slightly wrong, it returns a plausible but incorrect result. If verification is missing, that result is accepted and the next step is built on it. None of these is dramatic alone. A small planning error, a defensible tool choice, a skipped check. Together they produce an agent that confidently did the wrong thing using the wrong source and never noticed, because no stage was responsible for catching the previous stage's mistake.
This is why fixing one in isolation disappoints. Improve the planner and the wrong-tool-plus-no-verification path still fails. Improve tool selection and a bad plan still routes around it. The reliability lives in the joints and the joints are exactly what single-component fixes leave untouched.
How do you defend the seams?
Put a check at each handoff. After the plan picks a tool, confirm the tool fits the step. After the tool returns, verify the result before the next step consumes it. The checks are cheap relative to the compounded failure they prevent and they turn a chain of three soft weaknesses into a chain with three hard gates. The agent does not have to be excellent at planning and tool choice and verification all at once. It has to be unable to pass a wrong result from one stage into the next.
| Where you invest | Effect on compound failure |
|---|---|
| One component, harder | Other two seams still fail together |
| Checks at every handoff | Errors caught before they compound |
This coupling is the reason VibeModel operates as a Pattern Intelligence Layer rather than a single-step fixer. We model the patterns that connect planning, tool choice and verification and place the checks at the seams, so the failure that needs all three to go wrong at once cannot.
Frequently asked questions
Which seam matters most?
The verification gap, because it is what lets every upstream error pass through unchallenged. Start there.
Won't all these checks slow the agent?
Each check is small; the compounded failure is large. The trade favors the checks in almost every production case.
Is this a model problem or a system problem?
A system problem. The seams are architectural and no single model upgrade closes them.