<p><b>Key facts.</b></p>
<ul>
<li>Even state-of-the-art function-calling agents finish under 50% of realistic tool-use tasks, and consistency across repeated runs collapses (pass^8 under 25% in retail), a gap that compounds every time one tool output feeds the next call (Yao et al., <a href="https://arxiv.org/abs/2406.12045" target="_blank" rel="noopener">tau-bench, arXiv:2406.12045</a>, 2024).</li>
<li>A 2025 taxonomy of failures in tool-augmented LLMs treats parsing failures, where a tool call or its returned output cannot be correctly parsed, as a distinct root-cause category, separate from the model reasoning wrong (Winston and Just, A Taxonomy of Failures in Tool-Augmented LLMs, IEEE/ACM AST 2025).</li>
<li>Parameter-name hallucination is the single biggest driver of multi-turn tool failures; when every parameter name is correct, task accuracy approaches 100% (<a href="https://arxiv.org/abs/2412.16516" target="_blank" rel="noopener">HammerBench, arXiv:2412.16516</a>, 2024).</li>
<li>The Berkeley Function-Calling Leaderboard grades each generated call for structural correctness with an abstract-syntax-tree check, and multi-turn scores fall sharply below single-call scores, where one mis-formed or mis-read output derails the rest (BFCL, gorilla.cs.berkeley.edu, 2024-2026).</li>
</ul>
<h2>What does a tool-output parsing failure actually look like?</h2>
<p>Your agent takes a wrong action even when the tool worked. The model emitted a malformed call. Mis-read a returned field. Acted on a truncated payload. Treated an error body as success. No exception fires. The agent loop continues on corrupted state. Four failure modes: invalid JSON in the call with a missing brace or wrong syntax, making arguments unusable. Valid returned JSON that the agent mis-reads, pulling the wrong field. Output too long and silently clipped, so the model reasons over half a payload. Tool returns an error inside a success envelope and the agent treats it as done. None of these throws an exception. The loop continues. The state is wrong. Every downstream step inherits it.</p>
<h2>Why does a tool call that worked still corrupt the agent?</h2>
<p>Most agent code checks that a call returned, not what it returned. A REST endpoint can answer 200 OK with a body that says success is false and carries an error message inside. An agent that branches on the status code reads that as a completed action. Moves on. Confidently reports a refund issued or a record updated that never happened. The fix is to make the agent read the body, require an explicit success-or-error flag, and validate that before it claims anything is done.</p>
<pre style="background:#0d1117;color:#e6edf3;font-family:ui-monospace,Menlo,Consolas,monospace;font-size:0.9rem;line-height:1.65;padding:16px 18px;border-radius:12px;overflow-x:auto;margin:1.8rem 0;border:1px solid #20262e;white-space:pre;"><span style="color:#8b949e;"># HTTP 200 OK, yet the action never happened</span>
{ "status": "ok", "data": { "success": false, "error": "insufficient_balance" } } # Branch on the 200 alone and the agent reports "refund issued". It was not.
Why do long tool outputs quietly poison the next step?
The dangerous truncation is the one nobody configured. Raw tool results dropped straight into context: an 8,000-token HTML page, a sprawling JSON, a giant log. The framework clips it to fit and the model reasons over a fragment. The cut usually lands mid-structure. A record loses a field. A list loses its tail. The model gap-fills with a plausible wrong value rather than flagging the gap. Summarize or paginate large tool results. Mark them with a clear truncation signal so the model knows data is missing.
Why doesn't a bigger model fix this?
The model is a next-token predictor, not a guaranteed parser. The failures live in the seams between probabilistic generation and rigid downstream code. A stronger model nudges the numbers. It does not turn a malformed payload into a valid one or make a swallowed error surface. That is engineering, not model scale.
How reliable is function calling really?
Better than it was. Not safe to trust blind. Native function calling and strict structured outputs sharply cut malformed-JSON rates. The Berkeley leaderboard puts the best models in the high-70s percent on call correctness. But multi-turn and agentic scores run well below single-call scores. HammerBench shows accuracy swinging from near-perfect to poor purely on whether parameter names are clean. Schema-constrained generation is necessary and not sufficient. You still validate every argument and every returned body against a typed schema before the agent is allowed to act on it.
How do you stop it?
| Failure mode | What you see | Fix |
|---|---|---|
| Malformed call JSON | Args unusable, parse error or silent drop | Native function calling + strict structured outputs |
| Mis-read field or type | Wrong value, no error raised | Validate the returned body with a typed schema (Pydantic) |
| Error-as-success | "Done" on an action that failed | Read the body, require an explicit success/error flag |
| Truncated long output | Confident but incomplete answer | Summarize or paginate, mark the truncation |
| Param-name hallucination | Multi-turn failures, wrong tool wiring | Constrained names, validation, a JSON repair loop |
Constrain the call to a schema. Validate the returned body before the agent acts. Read the contents, not the status code. Bound and mark long outputs so nothing gets clipped in silence. Every tool result is a claim to be checked, not a fact to be trusted. Which tools in your specific workflow are the failure points and which are clean is what VibeModel maps as the Pattern Intelligence Layer.
The pattern under every one of these is that a tool returning is not a tool succeeding. Constrain the call to a schema, validate the returned body before the agent acts, read the contents instead of the status code, and bound and mark long outputs so nothing gets clipped in silence. None of that is a smarter model. It is a verification layer that treats every tool result as a claim to be checked rather than a fact to be trusted, which is what VibeModel builds as the Pattern Intelligence Layer.
Frequently asked questions
Doesn't strict JSON mode or function calling already solve this?
It solves the front half, the shape of the call the model emits, and that matters. It does nothing for the back half: a syntactically valid response whose body reports an error, carries the wrong field, or arrives truncated. You still validate the returned content before acting.
How is this different from the agent just hallucinating?
Hallucination is the model inventing an answer. This is the plumbing between a correctly run tool and the model corrupting otherwise good data: a mis-parsed field, a swallowed error, a clipped payload. A 2025 taxonomy of tool-augmented LLM failures treats parsing failures as their own class for exactly that reason.
What is the single highest-impact fix?
Validate every tool response against a typed schema with an explicit success flag before the agent is allowed to act, and never branch on the HTTP status code alone. That one gate catches error-as-success, mis-read fields, and most truncation.
How do I catch these in production?
Trace the raw tool request and response and record parse-success and validation-pass rates per tool. The failures are invisible in the final answer but obvious in the trace: an error body marked done, a response that failed schema validation but still advanced the loop.