Key facts.
- PALADIN (arXiv:2509.25238) documented that baseline tool recovery without structured failure handling was 32.76%; unstructured retries rarely recovered from semantic or state-related failures (PALADIN, arXiv:2509.25238).
- tau2-bench (arXiv:2506.07982) exposed agents to retry-eligible failures in realistic tool-agent-user interaction scenarios; models lacking explicit retry taxonomy continued failing across retries when the underlying error was semantic rather than transient.
- IMF Note 2026/004 on agentic payments highlighted that payment processing agents retrying failed transactions without idempotency keys risk duplicate charges, regulatory violations, and reconciliation failures that compound with each retry.
- LLM token consumption on retry spirals can exceed 10x the cost of a single successful call when the agent includes full conversation history on each attempt, making retry cost a significant production expense at scale.
- Non-idempotent calls, sending an email, creating a record, initiating a payment, retried without checking partial success create duplicate-action failures that are harder to fix than the original error.
Why retry logic breaks for LLM agents
TCP/IP retry model: operation failed, failure was transient, retrying will eventually succeed, retrying is safe because the operation is idempotent. LLM tool calls break all four regularly. A tool call fails because the model generated a wrong parameter: retrying with the same parameter fails again. A payment API returned an error on a call that actually succeeded on the backend: retrying creates a duplicate charge. A record creation call timed out but the record exists: retrying creates a duplicate record.
The cost amplification compounds because LLM agents typically include their full context window on each retry. A conversation that has reached 8,000 tokens adds 8,000 tokens to every retry attempt. Three retries cost 3x the base conversation cost, on top of the original failure. For high-volume agents, retry spirals are a leading cause of production cost overruns.
The four retry failure modes and what to do instead
Four distinct failure modes require four distinct responses. Transient failures (network timeout, rate limit 429) are the only category where exponential backoff is appropriate. Semantic failures (wrong parameter, unauthorized endpoint) require the agent to change its approach, not repeat it. State-uncertainty failures (operation may or may not have succeeded) require a verification call before any retry. Non-idempotent failures require an idempotency key created before the first attempt, and a query-before-retry pattern for all subsequent attempts.
Retry strategy by failure class
| Failure class | Correct response | What most agents do | Cost of wrong response |
|---|---|---|---|
| Transient (timeout, 429) | Exponential backoff, max 3 retries | Exponential backoff (correct) | Low if bounded |
| Semantic (wrong parameter) | Change approach, new parameter | Retry with same parameter | 3x token cost, same outcome |
| State uncertain (timeout on write) | Verify then conditionally retry | Retry blindly | Duplicate action, remediation cost |
| Non-idempotent | Idempotency key, query before retry | Retry blindly | Duplicate charge or record, compliance risk |
VibeModel's Pattern Intelligence Layer classifies tool failure types at execution time and routes each to the correct recovery path rather than applying a generic retry policy. The pattern layer knows, from historical data, that a specific tool's 400 responses are semantic failures that require parameter correction, while its 503 responses are transient failures that respond to backoff. That classification is learned from production failure patterns, not assumed from HTTP semantics - and it's the difference between a bounded retry cost and a runaway retry spiral.
Frequently asked questions
How do I add idempotency keys to tool calls that don't natively support them?
Handle it at the tool wrapper level. Generate a session-scoped key before the first attempt, store the key and result, return the stored result on subsequent calls instead of re-executing. Most payment and communication APIs accept idempotency keys natively.
what's a reasonable retry limit for agent tool calls?
Three retries maximum for transient failures, with exponential backoff. Zero retries for semantic failures (fix the parameter instead). One verification-then-conditional-retry for state-uncertain failures. The limit matters: agents that don't bound retries create unbounded cost exposure.
Does this require changes to the model or to the scaffolding?
Scaffolding. The model gets failure signals; the scaffolding decides how to handle them. Failure classification and retry routing belong in the tool wrapper layer, deterministic, testable, no model involvement.