Even a frontier model's tool accuracy decays as your catalog and context grow. In long-context function-calling evaluations, top models lose accuracy once the tool list and surrounding context expand (LongFuncEval, arXiv:2505.10570, 2025). If you run an enterprise agent against real, evolving APIs, your tool layer is one quiet schema change away from confidently calling the wrong thing.
Key facts.
- Tool calling injects a catalog of JSON schemas and descriptions into the model's context every turn; the model picks a function and writes arguments from that text alone, so a stale description is a stale decision. Manual schema definitions and their maintenance overhead are documented as the core friction (Unified Tool Integration for LLMs, arXiv:2508.02979, 2025).
- Tool-calling accuracy degrades as the tool set and context length grow, the conditions a sprawling, drifting catalog creates (LongFuncEval, arXiv:2505.10570, 2025).
- Contamination-free multi-step function-calling evaluation shows models mis-select and mis-fill calls when tool definitions are imperfect or shift, not just when the task is hard (Controllable multi-step function-calling evaluation, arXiv:2509.26553, 2025).
- Standardizing the boundary as an enforceable, discoverable schema is the documented fix for ad-hoc definitions that drift (Model Context Protocol, Anthropic, 2024).
What is tool schema drift?
APIs change under a frozen schema, so the agent calls a gone endpoint; a more capable model reads the same contract, rework grows. (arXiv:2505.10570)
Schema drift is the slow gap that opens between the tool definition your agent reads and the API it is supposed to describe. The model never calls your API directly. It reads a JSON schema plus a description, decides which tool fits, and writes arguments to match. That schema is usually authored once and checked in. Your backend teams keep shipping: a field becomes required, a parameter gets renamed for clarity, an endpoint is deprecated in favor of a v2. None of that reaches the frozen schema, and the description still promises the old behavior. So the model reasons correctly over a contract that quietly stopped being true, and produces a confident call the API rejects, or worse, accepts in the wrong shape.
Why does drift make the agent pick the wrong tool?
Because tool selection is a reading task over your descriptions. When two tools sound similar, get_order_status and get_order_details, the model leans on the wording to choose. Let those descriptions go stale or ambiguous and selection accuracy drops: benchmarks show that adding semantically related tools makes models start picking the wrong one (BFCL). Drift makes every description a little less trustworthy, so an already fragile choice gets worse. Then the model fills arguments from the same stale schema, so even the right tool gets the wrong inputs. It looks like a model problem. It is a contract problem.
// Schema the agent still reads { "name": "create_invoice", "parameters": { "customer_id": "string", "amount_cents": "integer" }, "required": ["customer_id", "amount_cents"] }// API after last sprint: currency is now required POST /v2/invoices -> 422 {"error":"missing required field: currency"} // The agent keeps emitting valid-looking calls. The API keeps rejecting them.
How do you keep schemas honest as systems evolve?
Stop hand-maintaining schemas. Generate them from the live source of truth, an OpenAPI spec or typed function signatures, so a backend change regenerates the agent's contract in the same pipeline. Version the tools and pin the agent to a known version, then run a regression check on every schema change. Validate each call at the agent-to-tool boundary against the current schema, so a drifted argument fails loudly at the seam instead of silently downstream. A contract standard like the Model Context Protocol helps, because it treats the schema as an enforceable, discoverable interface rather than a comment. The result you want: when an API changes, the agent's view of it changes with it.
Sources of drift and the fix
| Drift source | What it breaks | Fix |
|---|---|---|
| New required field | Invalid calls (422) | Regenerate schema from spec each release |
| Renamed parameter | Wrong argument names | Typed generation + boundary validation |
| Deprecated endpoint | Calls to dead routes | Version pinning + regression test |
| Ambiguous description | Wrong tool selected | Description-quality checks, distractor tests |
| Silent type change | Malformed payloads | Runtime schema validation at the seam |
The pattern is that an agent is only as current as the schemas and descriptions it reads, and enterprise APIs move faster than hand-checked JSON. Generate schemas from the live source, version them, and validate at the boundary, and the agent keeps choosing and calling tools correctly while your systems change around it. That is reliability at the pattern level, the contract between agent and tool treated as something you verify rather than hope holds, which is what VibeModel builds as the Pattern Intelligence Layer.
Frequently asked questions
What is tool schema drift in one line?
The gap between the tool definition your agent reads and the live API it describes, which widens every time the API changes and the schema does not.
Won't a bigger model fix this?
No. A bigger model reads the same stale schema and picks and fills tools more fluently from a contract that is still wrong. Drift is fixed in the pipeline, not the weights.
How do I catch drift before users do?
Generate schemas from the API spec and diff them in CI, validate calls at the boundary, and keep a regression suite of tool-selection cases that runs on every schema change.