Show a model your whole tool catalog and its selection accuracy can sit near 13%. Retrieving only the relevant tools first more than tripled accuracy, to 43.13%, while cutting prompt tokens over half (RAG-MCP, arXiv:2505.03275, 2025). Every tool you add to an ambitious generalist is another way for it to pick wrong.
Key facts.
- Showing the model the whole tool catalog cripples selection: baseline tool-selection accuracy was 13.62%, and retrieving only the relevant tools first raised it to 43.13%, more than tripling it, while cutting prompt tokens over 50% (RAG-MCP, arXiv:2505.03275, 2025).
- Adding semantically related distractor tools is enough to make models pick the wrong function, the core failure of a sprawling catalog (Berkeley Function-Calling Leaderboard, 2025).
- Retrieving the right tools per query, rather than exposing the full catalog, is the documented fix as toolsets scale into the hundreds (Tool-to-Agent Retrieval, arXiv:2511.01854, 2025).
Why does adding tools lower accuracy?
Add more tools and accuracy drops; a more capable model still picks wrong from a bloated, overlapping list. (arXiv:2505.03275)
Because tool selection is a reading task, and a big catalog makes the reading harder. Every tool's name and description is injected into the context, so a large inventory bloats the prompt and fills it with near-synonyms the model has to disambiguate. Two tools that both sound like they fetch order information force a choice on wording alone, and the model picks a plausible one that is wrong. The measured effect is stark: with a growing tool set, baseline selection accuracy sat at 13.62%, and simply not showing the whole catalog, retrieving the relevant tools first, more than tripled it to 43.13% (RAG-MCP). Distractors alone are enough to cause wrong selection (BFCL). More tools is more chances to choose wrong.
Narrow versus generalist: what actually differs?
A narrow, single-purpose agent has a small curated tool set, so selection precision is high and the chance of a confident wrong call is low, at the cost of scope. A generalist exposes a large inventory to cover more tasks, and pays for the breadth with prompt bloat, degraded selection, and a larger integration surface, more auth, more version skew, more variance per tool. The failure compounds across a multi-step plan: one wrong tool early corrupts everything after it. A generalist with a hundred tools is not a hundred times more capable than a focused one. It is markedly more likely to pick the wrong tool and execute a plausible but wrong plan with confidence (RAG-MCP).
How do you get breadth without the failure?
Stop putting the whole catalog in front of the model. Use semantic retrieval to pre-select the top handful of tools relevant to the current query and show only those, which is what tripled accuracy and halved prompt size in the RAG-MCP experiments (RAG-MCP). Organize tools hierarchically and use a router that delegates to narrow specialist agents, so each decision is made over a small, clean set. Validate selection with strict schemas and a few representative examples, and measure selection accuracy explicitly as the number of tools and domains grows, so sprawl shows up as a metric rather than a surprise. The agent keeps its breadth while every individual choice is made over a narrow set.
Where breadth bites and how to keep it
| Generalist failure | Cause | Fix |
|---|---|---|
| Wrong tool selected | Prompt bloat, blurred descriptions | Retrieve top-k relevant tools per query |
| Token cost and latency | Whole catalog in every prompt | Show only retrieved tools (>50% fewer tokens) |
| Overlapping tools | Near-synonym descriptions | Hierarchy + router to specialists |
| Unseen accuracy decay | No measurement as tools grow | Track selection accuracy vs catalog size |
| Maintenance sprawl | Many hand-kept schemas | Generate schemas from source, version them |
The pattern is that capability and selection reliability pull against each other: every tool you add is another way for the model to choose wrong, so a sprawling catalog quietly lowers accuracy. Retrieve the relevant tools per query, route through specialists, and measure selection as the catalog grows, and you get breadth without the wrong-tool tax. Making the agent decide over a small, relevant tool set at every step, rather than the whole inventory, is reliability at the pattern level, which is what VibeModel builds as the Pattern Intelligence Layer.
Frequently asked questions
So should I just build narrow agents?
You do not have to. Keep the breadth, but retrieve only the relevant tools per query and route through specialists, so each decision is made over a small set. That recovers narrow-agent accuracy with generalist scope.
How many tools is too many?
There is no fixed number, which is why you measure: track selection accuracy as the catalog grows, and add retrieval before the curve bends. RAG-MCP shows retrieval helps well before the catalog gets large.
Does retrieval add its own failure mode?
Yes, retrieval can miss the right tool, so tune recall and keep a fallback. It is still a large net gain over putting the entire catalog in front of the model.