Key facts.
- More tools, worse choices: retrieving only the relevant tools more than tripled tool-selection accuracy, from 13.62% to 43.13%, while cutting prompt tokens by over 50% (RAG-MCP, arXiv:2505.03275, 2025).
- Degradation is sharp: accuracy held up below about 30 tools and fell off steeply past roughly 100, as distractors and semantic overlap overwhelmed selection (RAG-MCP, 2025).
- The cost is double: every tool definition fills context window, so a sprawling toolset both confuses the model and inflates the token bill of every call (RAG-MCP, 2025).
Why does adding tools make the agent worse?
It reads every definition to choose; a bigger model still picks wrong at scale. (arXiv:2505.03275)
Every tool in context is a distractor the model has to eliminate. With a handful of tools, that's easy. With a hundred, many of them similar, the right choice is buried. The model can't reliably filter relevant from irrelevant, so it picks a plausible wrong one. Two things compound this: semantic overlap (several tools sound like they could do the job) and prompt bloat (tool definitions crowd out the actual task). More tools isn't more capability. Past a threshold, it's less.
How fast does it degrade?
Faster than you'd expect. RAG-MCP scaled from 1 to over 1,100 tools and measured selection accuracy at each step. Below 30 tools, the agent held up. Past about 100, accuracy dropped sharply. At the extreme end, failures dominated. The numbers: showing every tool gave 13.62% accuracy. Fetching only the relevant ones hit 43.13%, more than three times better, same tasks. A large tool registry isn't neutral. It actively limits how well your agent picks.
The fix: retrieval, not a bigger prompt
Treat tools like documents in RAG. Index their descriptions. At query time, pull only the handful relevant to the current task and show the model those, nothing else. RAG-MCP does this and more than triples selection accuracy while cutting prompt tokens by over half. Your registry can hold thousands of tools. The model only ever sees the few that matter for this request. Retrieve what's relevant. Don't stuff everything in.
How to keep the toolset sane
| Practice | What it does |
|---|---|
| Retrieve tools per query | Show the model only the few relevant tools, not the whole registry |
| Keep the per-call set small | Aim for a handful of tools in context, well under the degradation point |
| Remove near-duplicates | Merge or namespace tools that overlap, so the model isn't guessing between them |
| Group hierarchically | Pick a category first, then a tool within it, instead of one flat list |
| Name and describe clearly | Distinct names and crisp descriptions make the right tool easier to find |
| Measure selection accuracy | Track how often the agent picks the right tool as you add more |
Every tool you add makes the choice harder and the prompt heavier. Past a point, accuracy collapses under the weight of the toolbox itself. Retrieve the relevant tools per task. Keep the per-call set small. Remove overlap. A bigger model gets just as confused by a hundred near-identical tools. Find where your agent breaks, from the goal, before you build it.
Frequently asked questions
How many tools is too many?
Agents held up below about 30 tools in the RAG-MCP study and degraded sharply past roughly 100. A few dozen is a soft ceiling for what the model sees at once. Retrieve a smaller relevant subset per task.
Isn't a bigger context window the answer?
No. A bigger window lets you stuff in more tool definitions, which adds more distractors and more prompt bloat. The fix is showing fewer relevant tools, not making room for all of them.
What's the single best fix?
Tool retrieval: index your tool descriptions and fetch only the relevant ones per request. That alone more than tripled selection accuracy and halved prompt tokens in the study.