The Toolbox Is the Problem, Not the Model
We kept adding tools because each one was individually justified. Somewhere past thirty the agent began choosing plausibly wrong ones, and no amount of prompt work fixed it. ToolLLM describes the scale we were trying to reach; our answer was to stop trying.
How We Got to Forty
Nobody decides to give an agent forty tools. It accumulates. Each request is reasonable in isolation: the agent should also be able to look up a contract, also check delivery status, also open a ticket. Every addition was justified by a real user need.
The failure did not arrive as an error. It arrived as a slow rise in answers that were confidently derived from the wrong source — the contract lookup used where the order lookup was meant, returning a real value for the wrong question.
Why Prompting Did Not Fix It
We did what everyone does first: better tool descriptions, explicit disambiguation rules, examples in the system prompt. Each round improved a specific confusion and left the general one intact, because the problem is not that any single description is unclear.
The problem is that selection quality degrades as the option set grows and options overlap. Two tools that both plausibly answer a question will be confused some of the time, and the number of such pairs grows faster than the tool count.
What the Research Direction Says
Qin and colleagues went the other way, and instructively so. ToolLLM builds an instruction-tuning framework aimed at mastering large numbers of real-world REST APIs, with data collected across a very wide API surface.
That is a legitimate and interesting target. It is also a statement about where the difficulty lives: if teaching a model to handle thousands of APIs requires dedicated data collection and tuning, then handing an untuned model forty overlapping ones and hoping is not a plan.
What We Changed
We stopped scoping the toolbox by system and started scoping it by task. The agent handling delivery questions gets the nine tools a delivery question can need. The agent handling contracts gets its own set. Neither sees the other's.
Where a request spans both, a router decides which task it is before any tool is visible. The router is a small classifier with a fixed output set, not an agent — it has one job and it can be tested exhaustively.
| Scoping by system | Scoping by task |
|---|---|
| Every tool the platform has | Only what this task can need |
| Grows with the platform | Grows with the task, which is bounded |
| Overlap resolved by prompt | Overlap removed by construction |
| Wrong tool returns a real value | Wrong tool is not reachable |
What It Costs
A routing step adds a call and a failure mode of its own: route wrongly and the right tool is unreachable. We accept that trade because a routing error is visible and testable, while a tool-selection error inside a large toolbox looks like a plausible answer.
The maintenance cost is real too. Task scopes have to be revisited when the product changes, and a tool that becomes relevant to a second task has to be added deliberately rather than being present by default.
What We Do Not Claim
We do not claim nine is a magic number. It is what that task needed. The number that matters is not the count but the overlap: two tools that answer the same question are worse than ten that answer ten questions.
We also do not claim large toolboxes cannot work. With tuning aimed at exactly that, as the research shows, they can. We claim that most products do not have that tuning and should not design as if they did.