agent-must-work-over-very-large-tool-catalogue-wrapping CLM-0017When an agent must work over a very large tool catalogue, wrapping each tool in a natural-language interface that resolves the API schema internally — so raw schemas are retrieved on demand rather than enumerated in the model's prompt — contributes more to multi-step tool- call success than the surrounding planner, router, or verifier stages.
Evidence for: Braess's paradox (holds)
Capability: Using the tools it is given
Observed on
Multi-agent harness with an 8B backbone, over a repository of ~25.5k functions, evaluated on ToolBench and NESTFUL nested-call tasks; not tested with strong frontier backbones insi.
Sources
- Component ablation in one paper: removing ToolFace/Tool Primitives dropped ToolBench average pass rate from 75.1 to 16.1 and NESTFUL full-sequence accuracy from 0.44 to 0.06, larger than removing the verifier (47.6 / 0.15), planner, or router. Single system, single backbone; the ablation removes retrieval and the wrapper together, so it does not separate the two.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For LLM agents under a fixed ReAct harness answering questions over executable database-backed APIs plus document retrieval, accuracy falls sharply as the required reasoning depth grows from 1 to 5 hops (most models lose over half their accuracy), and the surviving failures concentrate in language-mediated steps — entity disambiguation, cross-source grounding, schema alignment — rather than in emitting syntactically valid tool calls.Carrying a chain of inference through to its conclusion · reviewed by AI
- For locally deployed open-weight models driving a stateful, dependency-ordered MCP tool server, cutting tool descriptions from full specifications (purpose, parameter semantics, constraints, failure conditions) down to one sentence each raises the fraction of calls the server rejects for every model tested, while its effect on task coverage is less consistent.Using the tools it is given · reviewed by AI
- When tool-augmented agents emit calls over an unconstrained raw-JSON surface rather than a schema-enforcing function-calling API, every tested hosted model fabricates tool names or undeclared arguments at rates of roughly 13-57%, and parameter count does not reduce this — an 8B and a 675B model hallucinate at comparable raw-JSON rates.Stating false facts confidently · reviewed by AI
- For single-agent procedural tasks where the relevant state fits a fixed schema known in advance, replacing the appended conversation transcript with a structured execution state that the model rewrites each step — discarding the reasoning trace once the state patch is validated — holds accuracy flat as the horizon grows while history-appending runtimes degrade, and the gain comes from the structure rather than the shorter prompt.Completing tasks that take many steps or many hours · reviewed by AI
- For long-horizon tool-calling agents (tasks needing five or more calls), embedding state-transition cues — preconditions, invariants, completion states — into the descriptions of the tools on the intended chain is what actually steers the agent's trajectory; runtime corrective text appended to tool results only patches residual drift, and the plausible user prompt alone (persona, deadlines, format constraints) does not establish the trajectory at all.Following instructions hidden in data · reviewed by AI
Notes
Ingested unreviewed on 2026-09-07 and deliberately inert until a human endorses it: it does not move a technique standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Drafted confidence: low. Falsifier as drafted: An ablation on comparable benchmarks where removing the natural-language tool wrapper (exposing raw schemas plus retrieval) costs less than removing the verifier or planner, or where the wrapper gives no gain once tools are retrieved on demand. Proposed technique, not yet catalogued: Natural- language tool wrappers with on-demand schema retrieval.