locally-deployed-open-weight-models-driving-stateful-dependency-ordered-mcp-tool CLM-0080For locally deployed open-weight models driving a stateful, dependency-ordered MCP tool server, cutting tool descriptions from full specifications (purpose, parameter semantics, constraints, failure conditions) down to one sentence each raises the fraction of calls the server rejects for every model tested, while its effect on task coverage is less consistent.
Capability: Using the tools it is given
Observed on
Seven 4-bit quantised open models (8B-31B) run through Ollama on an expert-informed hardware-design benchmark of 14 tools; single-agent ReAct loop with other configuration held fix.
Applies when
- Effect on the capability:
- helps
- Fails when:
- Tool descriptions are cut to one sentence: the fraction of calls the server rejects rises for every model tested, while task coverage moves less consistently.
Sources
- Measured as tool failure rate under matched configurations; the paper reports TFR rises for every model and roughly doubles for most, and comprehensive descriptions win in 35 of 42 model-suite best configurations. No absolute per-model numbers given for the ablation, and only one tool set of 14 tools on a proprietary replica server.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- A linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.Using the tools it is given · reviewed by AI
- For small open-weight models (0.6B-12B) on BFCL V3, presenting tool definitions as ordinary system-prompt instructions and having the model emit calls as normal assistant text yields higher function-calling accuracy than the model's native tool-call token/schema interface, with the largest gains in the smallest models and on parallel/multiple-call categories.Using the tools it is given · reviewed by AI
- A linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.Using the tools it is given · reviewed by AI
- On cross-system urban tasks where several services must be chained under matched tools and budgets, the task-success gap between orchestration strategies is not explained by required-tool coverage, call ordering, or asking for missing inputs — those are near-saturated across strategies — but by whether the agent goes beyond the minimal toolchain to refine coarse results and verify each candidate against the stated constraints before answering.Using the tools it is given · reviewed by AI
- For single-step tool-calling agents, near-ceiling accuracy on unperturbed calls does not predict robustness to perturbation, and corrupting or removing the tool's returned evidence hurts far more than perturbing tool names, descriptions, or the user request.Using the tools it is given · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An ablation on comparable stateful tool benchmarks where minimal one-sentence descriptions leave the tool failure rate unchanged or lower than comprehensive descriptions across models. Drafted stance toward shows-llms-hallucinate-api-names-arguments-when-calling: supports -- Both find that giving the model detailed tool documentation reduces malformed or invalid calls.