tool-use · activeUsing the tools it is given
Models often fail to reach for provided tools, skills, or documented procedures, or call them with wrong arguments.
Also called: tool calling, function calling, API use, skill utilization
Tags: agentic, autonomous-agent, coding-agent, customer-support
Given a set of tools, skills, or API documentation, a capable model selects the right one for the task, calls it with correct arguments, interprets the result, and chains calls when needed. It prefers a provided tool over improvising from memory, and it asks or stops when no tool fits.
Claims
- mechanismsingle paperGiving a coding agent a raw shell is worse than giving it a designed interface: SWE-agent's constrained Agent-Computer Interface — a file viewer windowed to 100 lines, a search tool that truncates large output, and an editor that lints before a change persists — roughly doubled SWE-bench resolution over raw shell access with the same model, because verbose output and unmanaged state are where the model loses its place.
- mechanismsingle paperAcross eight interactive environments, models fail mostly on long-horizon tool interaction, with open models far behind commercial ones.
- mechanismsingle paperWhen an agent must work over a very large tool catalogue, wrapping each tool in a natural-language interface that resolves the API schema internally — so raw schemas are retrieved on demand rather than enumerated in the model's prompt — contributes more to multi-step tool- call success than the surrounding planner, router, or verifier stages.
- mechanismsingle paperWhen a calculator-using model is trained with reinforcement learning on verifiable final-answer rewards for an arithmetic search task (Countdown), the gain lands almost entirely at low k — pass@1 rises sharply while pass@16 moves little or falls — because the update can only reinforce correct trajectories the starting policy already samples, and prompt groups with no correct sample supply no gradient.
- mechanismsingle paperFine-tuning on a large corpus of real API-call traces markedly improves multi-tool task completion in open models.
- observationsingle paperA linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.
- observationsingle paperFor locally deployed open-weight models driving a stateful, dependency-ordered MCP tool server, cutting tool descriptions from full specifications (purpose, parameter semantics, constraints, failure conditions) down to one sentence each raises the fraction of calls the server rejects for every model tested, while its effect on task coverage is less consistent.
- mechanismsingle paperSelf-supervised training on tool-call examples teaches a model when and how to call simple tools, narrowing the gap on tasks that need them.
- mechanismsingle paperShows LLMs hallucinate API names and arguments when calling large API collections from memory, and that retrieving documentation reduces this.
- mechanismsingle paperIn simulated retail and airline domains with tools and policy documents, frontier models succeeded on well under two thirds of tasks and success dropped sharply when the same task had to be passed several times in a row.
- observationmechanism reasoningWorked examples in a tool description, long the standard way to teach tool use, can constrain a capable model to the exploration space the examples describe; Anthropic now advises designing expressive parameters instead — a status enumeration that implies its own usage rather than a demonstration of it.
- observationsingle paperThe large reported gains from fine-tuning on tool-call trajectories come from small open models starting far behind — distilling a few hundred trajectories from a stronger model moves them a long way. Whether the same procedure helps a model that is already good at tool use is a separate question this evidence does not answer.
- observationsingle paperIn simulated retail customer-support tasks with real tools and policy documents, GPT-4o — the best performer in the study — still completed well under two-thirds of tasks correctly, and its rate of passing the identical task eight times in a row (a proxy for production reliability) was far lower than its single-attempt success rate.
- observationsingle paperUsed zero-shot as step-level action critics in long-horizon tool-calling tasks, frontier models are over-pessimistic — flagging a large share of correct actions and pushing the actor into revision loops that lower success below no critic at all — whereas small models fine-tuned on action-level verification rationales flag far fewer valid actions and raise repeated-run reliability.
Techniques
- Bound what the agent may do, not just what it may notprocessAn explicit allow/ask/deny policy plus rate, cost and time ceilings, enforced by the harness rather than requested of the model.
- Fine-tune on tool-call tracestrainingTrain the model on examples of deciding when to call a tool and how to use the result.
- Retrieve tool documentation at call timeretrievalPut the exact documentation for candidate tools into the prompt instead of relying on memory.
Capabilities are a way of carving up the subject, and carvings are arguable. Say so if this one is wrong — especially a proposed one, which a pipeline added because several papers used the same framing, not because anyone decided it was right.