Using the tools it is given

Claims

• observationsingle paper↑1↓0
  For 4B-30B Qwen3 backbones trained with self-play RL where the same model writes executable multi-turn tool environments (simulated tool schemas, backend state, per-instruction state checks) and then solves them, multi-turn function-calling scores rise, and the gain is largest on the benchmark whose task structure most closely mirrors the generated environments (ACEBench-Agent +13.9 at 30B-A3B) and smallest on the least similar one (tau2-bench +3.6).
• mechanismsingle paper↑1↓0
  For 4B-8B open base models mid-trained for tool use, grounding-oriented supervision (documentation/code turned into QA) and execution-oriented supervision (executable trajectories from real APIs and MCP skills) help different things, and only the combination improves everything: trajectories alone give the bigger function-calling gain while grounding alone transfers better to interactive and unseen-tool benchmarks, and each branch alone scores below no mid-training on MCP-Universe.
• mechanismsingle paper↑2↓0
  Giving a coding agent a raw shell is worse than giving it a designed interface: SWE-agent's constrained Agent-Computer Interface — a file viewer windowed to 100 lines, a search tool that truncates large output, and an editor that lints before a change persists — roughly doubled SWE-bench resolution over raw shell access with the same model, because verbose output and unmanaged state are where the model loses its place.
• mechanismsingle papercontested↑1↓1
  Across eight interactive environments, models fail mostly on long-horizon tool interaction, with open models far behind commercial ones.
• observationsingle paper↑1↓0
  For adapting Qwen3 instruction-tuned models from 0.6B to 32B to tool-calling corpora, LoRA supervised fine-tuning gives the best in-distribution exact-match accuracy (best in 15 of 18 settings, never below zero-shot), while GRPO holds only a sub-one-point average edge on cross-dataset transfer and SFT followed by GRPO under a matched data budget is rarely best at either.
• observationsingle paper↑1↓0
  With the agent loop, skills and prompts held fixed and only the model swapped, models split into two clusters on operating a multi-step asynchronous ML workflow: Claude Sonnet 3.5-4.0, Gemini 2.5 and GPT-5 completed tasks at 93-100%, while GPT-4 scored 13% and others could not run the loop at all, hallucinating workflow ids or failing to wait for asynchronous jobs; on open-ended exploration the effect of a stressful, competitive prompt reversed by family, with Sonnet 4.0 producing the largest improvement of any configuration under stress and GPT-5 turning conservative and giving back most of its basic-prompt gains.
• mechanismsingle paper↑1↓0
  When an agent must work over a very large tool catalogue, wrapping each tool in a natural-language interface that resolves the API schema internally — so raw schemas are retrieved on demand rather than enumerated in the model's prompt — contributes more to multi-step tool- call success than the surrounding planner, router, or verifier stages.
• mechanismsingle paper↑1↓0
  When a calculator-using model is trained with reinforcement learning on verifiable final-answer rewards for an arithmetic search task (Countdown), the gain lands almost entirely at low k — pass@1 rises sharply while pass@16 moves little or falls — because the update can only reinforce correct trajectories the starting policy already samples, and prompt groups with no correct sample supply no gradient.
• observationsingle paper↑1↓0
  On cross-system urban tasks where several services must be chained under matched tools and budgets, the task-success gap between orchestration strategies is not explained by required-tool coverage, call ordering, or asking for missing inputs — those are near-saturated across strategies — but by whether the agent goes beyond the minimal toolchain to refine coarse results and verify each candidate against the stated constraints before answering.
• mechanismsingle paper↑1↓0
  Fine-tuning on a large corpus of real API-call traces markedly improves multi-tool task completion in open models.
• mechanismsingle paper↑1↓0
  When fine-tuning small open models (4B-8B Qwen3) for multi-turn tool calling, supervising only the recovery steps generated after the first point where the student's rollout leaves the teacher-supported state region — and zeroing the loss on the student's own valid prefix — beats full-trajectory imitation and outcome-reward RL on in-domain and held-out tool benchmarks, and the trained student's successful trajectories get shorter across iterations rather than longer.
• observationsingle paper↑1↓0
  In a full-duplex speech-to-speech model that emits function calls on a dedicated parallel output stream, choosing the right tool from spoken, disfluent user requests is far easier than filling its arguments: tool-selection F1 reached 82.5% (above two closed realtime APIs) while argument accuracy was 42.2% and end-to-end Pass@1 33.0%, below both.
• observationsingle paper↑1↓0
  When instruction-tuned models (Qwen2.5 7B-32B) make function calls from non-English user requests, they often select the right tool and argument names but emit argument values in the wrong language, a failure that exact-match or AST tool-calling metrics record as a generic invocation error rather than a language mismatch.
• observationsingle paper↑1↓0
  A linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.
• observationsingle paper↑1↓0
  A linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.
• observationsingle paper↑1↓0
  For LLM agents acting through stateful tools over many turns, unsafe policy-violating completions accumulate with the interaction budget and are roughly doubled when the adversarial user conditions each next request on the agent's actual prior response, so single-turn or short-prefix safety evaluation understates the failure rate.
• observationsingle paper↑1↓0
  For LLM agents doing active clinical diagnosis in interactive patient environments, wrapping the model in a scaffold of role-specialized agents, evidence tools and an explicit evidence state, with a runtime observer that tracks uncertainty and evidence coverage to decide when to stop acquiring evidence, raises both diagnostic accuracy and execution stability over static case-to-answer prompting, with the size of the gain depending on the backbone model.
• mechanismsingle paper↑1↓0
  For LLM web agents on cluttered pages containing many concurrent interface components, success falls sharply as component count grows, and having the agent first emit a localized sub-goal plus a task-relevant sub-observation before choosing an action recovers much of the loss, because self-assessment of the action is more reliable in the reduced context.
• observationsingle paper↑1↓0
  For locally deployed open-weight models driving a stateful, dependency-ordered MCP tool server, cutting tool descriptions from full specifications (purpose, parameter semantics, constraints, failure conditions) down to one sentence each raises the fraction of calls the server rejects for every model tested, while its effect on task coverage is less consistent.
• mechanismsingle paper↑1↓0
  For long-horizon LLM agents on tool-use and terminal benchmarks, token-level confidence (predictive entropy, sequence probability) barely separates successful from failed trajectories, and the separable signal lives in the dependency structure of the trajectory: scoring a trajectory by propagating per-step uncertainty over a graph of reasoning, tool-call and environment-feedback nodes raises failure-detection AUROC over both local-confidence and sequence-based agent UQ baselines.
• mechanismsingle paper↑1↓0
  For multi-turn tool-using service agents, counting excess user-facing turns above a task-specific human reference predicts human efficiency preferences between two successful trajectories (about 91% accuracy on turn-differing pairs), while counting excess tool calls does not (chance on pure-tool pairs), because a raw call count cannot say whether a call verified a constraint or wasted effort.
• mechanismsingle paper↑1↓0
  For open models (8B-14B) filling structured tool-call parameters, a linear probe on the hidden state at the position just before a parameter value is written predicts whether that value will be correct better than token log-probability (in-domain AUC about 0.986 vs 0.914), and using that score to filter self-generated calls for fine-tuning beats self-training filtered by log-probability or majority vote.
• mechanismsingle paper↑1↓0
  Self-supervised training on tool-call examples teaches a model when and how to call simple tools, narrowing the gap on tasks that need them.
• mechanismsingle paper↑1↓0
  Shows LLMs hallucinate API names and arguments when calling large API collections from memory, and that retrieving documentation reduces this.
• mechanismsingle paper↑1↓0
  In simulated retail and airline domains with tools and policy documents, frontier models succeeded on well under two thirds of tasks and success dropped sharply when the same task had to be passed several times in a row.
• observationsingle paper↑1↓0
  For single-step tool-calling agents, near-ceiling accuracy on unperturbed calls does not predict robustness to perturbation, and corrupting or removing the tool's returned evidence hurts far more than perturbing tool names, descriptions, or the user request.
• observationsingle paper↑1↓0
  For single-step tool-calling agents, near-perfect clean accuracy does not predict robustness, and the dominant weak point is recovering the answer from perturbed tool returns rather than selecting the tool or binding arguments: across seven models, clean success averaged 0.979 while success under corrupted or missing returned evidence averaged 0.455, with no model above 0.60.
• mechanismsingle paper↑1↓0
  For small (4-8B) student agents on multi-turn tool-use tasks, transferring a stronger teacher's successful trajectories as flat task-level memory barely helps and can hurt, while splitting the same trajectories into a high-level workflow insight in prose plus concrete executable subtask examples raises accuracy sharply, with the subtask examples carrying most of the gain.
• observationsingle paper↑1↓0
  On a 30-task ladder spanning instruction following, tool use, multi-step coordination and long-horizon planning, evaluated over 16 open-weight models from 0.27B to 32B against GPT-5 across 16,542 scored runs, small and mid-sized open models are already sufficient for the short-horizon structured tool use that makes up most calls in real agent pipelines, and the strongest open-weight model matches GPT-5 in aggregate — but the gap reappears on long-horizon planning that needs sustained coordination and constraint tracking, where frontier models still lead and neither side is reliable, and the boundary is not explained by model scale alone.
• observationsingle paper↑1↓0
  For small open-weight models (0.6B-12B) on BFCL V3, presenting tool definitions as ordinary system-prompt instructions and having the model emit calls as normal assistant text yields higher function-calling accuracy than the model's native tool-call token/schema interface, with the largest gains in the smallest models and on parallel/multiple-call categories.
• observationsingle paper↑1↓0
  For sub-2B open-weight models emitting JSON tool calls from prompts that already name the tool and all argument values, reported success depends almost entirely on a post-hoc recovery layer that strips code fences and extracts a brace-delimited substring: across 1,000 generations only 5 whole raw responses parsed as JSON directly, while the best condition reached 79% after recovery.
• observationmechanism reasoning↑1↓0
  Worked examples in a tool description, long the standard way to teach tool use, can constrain a capable model to the exploration space the examples describe; Anthropic now advises designing expressive parameters instead — a status enumeration that implies its own usage rather than a demonstration of it.
• observationsingle paper↑1↓0
  The large reported gains from fine-tuning on tool-call trajectories come from small open models starting far behind — distilling a few hundred trajectories from a stronger model moves them a long way. Whether the same procedure helps a model that is already good at tool use is a separate question this evidence does not answer.
• observationsingle paper↑1↓0
  In simulated retail customer-support tasks with real tools and policy documents, GPT-4o — the best performer in the study — still completed well under two-thirds of tasks correctly, and its rate of passing the identical task eight times in a row (a proxy for production reliability) was far lower than its single-attempt success rate.
• mechanismsingle paper↑1↓0
  When a tool-using agent is trained with outcome-only RL and a surface prompt cue co-occurs with one tool in most training examples, the agent invokes that tool on tasks where it is useless, but only for tools it has already learned to use well and mainly when the cue is semantically related to the tool.
• observationsingle paper↑1↓0
  Used zero-shot as step-level action critics in long-horizon tool-calling tasks, frontier models are over-pessimistic — flagging a large share of correct actions and pushing the actor into revision loops that lower success below no critic at all — whereas small models fine-tuned on action-level verification rationales flag far fewer valid actions and raise repeated-run reliability.

Techniques

• Augment and constraint-check each candidate before committinprocess
  After an incremental reason-act-observe loop, the agent refines coarse matches, retries failed or geo-inconsistent calls, and a synthesis step checks the draft against task constraints and admits no fact absent from logged observations.
• Bound what the agent may do, not just what it may notprocess
  An explicit allow/ask/deny policy plus rate, cost and time ceilings, enforced by the harness rather than requested of the model.
• Decompose the observation into a sub-goal and sub-view beforprocess
  At inference, the agent splits each complex web observation into an aligned sub-goal and task-relevant sub-observation, self-scores the decomposition and action, and uses the passed/rejected trials as in-context demonstrations or as RL sign
• Distill teacher trajectories into workflow, subtask and funcretrieval
  Builds three memory banks from a large teacher's successful trajectories — a prose workflow insight, code-bearing subtask segments, and per-function call examples — injecting the first two into the student's system prompt and retrieving the
• Emit tool calls on a parallel function-token streamarchitecture
  Adds a dedicated autoregressive function channel, with its own head and start/end tool-call boundary tokens supervised alongside padding, running in parallel with the agent-text stream so tool calls are not serialized into the spoken respon
• Filter self-training data with a hidden-state parameter probtraining
  Train a per-layer logistic probe on the hidden state just before each parameter value is emitted, then use its score to keep only reliable self-generated tool calls for LoRA fine-tuning and to rerank sampled candidates at inference.
• Fine-tune on tool-call tracestraining
  Train the model on examples of deciding when to call a tool and how to use the result.
• hidden-state probe for tool-call errorsprocess
  hidden-state probe for tool-call errors, as introduced by the paper it was drafted from.
• Mid-train tool use on both grounded docs and executable trajtraining
  Builds a mid-training corpus that pairs filtered web/PDF/code sources augmented into tool-affordance QA with validated multi-turn trajectories synthesized from real API and MCP schemas, and trains the base model on the mixture before ordina
• Propagate uncertainty over a trajectory dependency graphprocess
  Represent an agent's execution history as a directed graph of reasoning, tool-call and environment nodes with typed dependency edges (sequential, repetition, progression, feedback, goal alignment), then propagate each node's local uncertain
• Retrieve tool documentation at call timeretrieval
  Put the exact documentation for candidate tools into the prompt instead of relying on memory.
• Reward each tool call by whether it was necessarytraining
  During RL, an LLM judge labels each tool invocation as necessary or not given the question and tool input, and unnecessary calls get a fixed per-step penalty added to the final-answer reward.
• Route routine calls to a smaller modelprocess
  Send the short, structured, routine calls in an agent pipeline to a small cheap model and reserve a frontier model for the steps that need deep planning or sustained constraint tracking.
• Scaffold diagnosis with evidence tools and a coverage-based process
  Builds patient-specific interactive environments, routes diagnosis through role-specialized agents querying evidence tools against an evolving evidence state, and uses an observer that tracks hypothesis uncertainty and evidence coverage to
• Score argument-value language consistency as its own metricprocess
  Decompose tool-call evaluation into a conditional hierarchy — invocation detection, tool selection, argument completion, argument language consistency, full call match — so wrong-language argument values are separated from structural errors
• Score successful trajectories by excess turns and tool callsprocess
  A success-gated metric multiplicatively discounts a completed trajectory for tool calls and user-facing turns beyond per-task human reference floors, with the two penalty weights fitted from blinded human pairwise efficiency preferences via
• Score whole agent trajectories on five safety outcomesprocess
  Run adaptive adversarial and matched benign user agents against a stateful tool environment and label each complete trajectory Safe Completion, Correct Refusal, Unsafe Completion, Over-Refusal, or Indeterminate using executed tool actions a
• Stage-aligned perturbation with cascade-aware failure attribprocess
  Perturb each stage of the tool-call pipeline separately (interface, intent, returned evidence, runtime feedback) and backtrack each observed failure deterministically to the earliest stage that caused it.

Sources

Every paper, post or observation behind this capability. Ones cited by a claim show the claim and its stance; ones filed directly are about the capability but have no claim drafted from them yet. Open one to add support or contest it.

• NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities2026-09-18
  ↑ supports a claim here
• RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents2026-09-16
  ↑ supports a claim here
• SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale2026-09-15
  ↑ supports a claim here
• What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity2026-09-15
  about this capability · no claim drafted yet
• BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents2026-09-14
  ↑ supports a claim here
• Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act2026-09-14
  ↑ supports a claim here
• Agentic ML Exploration (A-MLE) for Ads Ranking2026-09-08
  ↑ supports a claim here
• Beyond Fluent Generation: A CPU Reliability Benchmark for MCP-Style Tool Calling in Sub-2B Small Language Models for Edge Deployment2026-09-07
  ↑ supports a claim here
• Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives2026-09-01
  ↑ supports a claim here
• Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling2026-09-01
  about this capability · no claim drafted yet
• CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents2026-08-31
  ↑ supports a claim here
• Learning Simple Test-Time Environments for LLM Web Agents2026-08-29
  ↑ supports a claim here
• Learning to Use Tools: Reinforcement Learning for Tool-Integrated Mathematical Reasoning2026-08-28
  ↑ supports a claim here
• The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs2026-08-27
  ↑ supports a claim here · ↑ supports a claim here
• EviDx: Evidence-Aware Active Diagnosis with Scaffolded LLM Agents2026-08-25
  ↑ supports a claim here
• Benchmarking AI Agents for Hardware Design Automation via MCP Tool Calling2026-08-25
  ↑ supports a claim here
• ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents2026-08-23
  ↑ supports a claim here · ↑ supports a claim here
• Small Reasoning Models are Instruction Followers in Function Calling2026-08-23
  ↑ supports a claim here
• MidTool: Mid-training Data Synthesis for Agentic Tool Use2026-08-20
  ↑ supports a claim here
• SPADE: Self-Play in Adaptive Synthetic Executable Environments2026-08-19
  ↑ supports a claim here
• DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents2026-08-19
  ↑ supports a claim here
• From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents2026-08-17
  ↑ supports a claim here
• When the API Speaks the Wrong Language: Revisiting Post-Training for Multilingual Tool Use2026-08-12
  ↑ supports a claim here
• Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory2026-08-07
  ↑ supports a claim here
• UrbanAgent: A Tool-Augmented Agent for Cross-System Urban Tasks2026-08-04
  ↑ supports a claim here
• Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls2026-08-04
  ↑ supports a claim here
• AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go?2026-05-01
  ↓ contests a claim here · ↑ supports a claim here
• I Improved 15 LLMs at Coding in One Afternoon. Only the Harness Changed.2026-02-12 · post
  ↑ supports a claim here
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering2024-05-06
  ↑ supports a claim here
• FireAct: Toward Language Agent Fine-tuningThis source: FireAct: Toward Language Agent Fine-tuningHow much the field cites it — heavily cited in the last 12 months101 in the last 12 months · 252 totalpublished 2023checked 2026-09-04click for Semantic Scholar2023-10-09
  ↑ supports a claim here
• InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback2023-06-26
  about this capability · no claim drafted yet
• ReAct: Synergizing Reasoning and Acting in Language Models2022-10-06
  about this capability · no claim drafted yet
• AgentBench: Evaluating LLMs as AgentsThis source: AgentBench: Evaluating LLMs as AgentsHow much the field cites it — very heavily cited in the last 12 months797 in the last 12 months · 1243 totalpublished 2023checked 2026-09-042023
  ↑ supports a claim here
• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsThis source: ToolLLM: Facilitating Large Language Models to Master 16000+ Real-worl…How much the field cites it — very heavily cited in the last 12 months1000+ in the last 12 months · 2160 totalpublished 2023checked 2026-09-04count capped at 1000 by the fetch2023
  ↑ supports a claim here
• Toolformer: Language Models Can Teach Themselves to Use ToolsThis source: Toolformer: Language Models Can Teach Themselves to Use ToolsHow much the field cites it — very heavily cited in the last 12 months1000+ in the last 12 months · 5422 totalpublished 2023checked 2026-09-04count capped at 1000 by the fetchclick for Semantic Scholar2023
  ↑ supports a claim here
• Gorilla: Large Language Model Connected with Massive APIsThis source: Gorilla: Large Language Model Connected with Massive APIsHow much the field cites it — very heavily cited in the last 12 months805 in the last 12 months · 1585 totalpublished 2023checked 2026-09-042023
  ↑ supports a claim here
• τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World DomainsThis source: τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Dom…How much the field cites it — very heavily cited in the last 12 months889 in the last 12 months · 1075 totalpublished 2024checked 2026-09-042024
  ↑ supports a claim here · ↑ supports a claim here
• The new rules of context engineering for Claude 5 generation models2026 · vendor-doc
  ↑ supports a claim here