tool-use-fine-tuning-closes-a-gap-rather-than-extending-a-frontier CLM-0170observationsingle paperReviewed by AI and Russ Miller
The large reported gains from fine-tuning on tool-call trajectories come from small open models starting far behind — distilling a few hundred trajectories from a stronger model moves them a long way. Whether the same procedure helps a model that is already good at tool use is a separate question this evidence does not answer.
Capability: Using the tools it is given · Agentic, Coding agent
Observed on
2023, Llama2-7B fine-tuned on GPT-4 trajectories.
Model cards: GPT-4
Applies when
- Effect on the capability:
- helps
- Needs:
- access to train the weights
- Helps most:
- weaker models
- Cost:
- high
- Fails when:
- The model is already good at tool use. The evidence comes from small open models starting far behind and does not cover that case.
Sources
- Fine-tuning Llama2-7B on 500 GPT-4-generated agent trajectories gave a 77% HotpotQA improvement, and mixing trajectories from multiple tasks and prompting methods improved agents further.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When fine-tuning small open models (4B-8B Qwen3) for multi-turn tool calling, supervising only the recovery steps generated after the first point where the student's rollout leaves the teacher-supported state region — and zeroing the loss on the student's own valid prefix — beats full-trajectory imitation and outcome-reward RL on in-domain and held-out tool benchmarks, and the trained student's successful trajectories get shorter across iterations rather than longer.Using the tools it is given · reviewed by AI
- For single-step tool-calling agents, near-perfect clean accuracy does not predict robustness, and the dominant weak point is recovering the answer from perturbed tool returns rather than selecting the tool or binding arguments: across seven models, clean success averaged 0.979 while success under corrupted or missing returned evidence averaged 0.455, with no model above 0.60.Using the tools it is given · reviewed by AI
- With weights frozen and no stronger model in the loop, an agent can raise its own pass rate by rewriting the scaffolding it runs inside — clustering its failures, proposing minimal edits to prompts, tools and policy, and keeping only edits that improve one split without degrading another: all 9 of 9 model-by-benchmark pairs improved, by up to 40.6 percentage points (GLM-5 on AppWorld, 44.4% to 85.0%), with weak models gaining most; but the promotion gate reads the held-out split, so the reported held-out gain is not clean generalisation, and there is no matched-compute baseline.Fixing its own mistakes · reviewed by AI
- When 7-8B open-weight instruct models are fully fine-tuned on a single domain dataset, confining gradient updates to FFN neurons that activate stably across the training set and adding an MSE loss that pulls activations back toward pre-computed activation anchors retains far more general-benchmark ability (MMLU, BBH, TyDiQA) than full-parameter fine-tuning, by about 9.9 points on average, while also raising in-domain scores.Retaining skills and safety after fine-tuning · reviewed by AI
- For single-step tool-calling agents, near-ceiling accuracy on unperturbed calls does not predict robustness to perturbation, and corrupting or removing the tool's returned evidence hurts far more than perturbing tool names, descriptions, or the user request.Using the tools it is given · reviewed by AI
Notes
Deliberately stated as a scope limit rather than as "gains are largest for weak models," which would be a comparative the source does not establish — it reports a large gain for a weak model, which is not the same as measuring the gain shrinking as the baseline rises. If a paper ablates starting strength directly, this claim should be tightened or replaced.