adapting-qwen3-instruction-tuned-models-0-6b-32b-tool-calling-corpora CLM-0354For adapting Qwen3 instruction-tuned models from 0.6B to 32B to tool-calling corpora, LoRA supervised fine-tuning gives the best in-distribution exact-match accuracy (best in 15 of 18 settings, never below zero-shot), while GRPO holds only a sub-one-point average edge on cross-dataset transfer and SFT followed by GRPO under a matched data budget is rarely best at either.
Capability: Using the tools it is given
Observed on
Single model family (Qwen3), three public tool-calling corpora with duplicates removed, identical prompt format, exact-match scoring, no compute-normalized comparison (GRPO uses mo.
Sources
- Measured: 6 model scales x 4 training-data settings x 3 test splits, exact match against zero-shot baselines; win counts reported (15/2/1 in-distribution, 29/19/6 on 54 transfer settings). Design controls prompt format and total training data across methods but not compute, and uses one model family; LoRA-vs-full-SFT comparison only at 4B.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For 8B multimodal models, preference tuning and plain outcome-reward GRPO lower hallucination rates while making stated confidence worse calibrated than the untuned backbone, because rank- or correctness-based rewards never score the numerical accuracy of the model's own confidence; adding a Brier-plus-group-ECE calibration term to the GRPO reward cuts calibration error sharply and leaves accuracy metrics largely unchanged.Grounding answers in the image · reviewed by AI
- For multi-hop factual QA with Qwen2.5-7B/Qwen3-8B given a deliberately insufficient retrieved triple subgraph, training a policy with RL to insert bridging triples that let a separate frozen answerer reach the gold answer raises answer accuracy over standard GRPO and over prompting methods like Self-Ask and RECITE, and the improvement carries over to direct chain-of-thought answering with no triples inserted at test time.Stating false facts confidently · reviewed by AI
- When training 4B-8B Qwen3 agents with group-based RL (GRPO, DAPO, GSPO) on sparse-reward long-horizon tasks, injecting hints into the environment's feedback — procedural action guidance in early episode steps and early training, switching to enriched state observations in later steps and later training — raises success on the unmodified environments by a few points on average, while lowering performance on tasks that require questioning whether the request is answerable at all.Completing tasks that take many steps or many hours · reviewed by AI
- For small instruction-tuned models (3-4B) answering closed-form true/false questions under a stated user opinion, scoring each sampled response by how much more common its answer is than the group itself predicted, and using that score as the GRPO reward, reduces answer flips under user pressure and raises accuracy without any labels, roughly matching label-supervised desycophancy fine-tuning at higher compute cost.Telling the user what they want to hear · reviewed by AI
- In GRPO-style RLVR on math for 7B models, injecting partial reasoning prefixes from a small out-of-family model (Gemma-2-2B, LLaMA-3.2-1B) into a fraction of rollouts keeps policy entropy higher early in training and raises pass@k, with gains growing as k rises; the effect comes from distributional unfamiliarity rather than prefix quality, since prefixes from same-family models of equal or higher quality give no gain over vanilla GRPO.Retaining skills and safety after fine-tuning · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A controlled comparison at matched data on another model family, or Qwen3 with a different reward/GRPO configuration, in which GRPO beats LoRA SFT in-distribution by a clear margin, or SFT→GRPO is consistently strongest.