used-zero-shot-step-level-action-critics-long-horizon-tool-calling-tasks-frontier CLM-0185Used zero-shot as step-level action critics in long-horizon tool-calling tasks, frontier models are over-pessimistic — flagging a large share of correct actions and pushing the actor into revision loops that lower success below no critic at all — whereas small models fine-tuned on action-level verification rationales flag far fewer valid actions and raise repeated-run reliability.
Capability: Using the tools it is given
Observed on
tau-Bench/tau-Trait style multi-turn tool-calling domains (retail, airline, telecom, telehealth); actor fixed as Qwen3-32B; critics were GPT-4.1, Qwen3-235B, Qwen2.5-72B prompted, .
Model cards: GPT-4
Sources
- Measured on four tool-calling domains with pass^1/pass^3/pass^4; reported GPT-4.1 critic flagging 46.8% of correct actions versus 13.6% and 11.4% for the trained 4B/8B critics, and average pass^4 falling from 11.5% (no critique) to 6.0% with GPT-4.1 critique while rising to 15.9%/19.5% with trained critics. Single paper, one model family for the trained critics, small out-of-domain task counts (18-20 tasks in telecom/telehealth), and prompting for the frontier critics was not tuned.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For a frozen LLM run in a generate-critique-revise loop with no external feedback, the accuracy gain is concentrated where single-shot reasoning is already unreliable: on clean multi-step reasoning (BBH, ~74% single-shot) depth makes no difference beyond the confidence interval, while on harder math (MATH, 26.2% single-shot) the same loop gains +14.2 points, with GSM8K in between (+4.2).Fixing its own mistakes · reviewed by AI
- General-purpose factuality verifiers (entailment-, consistency-, and RAG-trained) and zero-shot frontier LLM judges perform near chance at spotting unsupported claims in scientific peer reviews that must be checked against the full submitted paper, because they cannot separate paper-unsupported assertions from legitimate evaluative critique; small models fine-tuned on in-domain examples beat them by a wide margin.Stating false facts confidently · reviewed by AI
- General-purpose factuality verifiers (entailment-, consistency-, and RAG-trained) and zero-shot frontier LLM judges perform near chance at spotting unsupported claims in scientific peer reviews that must be checked against the full submitted paper, because they cannot separate paper-unsupported assertions from legitimate evaluative critique; small models fine-tuned on in-domain examples beat them by a wide margin.Stating false facts confidently · reviewed by AI
- For long-horizon repository-level coding agents, a small (4B) critic trained to diagnose each just-completed step with a severity label, error type and causal rationale, and to redirect the next action when severity is critical, repairs more initially-failing tasks with a single branch (14.1% fail-to-pass) than five full resamples of the base agent (13.7%), at about one-fifth the base-agent output tokens.Judging another's work as well as a careful reviewer would · reviewed by AI
- When training small instruction-tuned models with reinforcement learning plus factual supervision, routing each atomic fact's verification score only to the tokens that produced it, and down-weighting verifier judgements that do not change when their key evidence is removed, improves factuality benchmark scores over trajectory-level or reasoning- step-level factual rewards.Checking claims against evidence · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A study where prompted frontier-model critics improve pass^k over no-critique baselines on the same benchmarks, or where their false-flag rate on valid actions matches that of small critics trained on verification rationales. Drafted stance toward self-critique-without-grounding-unreliable: supports -- Prompted frontier critics without training degraded actor performance relative to running with no critique agent, consistent with unaided critique being an unreliable improvement signal. Proposed technique, not catalogued: critique-aware supervision from action-level verification rationales.