error-injected-into-supplied-chain-thought-models-mostly-carry CLM-0298observationsingle paperReviewed by AI
When an error is injected into a supplied chain of thought, models mostly carry it to the answer: a planted arithmetic error costs small models 50-60% accuracy and 18% on average, though the largest models lose only 3-7%; unit-conversion errors cost about 9% at every scale; skipped steps and sycophantic steps cost small models about 10%; extra correct steps cost almost nothing (0-6%).
Capability: Carrying a chain of inference through to its conclusion
Observed on
13 models over three orders of magnitude, mathematical reasoning tasks, five perturbation types injected into the chain, 2026..
Sources
- Perturbations are injected into a given chain rather than arising from the model's own reasoning; the scaling pattern by error type is the informative part.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Instruction tuning that forces an answer to every question teaches models to fabricate when the question lies outside their parametric knowledge; splitting the tuning data by whether the base model already answers correctly and appending "I am unsure" to the rest (refusal-aware tuning) raises average precision on known-versus-unknown separation over vanilla tuning, transfers to out-of-domain datasets, and yields better calibrated confidence than post-hoc uncertainty scoring.Saying it does not know · reviewed by AI
- In closed-book multi-hop QA the accuracy gain from decomposed prompting shrinks to little in frontier models, but answers still change between task-equivalent prompt regimes (agreement only about 70%, and about 60% on MuSiQue), and because stable facts survive re-prompting while hallucinations do not, disagreement across regimes detects errors better than standard uncertainty baselines on F1 and AUROC without retrieval or training.Recalling facts it was trained on · reviewed by AI
- Verifying a chain of thought by decomposing it into nodes and asking binary consistency questions per node localizes errors better than judging the whole chain at once (F1 up 10-25% over CoT-based verifiers, error-localization accuracy far above an 8-vote end-to-end judge) while using 6-58 times fewer tokens, because whole-chain judging dilutes attention and multi-sampling pays for long generations.Carrying a chain of inference through to its conclusion · reviewed by AI
- An abstention rule built from self-consistency, where the model samples several answers and itself scores how many agree with a candidate, can be thresholded by conformal prediction to bound the hallucination rate at a chosen level on closed-book QA; against log-probability confidence it abstains much less often on long-form answers (Temporal Sequences) for the same error bound and matches it on short answers (TriviaQA).Saying it does not know · reviewed by AI
- For image-borne injected instructions delivered to tool-using agents, completion rate understates exposure: agents act on the injected instruction about ten times more often than they complete the objective (12.8% attempted vs ~1% completed across 720 runs), and most failures stop at the planning step where the model read the payload and simply did not act on it rather than at any architectural barrier.Following instructions hidden in data · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Models that detect and correct injected math errors at small scale, or unit-conversion robustness improving with scale.