arxiv-2603-03332 SRC-0348 · paper

Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations

Created: 2026-02-11 · Ingested: 2026-09-19

Chain-of-Thought (CoT) prompting has emerged as a foundational technique for eliciting reasoning from Large Language Models (LLMs), yet the robustness of this approach to corruptions in intermediate reasoning steps remains poorly understood. This paper presents a comprehensive empirical evaluation of LLM robustness to a structured taxonomy of 5 CoT perturbation types: \textit{MathError, UnitConversion, Sycophancy, SkippedSteps,} and \textit{ExtraSteps}. We evaluate 13 models spanning three orders of magnitude in parameter count, testing their ability to complete mathematical reasoning tasks de

In brief

Corrupting an intermediate step of a chain of thought does not degrade models uniformly, and scale protects against some corruption types and barely at all against others. Injected arithmetic errors are the worst case; redundant filler text is nearly free.

The setup is partial-trace completion on GSM8K: given a question and the first few solution steps, the model finishes the chain. 5 perturbation types (MathError, UnitConversion, Sycophancy, SkippedSteps, ExtraSteps) were injected into the partial trace, and 13 models from 3B to 685B, plus closed Anthropic/OpenAI/Google models, were tested on 100 randomly selected problems per perturbation type against a clean-trace control.

MathError cost 18% accuracy on average, with Ministral 3B down 0.58 and Gemma 3 4B-it down 0.56, versus 0.07 for GPT 5.2 and 0.03 for Qwen3 235B. UnitConversion averaged -0.086 and persisted at scale; Sycophancy -0.028, SkippedSteps -0.022, ExtraSteps -0.004. Fixed-effects regression on log10 parameters gives slopes of 0.194 for MathError, 0.065 for UnitConversion, 0.026 and 0.031 for Sycophancy and SkippedSteps, 0.006 for ExtraSteps.

Evidence is measured with a matched clean baseline per model and significance marks at 90% and 99%, but parameter counts for closed models are guessed (500B/1.2T/2T), which the scaling regression depends on. Training methodology is not isolated. Chains are short (3-10 steps) and GSM8K-easy; deeper injections and harder problems are untested.

If you pipe upstream reasoning into a model, add external arithmetic and unit checks rather than trusting a larger model.

Written from the abstract by claude-opus-5 on 2026-10-17, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsWhen an error is injected into a supplied chain of thought, models mostly carry it to the answer: a planted arithmetic error costs small models 50-60% accuracy and 18% on average, though the largest models lose only 3-7%; unit-conversion errors cost about 9% at every scale; skipped steps and sycophantic steps cost small models about 10%; extra correct steps cost almost nothing (0-6%).