arxiv-2609-23065 SRC-0520 · paper

From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness

Created: 2026-09-19 · Ingested: 2026-10-17

Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through input--output behavior or input attributions, leaving internal computation largely underexplored. We instead cast faithfulness as internal concept grounding: Does a large language model's (LLM) CoT reasoning engage the same internal concepts that support the LLM's direct prediction, and do the shared concepts causally drive its answer? Encoding a prediction pass and a CoT pass with a single shared sparse autoencoder (SAE), a reliable approximator of

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsWhen chain-of-thought faithfulness is measured by how much the internal concepts (sparse-autoencoder features) active during a CoT pass overlap with those active during a direct-answer pass, that overlap barely predicts whether the shared concepts causally drive the answer: across five open models and four reasoning datasets, above-median concept alignment raised the chance of an above-median causal ablation effect by only about 8%, so representational correspondence cannot substitute for an ablation test.