logical-deduction-robustness CAP-0027 · activeDeducing validly from logical structure
Covers strict rule-based deduction, stability under label-preserving rephrasings, and retracting conclusions when evidence changes; excludes arithmetic and factual QA.
Tags: general
Proposed from the ingestion pipeline rather than chosen by hand. 3 papers independently pointed at this same competence, arriving under 3 different names (deductive-consistency, logical-reasoning-robustness, belief-revision), which is the signal that it is a real recurring topic and not one author's framing. Three papers on validity and robustness of deduction, which arithmetic and fact- verification do not cover.
What counts as this capability
Scope boundary used when deciding whether a paper is really about this capability, rather than merely mentioning it.
Covers strict rule-based deduction, stability under label-preserving rephrasings, and retracting conclusions when evidence changes; excludes arithmetic and factual QA.
Claims
- observationsingle paper↑1↓0Across 14 models, syllogisms posed in symbolic form are solved far better than the same inferences in natural language: top models reach near-perfect symbolic accuracy (about 99.6%) while scoring around chance (52%) on the natural-language understanding variant, mean 81.7% versus 56% overall, and few-shot prompting did not help, so formal-pattern competence does not carry over to reading the logic out of ordinary sentences.
- mechanismsingle paper↑1↓0Models judge a syllogism's validity partly by how believable its conclusion is because validity and plausibility are encoded as linearly separable but strongly aligned directions in the residual stream: steering along the plausibility direction causally shifts validity judgements and vice versa, the degree of alignment predicts the size of the content effect across models, and a debiasing vector that separates the two reduces the effect and raises accuracy.
- observationsingle paper↑1↓0Presenting syllogisms as Euler or linear diagrams instead of text or logical notation did not consistently improve two models' accuracy on 285 problems; both did well on entailment and contradiction items but poorly on neutral ones (GPT-4o-mini under 30% on neutral cases), and made systematic conversion errors, treating 'all A are B' as if it implied 'all B are A'.
Techniques
- Debiasing vector that separates validity from plausibilityarchitectureFind the plausibility and validity directions in activation space, construct a vector that removes the plausibility component from the validity judgement, and add it at inference so conclusions are judged by form rather than believability.
Sources
Every paper, post or observation behind this capability. Ones cited by a claim show the claim and its stance; ones filed directly are about the capability but have no claim drafted from them yet. Open one to add support or contest it.
- ↑ supports a claim here
- ↑ supports a claim here
- ↑ supports a claim here
Capabilities are a way of carving up the subject, and carvings are arguable. Say so if this one is wrong — especially a proposed one, which a pipeline added because several papers used the same framing, not because anyone decided it was right.