presenting-syllogisms-euler-or-linear-diagrams-instead-text-or CLM-0318observationsingle paperReviewed by AI
Presenting syllogisms as Euler or linear diagrams instead of text or logical notation did not consistently improve two models' accuracy on 285 problems; both did well on entailment and contradiction items but poorly on neutral ones (GPT-4o-mini under 30% on neutral cases), and made systematic conversion errors, treating 'all A are B' as if it implied 'all B are A'.
Capability: Deducing validly from logical structure
Observed on
Claude 3.5 Sonnet and GPT-4o-mini, 285 syllogisms from Ando et al. 2024 in four representations, 2026..
Sources
- Two models and one problem set; the neutral-case weakness and conversion errors match older findings on human reasoners, which is what makes them worth recording.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Across 14 models, syllogisms posed in symbolic form are solved far better than the same inferences in natural language: top models reach near-perfect symbolic accuracy (about 99.6%) while scoring around chance (52%) on the natural-language understanding variant, mean 81.7% versus 56% overall, and few-shot prompting did not help, so formal-pattern competence does not carry over to reading the logic out of ordinary sentences.Deducing validly from logical structure · reviewed by AI
- Models judge a syllogism's validity partly by how believable its conclusion is because validity and plausibility are encoded as linearly separable but strongly aligned directions in the residual stream: steering along the plausibility direction causally shifts validity judgements and vice versa, the degree of alignment predicts the size of the content effect across models, and a debiasing vector that separates the two reduces the effect and raises accuracy.Deducing validly from logical structure · reviewed by AI
- When an 8B model verifies natural-language claims against knowledge-graph triples, scoring the evidence twice under opposing prompts (argue it supports, argue it refutes) and adjudicating between the two arguments avoids the bias toward a "Supported" verdict that a single-perspective assessment shows, and is the largest single contributor to accuracy in that pipeline.Checking claims against evidence · reviewed by AI
- For a domain-specific language essentially absent from pretraining corpora (FORM symbolic algebra), QLoRA fine-tuning an 8B model on a few thousand examples validated by executing them through the language's own interpreter beats frontier open-weight models of up to 756B parameters given syntax documentation in context, on execution rate and strict output match for the larger benchmarks, while general reasoning scores drop under 3 points.Generating and editing working code · reviewed by AI
- For 0.6B-8B Qwen3 models on math, logic and multiple-choice reasoning, quantifying reasoning uncertainty from a directed acyclic graph built by sampling several chains and merging semantically equivalent steps with an NLI model separates correct from incorrect generations better than sequence-level entropy, likelihood and self-prompting uncertainty measures.Carrying a chain of inference through to its conclusion · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Diagram input helping a model that reads diagrams natively, or neutral-case accuracy above 70% on the same set.