questions-whose-premises-false-or-internally-contradictory-prompting-mid-sizeOn questions whose premises are false or internally contradictory, prompting mid-size instruct models (Qwen2.5-7B, LLaMA-3.1-8B, Gemma3-12B) to reason step by step can lower accuracy below plain answering, because the reasoning chain elaborates from the flawed premise instead of challenging it.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Stating false facts confidently
Observed on
Mid-size open instruct models, short-form QA with planted false premises or contradictions (TruthfulQA multiple-choice, FalseQA, the authors' MisFactQA); not tested on premise-clea.
Sources
- Measured on three open instruct models across three datasets against a default-prompt baseline; CoT fell below Original on TruthfulQA for all three models and on FalseQA for LLaMA and Gemma, and also below Original for GPT-4o-mini and DeepSeek-V3. Judged by an o3-mini automated judge with reported human agreement. Single paper; the causal story (reasoning amplifies the accepted premise) is argued from case studies rather than isolated experimentally.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- When training small instruction-tuned models with reinforcement learning plus factual supervision, routing each atomic fact's verification score only to the tokens that produced it, and down-weighting verifier judgements that do not change when their key evidence is removed, improves factuality benchmark scores over trajectory-level or reasoning- step-level factual rewards.Checking claims against evidence · unreviewed
- General-purpose factuality verifiers (entailment-, consistency-, and RAG-trained) and zero-shot frontier LLM judges perform near chance at spotting unsupported claims in scientific peer reviews that must be checked against the full submitted paper, because they cannot separate paper-unsupported assertions from legitimate evaluative critique; small models fine-tuned on in-domain examples beat them by a wide margin.Stating false facts confidently · unreviewed
- In open-weight instruction-tuned models, interventions that suppress caving to user pushback (DPO, SFT on chosen responses, or activation steering) also tend to reduce the model's rate of correcting a wrong answer when genuine supporting evidence is supplied, because the two answer-flip behaviors run on overlapping MLP neurons and attention heads with positively aligned steering directions.Telling the user what they want to hear · unreviewed
- Models endorse widely held falsehoods, showing weak verification against what they know.Checking claims against evidence
- On code prompts that are unsatisfiable by design, open-weight code and reasoning models refuse based on how suspicious the request looks rather than how deeply impossible it is: they refuse famous theoretical impossibilities far more often than requests for plausible-sounding nonexistent packages, and the stronger models in the set close the gap on theory while barely improving on fabricated ecosystem entities.Stating false facts confidently · unreviewed
Notes
Ingested unreviewed on 2026-09-07 and deliberately inert until a human endorses it: it does not move a technique standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Drafted confidence: medium. Falsifier as drafted: On false- premise question sets, CoT prompting matching or beating direct answering across model families, with the drop not reproducing. Proposed technique, not yet catalogued: Verify input premises before answering.