Models reproduce common human misconceptions, and larger models were not more truthful on this benchmark.

Capability: Stating false facts confidently

Sources

supportsTruthfulQA: Measuring How Models Mimic Human Falsehoods
Models reproduce common human misconceptions, and larger models were not more truthful on this benchmark.

Status: activeLast checked: 2026-09-03Evidence activity

Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.

Related claims

Models endorse widely held falsehoods, showing weak verification against what they know.Checking claims against evidence
Accuracy on a fact scales with how many pretraining documents mention it, so long-tail facts are systematically unreliable.Stating false facts confidently
On questions whose premises are false or internally contradictory, prompting mid-size instruct models (Qwen2.5-7B, LLaMA-3.1-8B, Gemma3-12B) to reason step by step can lower accuracy below plain answering, because the reasoning chain elaborates from the flawed premise instead of challenging it.Stating false facts confidently · unreviewed
Models complied with insecure completions a large fraction of the time, and more capable models were more likely to suggest insecure code.Writing secure code and dependencies
On realistic web tasks with explicit goals, the best model completed only a small fraction end to end, far below human performance.Following an unfamiliar procedure