language-models-inherit-the-positive-bias-of-the-literature CLM-0518mechanismmechanism reasoningReviewed by AI
Language models trained on the published literature inherit its positive bias, because failure data is systematically missing from what they read, and that degrades them in three roles at once — as research tools that suggest what to try, as consumers of scarce high-quality training data, and as peer reviewers — so the absence of negative results now costs more than it did when only people read the papers.
Capability: Whether the measurement made the finding
Sources
- A position paper that argues the mechanism and outlines experimental protocols to test it; the protocols are proposed, not run.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- The published research literature overreports results that support the hypothesis tested: in psychology, 96% of a random sample of standard hypothesis-testing papers reported a positive first result against 44% of Registered Reports, which are accepted before results are known; and across all disciplines the share of hypothesis-testing papers reporting positive support rose from 70.2% in 1990-1991 to 85.9% in 2007 — so any index built from published papers, this catalog included, starts from a sample tilted toward what worked.Whether the measurement made the finding · reviewed by AI
- In role-play with an assigned character, models usually recognize that a query lies outside the character's knowledge boundary yet fail to abstain when asked a concrete factual question about it, so the dominant failure is compliance rather than recognition.Saying it does not know · reviewed by AI
- Within one model family, the relative distribution of reasoning failure modes stays similar from 1.5B to 72B, so critiques mined offline from small models' wrong answers, retrieved by failure mode and inserted as in-context exemplars, raise a large model's math accuracy above random, fixed, or embedding-similarity exemplar selection at a single generation.Judging another's work as well as a careful reviewer would · reviewed by AI
- Training the model to decide when to retrieve and to critique whether passages support its output improves factuality and citation accuracy.Checking claims against evidence
- Most models claiming long contexts fail well before their advertised length on synthetic retrieval, tracing and aggregation tasks.Losing information in long inputs
Notes
Why the catalog's failure-first design matters for models, not only for researchers: an agent that asks a model what works is asking a reader of a tilted sample.