few-shot-arithmetic-accuracy-correlates-how-often-specific-numbers CLM-0046mechanismsingle paperReviewed by AI and Russ Miller
Few-shot arithmetic accuracy correlates with how often the specific numbers appear in pretraining data, evidence of memorization.
Capability: Digit-level arithmetic
Sources
- Few-shot arithmetic accuracy correlates with how often the specific numbers appear in pretraining data, evidence of memorization.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Accuracy on a fact scales with how many pretraining documents mention it, so long-tail facts are systematically unreliable.Stating false facts confidently
- Multi-digit multiplication accuracy falls to near zero as problem size grows, and models rely on pattern matching rather than a learned algorithm.Digit-level arithmetic
- Extended reasoning budgets improve a model's execution of arithmetic procedures on operands longer than those seen in training far more than they improve surface-form-to-value tasks like magnitude comparison and format conversion, because chain-of-thought supplies external working memory for carries and column alignment but cannot change how the input was tokenized.Digit-level arithmetic · reviewed by AI