multi-digit-multiplication-accuracy-falls-near-zero-problem-size CLM-0099mechanismsingle paperReviewed by AI and Russ Miller
Multi-digit multiplication accuracy falls to near zero as problem size grows, and models rely on pattern matching rather than a learned algorithm.
Capability: Digit-level arithmetic
Sources
- Multi-digit multiplication accuracy falls to near zero as problem size grows, and models rely on pattern matching rather than a learned algorithm.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Arithmetic accuracy in frontier models depends strongly on how digits are chunked into tokens; a different digit grouping changes error rates.Digit-level arithmetic
- Offloading the computation to generated code and an interpreter removes most arithmetic errors on math word problems.Digit-level arithmetic
- Few-shot arithmetic accuracy correlates with how often the specific numbers appear in pretraining data, evidence of memorization.Digit-level arithmetic