introduced-grade-school-math-word-problems-2021-models-made CLM-0074mechanismsingle paperReviewed by AI and Russ Miller
Introduced grade-school math word problems on which 2021 models made frequent arithmetic slips, motivating verifier training.
Capability: Digit-level arithmetic
Sources
- Introduced grade-school math word problems on which 2021 models made frequent arithmetic slips, motivating verifier training.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Offloading the computation to generated code and an interpreter removes most arithmetic errors on math word problems.Digit-level arithmetic
- On human-annotated step-level verification of frontier-model proofs to recent IMO and Putnam problems, most of 29 generative critics and process reward models do little better than chance at finding the first wrong step, a leading math PRM (Qwen2.5-Math-PRM-72B) reaches only about 37% balanced F1, and beyond a few standouts open-source verifiers lag closed models; verification is harder than generation for the same frontier models.Spotting an error in a given artifact · reviewed by AI
- Extended reasoning budgets improve a model's execution of arithmetic procedures on operands longer than those seen in training far more than they improve surface-form-to-value tasks like magnitude comparison and format conversion, because chain-of-thought supplies external working memory for carries and column alignment but cannot change how the input was tokenized.Digit-level arithmetic · reviewed by AI
- Writing out reasoning steps improves how a problem is decomposed but does not fix the arithmetic inside a step — the model can decompose correctly and still get the calculation wrong, at the same confident register as when it is right. For exact arithmetic the fix is to move the computation out of the text, not to add more steps.Digit-level arithmetic · contested