introduced-grade-school-math-word-problems-2021-models-mademechanismsingle paper
Introduced grade-school math word problems on which 2021 models made frequent arithmetic slips, motivating verifier training.
Capability: Digit-level arithmetic
Sources
- Introduced grade-school math word problems on which 2021 models made frequent arithmetic slips, motivating verifier training.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- Offloading the computation to generated code and an interpreter removes most arithmetic errors on math word problems.Digit-level arithmetic
- Writing out reasoning steps improves how a problem is decomposed but does not fix the arithmetic inside a step — the model can decompose correctly and still get the calculation wrong, at the same confident register as when it is right. For exact arithmetic the fix is to move the computation out of the text, not to add more steps.Digit-level arithmetic · contested