Digit-level arithmetic

Claims

• mechanismsingle paper↑1↓0
  Arithmetic accuracy in frontier models depends strongly on how digits are chunked into tokens; a different digit grouping changes error rates.
• mechanismreplicatedcontested↑2↓1
  Writing out reasoning steps improves how a problem is decomposed but does not fix the arithmetic inside a step — the model can decompose correctly and still get the calculation wrong, at the same confident register as when it is right. For exact arithmetic the fix is to move the computation out of the text, not to add more steps.
• observationsingle paper↑1↓0
  On clinical-calculator tasks where the formula, gold variables and full note are all supplied, having an open-weight model write case-specific Python for a restricted executor instead of doing the arithmetic itself gives no reliable gain at 7B (+3.29 pp, calculator-cluster CI crossing zero) but a clear gain at 32B (+7.05 pp, CI clear of zero), and the remaining errors are invented unit conversions and one-tier slips in banded scores rather than raw computation.
• mechanismsingle paper↑1↓0
  Extended reasoning budgets improve a model's execution of arithmetic procedures on operands longer than those seen in training far more than they improve surface-form-to-value tasks like magnitude comparison and format conversion, because chain-of-thought supplies external working memory for carries and column alignment but cannot change how the input was tokenized.
• mechanismsingle paper↑1↓0
  Few-shot arithmetic accuracy correlates with how often the specific numbers appear in pretraining data, evidence of memorization.
• mechanismsingle paper↑1↓0
  Introduced grade-school math word problems on which 2021 models made frequent arithmetic slips, motivating verifier training.
• mechanismsingle paper↑1↓0
  Multi-digit multiplication accuracy falls to near zero as problem size grows, and models rely on pattern matching rather than a learned algorithm.
• mechanismsingle paper↑1↓0
  On numeric sequences whose first differences repeat only after a long prefix, LLaMA 3.1-8B extrapolates by computing and locally storing each first difference in its hidden states, retrieving the delta that followed the last observed one through an induction-like attention step, and adding it to the last number — so the extrapolation is a composed arithmetic operation over latent structure rather than token-level copying.
• mechanismsingle paper↑1↓0
  Offloading the computation to generated code and an interpreter removes most arithmetic errors on math word problems.
• mechanismsingle paper↑1↓0
  Position embeddings that encode digit place let small transformers add 100-digit numbers, showing the limit is representational not fundamental.
• mechanismmechanism reasoning↑1↓0
  Offloading the calculation to generated code removes the arithmetic as a source of error but does not remove error — what remains is the translation from problem to program, which is now the only thing the model is doing. Verification has to move there with it.

Techniques

• Encode digit position in the modelarchitecture
  Add positional embeddings that tell the model which place each digit occupies.
• Offload arithmetic to codetooling
  Have the model write a short program for the numeric part and run it, instead of computing in text.
• Show intermediate stepsprompting
  Prompt the model to write out intermediate reasoning steps before the final answer.

Sources

Every paper, post or observation behind this capability. Ones cited by a claim show the claim and its stance; ones filed directly are about the capability but have no claim drafted from them yet. Open one to add support or contest it.

• Towards a Deterministic Math Solver for Clinical Language Models2026-09-09
  ↑ supports a claim here
• Mechanistic Interpretability of Structure-Aware Numerical Reasoning in LLaMA 3.1 8B2026-08-19
  ↑ supports a claim here
• Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvement2026-08-13
  ↑ supports a claim here
• Tokenization counts: the impact of tokenization on arithmetic in frontier LLMsThis source: Tokenization counts: the impact of tokenization on arithmetic in front…How much the field cites it — steadily cited in the last 12 months44 in the last 12 months · 126 totalpublished 2024checked 2026-09-04click for Semantic Scholar2024
  ↑ supports a claim here · ↓ contests a claim here
• PAL: Program-aided Language ModelsThis source: PAL: Program-aided Language ModelsHow much the field cites it — very heavily cited in the last 12 months268 in the last 12 months · 824 totalpublished 2022checked 2026-09-042022
  ↑ supports a claim here · ↑ supports a claim here · ↑ supports a claim here
• Faith and Fate: Limits of Transformers on CompositionalityThis source: Faith and Fate: Limits of Transformers on CompositionalityHow much the field cites it — very heavily cited in the last 12 months229 in the last 12 months · 690 totalpublished 2023checked 2026-09-042023
  ↑ supports a claim here · ↑ supports a claim here
• Impact of Pretraining Term Frequencies on Few-Shot ReasoningThis source: Impact of Pretraining Term Frequencies on Few-Shot ReasoningHow much the field cites it — steadily cited in the last 12 months15 in the last 12 months · 191 totalpublished 2022checked 2026-09-04click for Semantic Scholar2022
  ↑ supports a claim here
• Training Verifiers to Solve Math Word ProblemsThis source: Training Verifiers to Solve Math Word ProblemsHow much the field cites it — very heavily cited in the last 12 months1000+ in the last 12 months · 10385 totalpublished 2021checked 2026-09-04count capped at 1000 by the fetchclick for Semantic Scholar2021
  ↑ supports a claim here
• Transformers Can Do Arithmetic with the Right EmbeddingsThis source: Transformers Can Do Arithmetic with the Right EmbeddingsHow much the field cites it — steadily cited in the last 12 months36 in the last 12 months · 91 totalpublished 2024checked 2026-09-04click for Semantic Scholar2024
  ↑ supports a claim here