Position embeddings that encode digit place let small transformers add 100-digit numbers, showing the limit is representational not fundamental.

Sources

supportsTransformers Can Do Arithmetic with the Right Embeddings
Position embeddings that encode digit place let small transformers add 100-digit numbers, showing the limit is representational not fundamental.

Reviewed by AI and Russ MillerLast checked: 2026-09-03Evidence activity

Either way, bring evidence: a link, or what you saw.

Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.