Position embeddings that encode digit place let small transformers add 100-digit numbers, showing the limit is representational not fundamental.

Sources

supportsTransformers Can Do Arithmetic with the Right Embeddings
Position embeddings that encode digit place let small transformers add 100-digit numbers, showing the limit is representational not fundamental.

Status: activeLast checked: 2026-09-03Evidence activity

Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.