architecture · digit-position-embeddings TEC-0008

Encode digit position in the model

Add positional embeddings that tell the model which place each digit occupies.

Abacus embeddings give every digit an embedding for its position within the number, so alignment of digits across operands becomes trivial. Small transformers trained this way generalize to numbers far longer than seen in training.

Requires: A training-time change; not applicable to a model you cannot retrain.

Addresses: Digit-level arithmetic

Contexts: Math

Does it work?

nothing measured0 supporting · 0 contesting sources

Efficacy claims — what this technique actually moves, under which conditions, and whether that has been contested.

No efficacy claim filed yet. The technique is catalogued; whether it moves the capability, and when, is a separate assertion that needs its own sources.

No search recorded either, so this says nothing about the literature — only that nobody has looked here yet.

Code

https://github.com/mcleish7/arithmetic — Training and evaluation code for abacus embeddings.verified 2026-09-02

Sources

Transformers Can Do Arithmetic with the Right Embeddings (2024)

Either way, bring evidence: a link, or what you saw.

Whether a technique works is a claim. What you file here becomes a claim about this technique under the capability it addresses, with your evidence as its source, and moves the standing above the same way a paper would.