hashimotos-ratchet

Hashimoto's ratchet

“Every time the agent makes a mistake, engineer a solution so that mistake never happens again.”

Also: Hashimoto's law · One rule per failure

Origin

Mitchell Hashimoto's practice as described in "My AI Adoption Journey" (February 2026) and named the ratchet by the Harness Engineering playbook. From software operations, where the older form is the blameless post-mortem whose output is a permanent fix, not a reminder.

Why it should, or should not, apply to models

The mechanism is that a fix placed in structure (a guide rule, a sensor, a permission) is applied on every future run without anyone remembering it, while a fix given in conversation dies with the session. That holds for models more strongly than for people, because a model has no memory across sessions to carry a spoken correction. The place it can fail is the guide layer: a rule the model reads but does not follow is a ratchet tooth that does not catch, so the ratchet works only where the fix is enforced rather than advised.

Standing: Untested

From reviewed claims only. Unreviewed claims are listed below and marked, and move nothing.

Where it holds only under a condition

Claims that keep the adage but bound it.

Giving a coding agent a repository context file — AGENTS.md, CLAUDE.md — does not raise its success rate on benchmark coding tasks and costs about 20% more inference: across 4 agents, 2 benchmarks and 3 conditions, LLM-generated files hurt slightly in 5 of 8 settings while developer-written ones gained 2.4% (p=0.21), and agents obey the files — which is why they spend more — so the files do not carry success-relevant information rather than being ignored.
Following an unfamiliar procedurecontestedpending review
Rules written into a context file did not raise success on benchmark coding tasks. The ratchet's guide layer is advisory; the finding narrows the adage to fixes the harness enforces (sensors, permissions) rather than fixes it merely states.

Where it holds

Claims showing the adage applying to models.

A correction made in conversation fixes one exchange and is gone when the session ends; the same correction encoded as a guide rule, a sensor, or a permission fixes every future run — so an agent system improves over time only to the extent that failures are converted into harness structure rather than re-applied as prompts, and the rate of new guide rules per week falling is the sign the conversion is working.
Following an unfamiliar procedurepending review
States the mechanism directly: corrections in conversation are lost at session end; corrections as harness structure apply to every run.
With weights frozen and no stronger model in the loop, an agent can raise its own pass rate by rewriting the scaffolding it runs inside — clustering its failures, proposing minimal edits to prompts, tools and policy, and keeping only edits that improve one split without degrading another: all 9 of 9 model-by-benchmark pairs improved, by up to 40.6 percentage points (GLM-5 on AppWorld, 44.4% to 85.0%), with weak models gaining most; but the promotion gate reads the held-out split, so the reported held-out gain is not clean generalisation, and there is no matched-compute baseline.
Fixing its own mistakespending review
A loop that turns each failure cluster into a kept harness edit raised pass rate with the model frozen, which is the ratchet run by the agent itself.

Add a case where it holds or breaks

A break is worth more than a hold. If you have seen this fail for a model, with the setup written down, that is the most useful thing you can add here.