practice-makes-perfect ADG-0022

Practice makes perfect

“Doing a thing repeatedly makes you better at it.”

Origin

English proverb, from the Latin usus promptos facit; sixteenth-century English usage. About skill acquired through repetition, resting on memory that persists between attempts.

Why it should, or should not, apply to models

It should not transfer, and the reason is precise: the premise is memory across attempts, and a deployed model's weights do not change between sessions. Repetition within a session can help through in-context examples, but nothing carries over. Where the proverb appears to hold for an agent system, the practice is accumulating somewhere other than the model — in its harness, its guide files, its memory store — which is a different claim with a different failure mode.

Standing: Untested

Standing counts claims a person has reviewed. Claims reviewed by AI alone are listed and marked, and do not move it.

Where it breaks

Claims showing the heuristic failing for models.

A correction made in conversation fixes one exchange and is gone when the session ends; the same correction encoded as a guide rule, a sensor, or a permission fixes every future run — so an agent system improves over time only to the extent that failures are converted into harness structure rather than re-applied as prompts, and the rate of new guide rules per week falling is the sign the conversion is working.
Following an unfamiliar procedureReviewed by AI
A correction made in conversation is gone when the session ends. The model does not get better from having been corrected; the system does only if the correction is moved into structure.

Where it holds only under a condition

Claims that keep the heuristic but bound it.

On web-browsing (WebArena, Mind2Web) and software-engineering (SWE-Bench-Verified) benchmarks, an agent that distills reusable strategies from both its successful and its failed attempts, judged by an LLM without ground truth, and retrieves the relevant ones on later tasks, outperforms memories that store raw trajectories or only successful routines — up to 20% relative improvement in success and up to 16% fewer interaction steps — with the weights unchanged; including the failures is part of the gain, not noise.
Improving its own scaffolding from failuresReviewed by AI
Weights frozen: practice made the agent's memory better, and the agent better with it, across tasks — the system improves, the model does not.
With weights frozen and no stronger model in the loop, an agent can raise its own pass rate by rewriting the scaffolding it runs inside — clustering its failures, proposing minimal edits to prompts, tools and policy, and keeping only edits that improve one split without degrading another: all 9 of 9 model-by-benchmark pairs improved, by up to 40.6 percentage points (GLM-5 on AppWorld, 44.4% to 85.0%), with weak models gaining most; but the promotion gate reads the held-out split, so the reported held-out gain is not clean generalisation, and there is no matched-compute baseline.
Fixing its own mistakesReviewed by AI
Practice makes the harness perfect, not the model: with weights frozen, repeated failure-driven edits to the scaffolding raised pass rate.

Add a case where it holds or breaks

A break is worth more than a hold. If you have seen this fail for a model, with the setup written down, that is the most useful thing you can add here.

Notes

Wanted: a measured comparison of the same model on the same task at attempt 1 and attempt 20 within one session, with and without the earlier attempts in context, to pin down how much in-session repetition buys.