process · external-feedback-repair

Repair with external feedback

Give the model a concrete external signal, such as test output, and let it reflect and retry.

Instead of asking the model to review its own answer, run the answer against something that can fail: unit tests, a compiler, a verifier, or an environment. Feed the failure back, have the model write a short reflection on what went wrong, and retry with that reflection in context. Bound the number of attempts and stop when the signal stops changing.

Addresses: Fixing its own mistakes

Contexts: Coding agent, Math, Autonomous agent

Does it work?

contested2 supporting · 1 contesting sources · last moved 2026-09-04

Efficacy claims — what this technique actually moves, under which conditions, and whether that has been contested.

mechanismreplicatedcontested
Reflect-and-retry raises task success when the feedback in the loop is a genuine external signal — a failing test, a compiler error, an environment outcome — but not when the "feedback" is the model's own unaided critique. The grounding, not the reflection step, is what does the work.

Code

https://github.com/noahshinn/reflexion — Reflexion reference implementation for coding and decision tasks.verified 2026-09-02
https://github.com/madaan/self-refine — Self-Refine prompts and evaluation harness.verified 2026-09-02

Does it work?

Code

Sources