Repair with external feedback
Give the model a concrete external signal, such as test output, and let it reflect and retry.
Instead of asking the model to review its own answer, run the answer against something that can fail: unit tests, a compiler, a verifier, or an environment. Feed the failure back, have the model write a short reflection on what went wrong, and retry with that reflection in context. Bound the number of attempts and stop when the signal stops changing.
Contexts: Coding agent, Math, Autonomous agent
Does it work?
contested2 supporting · 1 contesting sources · last moved 2026-09-04
Efficacy claims — what this technique actually moves, under which conditions, and whether that has been contested.
Needs: an external signal (a test, a compiler, an environment outcome)
Fails when:
- The feedback in the loop is the model's own unaided critique rather than a test result, compiler error or environment outcome. (helps)
2 more conditions from claims reviewed by AI, listed with the claims below.
- mechanismreplicatedcontestedReflect-and-retry raises task success when the feedback in the loop is a genuine external signal — a failing test, a compiler error, an environment outcome — but not when the "feedback" is the model's own unaided critique. The grounding, not the reflection step, is what does the work.
- supports Reflexion: Language Agents with Verbal Reinforcement Learning (2023)
- supports Large Language Models Cannot Self-Correct Reasoning Yet (2023)
- contests Self-Refine: Iterative Refinement with Self-Feedback (2023)
Code
- https://github.com/noahshinn/reflexion — Reflexion reference implementation for coding and decision tasks.verified 2026-09-02
- https://github.com/madaan/self-refine — Self-Refine prompts and evaluation harness.verified 2026-09-02
Sources
Whether a technique works is a claim. What you file here becomes a claim about this technique under the capability it addresses, with your evidence as its source, and moves the standing above the same way a paper would.