self-feedback-improves-generation-quality CLM-0146mechanismsingle papercontestedReviewed by AI and Russ Miller
Prompting a model to generate its own feedback on a draft output and revise accordingly improves quality on open-ended generation tasks (dialogue, code, review-writing) over a single-shot attempt.
Capability: Fixing its own mistakes · Reasoning
Observed on
2023, GPT-3.5/GPT-4 class.
Model cards: GPT-4
Sources
- Iterative self-feedback and refinement improved outputs across several generation tasks in the original evaluation.
- Re-examines self-feedback-style claims and argues the measured gains often trace back to how the comparison was set up (e.g. oracle-like stopping, or the revision prompt leaking task-specific hints) rather than the self-feedback step itself doing real work.
Suspected axis of disagreement(a guess, not verified)
Likely how rigorously the baseline (no-revision) condition was set up and evaluated, and possibly task family — open-ended generation tasks (where "better" is fuzzier and easier to nudge with a second pass) vs. the more checkable reasoning benchmarks in the contesting paper. I haven't reconciled the two papers' exact experimental setups myself.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In UI-to-code generation, where a rendered screenshot gives genuine external feedback, iterative self-critique still drifts below the initial draft over ten rounds, because a code edit changes the rendering non-locally and a fix for one region breaks regions that were already faithful; restricting each round to one typed, scoped repair target and carrying past targets forward as an avoid-list reduces that drift.Generating and editing working code · reviewed by AI
- For a frozen LLM run in a generate-critique-revise loop with no external feedback, the accuracy gain is concentrated where single-shot reasoning is already unreliable: on clean multi-step reasoning (BBH, ~74% single-shot) depth makes no difference beyond the confidence interval, while on harder math (MATH, 26.2% single-shot) the same loop gains +14.2 points, with GSM8K in between (+4.2).Fixing its own mistakes · reviewed by AI
- When automatically rewriting the prompts of a multi-agent LLM system from its failures, locating the responsible agent by injecting a hint into one agent at a time and using that agent's hint-corrected intermediate output as the supervision signal produces prompt updates that generalize better than critique derived only from the system-level output.Improving its own scaffolding from failures · reviewed by AI
- When a prompt contains several legitimate but conflicting evidence passages, models internally register the conflict — linear probes on mid-to-late-layer hidden states separate conflicting from consistent inputs at high AUC — while still resolving the conflict badly at the output, so the failure is in evidence integration rather than in conflict detection.Checking claims against evidence · reviewed by AI
- Used zero-shot as step-level action critics in long-horizon tool-calling tasks, frontier models are over-pessimistic — flagging a large share of correct actions and pushing the actor into revision loops that lower success below no critic at all — whereas small models fine-tuned on action-level verification rationales flag far fewer valid actions and raise repeated-run reliability.Using the tools it is given · reviewed by AI
Notes
This is the pair I'd most want to actually dig into further — the disagreement axis above is a first guess, not a verified reconciliation.