llm-driven-iterative-prompt-optimization-restricting-gradient-generation-examples-current CLM-0420In LLM-driven iterative prompt optimization, restricting gradient generation to examples the current prompt gets wrong and accepting a candidate edit only when its gain on hard cases exceeds a penalized regression on a preservation set of initially-correct examples yields higher final accuracy than unfiltered textual-gradient optimization, which degrades on some reasoning tasks below the zero-shot prompt.
Capability: Improving its own scaffolding from failures
Observed on
Reasoning and code benchmarks with exact-match scoring, a weaker executor model driven by a stronger evaluator/optimizer, 12 optimization steps; cost is higher because several cand.
Sources
- Validated by: Accuracy on 10 reasoning benchmarks across three evaluator/optimizer models against Zero-shot CoT, Three-shot CoT, TextGrad and REVOLVE (average ~3.5 points over REVOLVE), plus a modernized run with gpt-5.4-mini/gpt-5.4 on BBEH, GSM8K-Platinum and DS-1000; ablations remove the gate (lower) and swap in full-dataset or correct-only gradients (lower, correct-only falls below the initial prompt).. Measured on 10 benchmarks x 3 optimizer models plus 4 modernized tasks, mean of three runs, with component ablations on three datasets. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When training an LLM with policy-gradient RL on tasks that decompose into many dependent sub-problems, rewarding each trajectory segment by how many reference reasoning points it newly reaches beats a sparse final-answer reward by a margin that widens with task horizon, and still learns on problem sets where the base policy never samples a success and sparse reward gives no gradient at all.Completing tasks that take many steps or many hours · reviewed by AI
- For a frozen LLM run in a generate-critique-revise loop with no external feedback, the accuracy gain is concentrated where single-shot reasoning is already unreliable: on clean multi-step reasoning (BBH, ~74% single-shot) depth makes no difference beyond the confidence interval, while on harder math (MATH, 26.2% single-shot) the same loop gains +14.2 points, with GSM8K in between (+4.2).Fixing its own mistakes · reviewed by AI
- For 8B instruction-tuned backbones, training a LoRA corrector to diagnose a typed, executable error operator (error type, step location, corrupted and repaired proposition) and apply its inverse as a localized edit raises accuracy over free-form self-revision baselines (Self-Refine, RISE, SCoRe) on math, code and commonsense tasks, and dropping either the operator-prediction loss or the adjacent-state reconstruction loss loses most of the gain.Fixing its own mistakes · reviewed by AI
- When asked to optimize code that is already at its performance ceiling, frontier LLMs (GPT, Claude, Gemini families) never say it is already optimal under a plain "optimize this code for speed" prompt — they always emit an edit with a confident, unmeasured speedup claim; adding an explicit instruction to abstain unless >90% confident of an improvement produces abstention on a minority of such cases while leaving the edit rate on genuinely sub-optimal code unchanged.Stating false facts confidently · reviewed by AI
- For LLM data-preparation pipelines in specialist domains (math, finance, law, medicine), diagnosing a small pilot set of generated samples against domain-selected quality criteria and rewriting the individual pipeline components (chunking, instruction builder, response generator) before full-scale generation yields training data with higher downstream fine-tuning utility than post-hoc filtering or one-shot agent/skill generators using the same backbone — but the gain is non-monotonic in rounds, peaking at two and degrading after.Improving its own scaffolding from failures · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An evaluation where unfiltered textual-gradient optimization with the same candidate budget and validation-based checkpoint selection matches or beats the error-only plus preservation-gate variant across tasks. Drafted stance toward a-fixed-model-can-improve-its-own-harness-with-a-regression-gated-loop: supports -- It provides independent evidence that gating self-generated harness edits on a non-regression check is what makes the improvement loop stable.