arxiv-2609-23716 SRC-0522 · paper

STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification

Created: 2026-09-20 · Ingested: 2026-10-17

Textual-gradient methods automate prompt optimization through natural-language feedback, but their iterative updates can be unstable. We identify two sources of this instability: noisy gradients produced from already-correct examples and over-specialization to hard cases that degrades performance on simpler inputs. We introduce STEVE, a stabilization framework with two coupled mechanisms. Error-Driven Refinement generates gradients only from incorrectly handled examples, concentrating updates on informative failures. Regularized Verification treats every update as provisional and accepts it on

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsIn LLM-driven iterative prompt optimization, restricting gradient generation to examples the current prompt gets wrong and accepting a candidate edit only when its gain on hard cases exceeds a penalized regression on a preservation set of initially-correct examples yields higher final accuracy than unfiltered textual-gradient optimization, which degrades on some reasoning tasks below the zero-shot prompt.