llm-agents-score-their-own-actions-during-exploration-text-based CLM-0417mechanismsingle paperReviewed by AI
For LLM agents that score their own actions during exploration in text-based games and then reuse that experience, the accuracy of those self-scores against the environment verifier does not predict later gains on held-out stricter configurations: block-level correlations between judgment agreement or calibration and the next checkpoint's score change sit near zero, so recognizing a good action is not the bottleneck — converting it into a transferable policy is.
Capability: Improving its own scaffolding from failures
Observed on
Seven text-based games with executable verifiers; seven proprietary models; verifier rewards hidden from the agent during exploration; context-level (history or summary) reuse path.
Sources
- Validated by: 116,117 transitions over 98 runs: event agreement and normalized MAE per game, plus blockwise and run-level correlations with normalized score change (rho(A,g) = -0.010, rho(-E,g) = -0.018; run-level r = -0.23 with normalized above-baseline area).. Measured across 7 games and 7 models with an executable verifier as ground truth; correlations are near zero and sometimes slightly negative, but improvement itself is sparse and noisy in several games (many zero scores), which limits how much signal the correlation could show. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In LLM multi-agent systems that reflect after failure, restricting reflection to the single agent responsible for the first incorrect step — located by a trained step-level correctness scorer and paired with a counterfactually generated corrected step — raises task success more than having every agent reflect on the trajectory, because reflections written by agents that behaved correctly enter their memory as misleading guidance for later tasks.Getting more from several models than from one · reviewed by AI
- For LLM web agents on cluttered pages containing many concurrent interface components, success falls sharply as component count grows, and having the agent first emit a localized sub-goal plus a task-relevant sub-observation before choosing an action recovers much of the loss, because self-assessment of the action is more reliable in the reduced context.Using the tools it is given · reviewed by AI
- For a small agent policy (Qwen3.5-4B) repeatedly deployed on the same task family with environment-level feedback, distilling only the trajectories whose textual-state guidance is newly or persistently useful relative to state-free execution raises the state-free policy's accuracy more than distilling all state-conditioned trajectories, and the resulting policy keeps improving over alternating rounds of state editing and consolidation.Improving its own scaffolding from failures · reviewed by AI
- For VLA and world-action manipulation policies that score near-ceiling on standard LIBERO, recovery success after a real execution failure falls sharply as recovery requires deeper state reasoning \u2014 retrying or adapting the immediate action works far more often than restoring a displaced task object or clearing an environmental obstruction \u2014 and rankings on the standard benchmark do not predict recovery rankings.Turning plans into physical action · reviewed by AI
- For LLM multi-agent systems failing on long-horizon web tasks, re-running the whole task from scratch — plain, with self-reflection, or with a critic agent — repairs few failures (at most 6.90% within three attempts, with feedback variants below plain rerun), while reconstructing the recorded execution prefix and intervening once at a symptom-flagged trace node repairs 20.15% with a single attempt.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A setup where agents whose step-level self-judgments agree more closely with verifier rewards show reliably larger held-out improvement under the same experience-reuse pathway.