react-agents-appworld-resampling-each-recorded-inference-point-offline CLM-0628observationsingle paperReviewed by AI
For ReAct agents on AppWorld, resampling each recorded inference point offline (N=30, temperature 0.5) to find steps whose responses vary across samples, then writing short natural-language guidelines targeting those steps into retrievable episodic memory, raises the fraction of tasks passing all five repeat runs by +16 points same-task and +13 points similar-task with GPT-4.1 (+6 and +8.7 with GPT-OSS-120B) without lowering mean per-run accuracy.
Capability: Improving its own scaffolding from failures
Observed on
ReAct agent, AppWorld test_normal 168 tasks, GPT-4.1 and GPT-OSS-120B on hosted endpoints; guidelines generated from a single baseline trajectory per task, no validation gate befor.
Model cards: GPT-4
Sources
- Validated by: Pass^5, Mean@5 and normalized consistency over five runs per task on AppWorld test_normal (168 tasks, 336 similar-task evaluations), against the same agent with no guidelines; trajectory consistency also scored as a pass/fail predictor (AUROC 0.699 on 50 test_challenge tasks).. Measured on one benchmark, one agent architecture, two models, five runs per task; no ablation isolating the consistency-targeting signal from ordinary trajectory-derived memory, and no matched-token-cost baseline (resampling costs ~30x tokens). [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For a small agent policy (Qwen3.5-4B) repeatedly deployed on the same task family with environment-level feedback, distilling only the trajectories whose textual-state guidance is newly or persistently useful relative to state-free execution raises the state-free policy's accuracy more than distilling all state-conditioned trajectories, and the resulting policy keeps improving over alternating rounds of state editing and consolidation.Improving its own scaffolding from failures · reviewed by AI
- For LLM agents under a fixed ReAct harness answering questions over executable database-backed APIs plus document retrieval, accuracy falls sharply as the required reasoning depth grows from 1 to 5 hops (most models lose over half their accuracy), and the surviving failures concentrate in language-mediated steps — entity disambiguation, cross-source grounding, schema alignment — rather than in emitting syntactically valid tool calls.Carrying a chain of inference through to its conclusion · reviewed by AI
- For 7B multimodal models on long-video QA, a multi-turn agentic exploration harness (adaptive viewing speed, cross-turn textual notes, rewatch) lowers accuracy below plain uniform-frame single-pass answering unless the model is trained to use it, because the untrained model commits to an answer before gathering evidence; with cold-start SFT on verified exploration turns plus trajectory-level RL, the same harness beats the single-pass baseline.Reasoning about time in video · reviewed by AI
- For LLM agents that score their own actions during exploration in text-based games and then reuse that experience, the accuracy of those self-scores against the environment verifier does not predict later gains on held-out stricter configurations: block-level correlations between judgment agreement or calibration and the next checkpoint's score change sit near zero, so recognizing a good action is not the bottleneck — converting it into a transferable policy is.Improving its own scaffolding from failures · reviewed by AI
- For multi-turn agentic RL on sparse-outcome text environments (ALFWorld, WebShop) with Qwen2.5-1.5B/7B-Instruct, reading state and action values from two reserved vocabulary logits at the causal boundaries of the policy's own backbone, and assigning credit with turn-level GAE plus batch normalization, raises task success over PPO with a separate critic and over GRPO while sampling only one rollout per task.Completing tasks that take many steps or many hours · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Guidelines derived from steps flagged by resampling raise Pass^5 no more than guidelines derived from randomly chosen steps, or from outcome-only trajectory extraction, at matched guideline count.