single-agent-procedural-tasks-where-relevant-state-fits-fixed-schema CLM-0636For single-agent procedural tasks where the relevant state fits a fixed schema known in advance, replacing the appended conversation transcript with a structured execution state that the model rewrites each step — discarding the reasoning trace once the state patch is validated — holds accuracy flat as the horizon grows while history-appending runtimes degrade, and the gain comes from the structure rather than the shorter prompt.
Capability: Completing tasks that take many steps or many hours
Observed on
Single-agent procedural execution with a per-domain schema authored in advance; fails where the schema must be discovered during execution, where an observation's relevance is reco.
Sources
- Validated by: On a 500-shelf warehouse simulation with Gemini-3-Flash over 5 seeds, accuracy at T=200 was 0.94 vs 0.74 (ReAct), 0.84 (summary memory) and 0.88 (LangGraph-style structured state plus full transcript), with 122k vs 2.6-6.2M cumulative tokens; at T=100 under a matched ~1,800-token budget it scored 0.94 vs 0.18 sliding window, 0.52 summary-capped and 0.22 LLMLingua. Also highest pass rate on InterCode CTF (54.2% vs 46.4% best baseline) and both tau-Bench domains.. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For LLM agents on long-horizon web penetration testing, holding the backbone model fixed and moving global state out of the context window onto a persistent graph of verified facts and fact-conditioned intent edges raises task success most on the hardest tasks, where success depends on facts obtained dozens of steps earlier (88.2% overall and 75.0% hard, vs 44.1% and 25.0% for the strongest shared-plain-text-memory baseline).Completing tasks that take many steps or many hours · reviewed by AI
- For long-horizon tool-calling agents (tasks needing five or more calls), embedding state-transition cues — preconditions, invariants, completion states — into the descriptions of the tools on the intended chain is what actually steers the agent's trajectory; runtime corrective text appended to tool results only patches residual drift, and the plausible user prompt alone (persona, deadlines, format constraints) does not establish the trajectory at all.Following instructions hidden in data · reviewed by AI
- For LLM agents under a fixed ReAct harness answering questions over executable database-backed APIs plus document retrieval, accuracy falls sharply as the required reasoning depth grows from 1 to 5 hops (most models lose over half their accuracy), and the surviving failures concentrate in language-mediated steps — entity disambiguation, cross-source grounding, schema alignment — rather than in emitting syntactically valid tool calls.Carrying a chain of inference through to its conclusion · reviewed by AI
- For LLM web agents on cluttered pages containing many concurrent interface components, success falls sharply as component count grows, and having the agent first emit a localized sub-goal plus a task-relevant sub-observation before choosing an action recovers much of the loss, because self-assessment of the action is more reliable in the reduced context.Using the tools it is given · reviewed by AI
- In multi-agent LLM pipelines over reasoning and code tasks with a pool of 3B-32B open-weight backbones, conditioning each per-step routing decision on a learned filtered memory rather than the full accumulated trajectory leaves accuracy about the same while cutting inference cost by roughly 40-50%, so most of the accumulated history is cost without accuracy value.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A matched-budget comparison on long-horizon tasks where a transcript-based or compression-based runtime equals or beats the state-rewriting runtime on accuracy, or where the structured-state gain disappears once prompt length is held equal. Drafted stance toward llm-agents-long-horizon-web-penetration-testing-holding-backbone-model: supports -- Both find that moving global state out of the growing context onto an explicit external structure raises success most on the longest-horizon tasks.