Completing tasks that take many steps or many hours

Claims

• observationsingle paper↑1↓0
  In a 20-year deterministic football-management simulation with hidden state, adaptive rivals and 340-400 decision stops, final score across 15 frontier models is not ordered by model scale, price, vendor or token spend, and early standings barely predict the final order (rank correlation with the final order about 0.19 at year 5, rising to 0.78 by year 15), so short-horizon measurements would rank a different set of models.
• observationsingle paper↑1↓0
  In a 20-year deterministic football-management simulation with hidden state and roughly 340-400 decision stops, what separates frontier models' final scores is managerial behavior rather than deliberation volume: cutting slow-payoff investment as the horizon ends (Spearman -0.58 with late-minus-mid investment shift), keeping cash deployed rather than idle (-0.50), and opening contract renewals early (+0.45) track the score on every seed, while token spend is uncorrelated under every accounting (r_s -0.19, all p > 0.38).
• observationsingle paper↑1↓0
  In a 241-day simulated crop-management task where decisions accumulate irreversibly, a zero-shot multi-agent LLM framework (perception, manager, decision, revision, reflection) scores below a PPO policy trained in that same environment but far above the same PPO policy transferred to a shifted weather/location/soil environment, while using under 28% of its fertilizer and irrigation.
• mechanismsingle paper↑1↓0
  When 4B-8B LLM agents are trained with plain group-based RL (GRPO) to emit variable-length action chunks in text environments, the policy degenerates — collapsing to one action per round on one backbone and over-committing to 5-6-action chunks with lower success on another — because a sparse terminal reward carries no information about where chunk boundaries belong; supplying boundary targets from subskills segmented out of successful trajectories yields an intermediate 3-4-action regime with both higher success and fewer decision rounds.
• observationsingle paper↑1↓0
  On 170 software and research tasks timed with human domain experts, the length of task that frontier agents complete at 50% reliability, measured in human time, has doubled roughly every seven months since 2019 and was around 110 minutes for o3 in mid-2025 — but the horizon at 80% reliability follows the same trend at roughly five times shorter, and success drops as tasks carry more of the 16 "messiness" factors that separate benchmark tasks from real work, so the headline horizon overstates what an agent can be relied on to finish.
• observationsingle paper↑1↓0
  Across 393 model–benchmark observations in ten capability domains for models released between 2023 and September 2026, the share of benchmark headroom closed is uneven: bounded, readily verified domains are near their ceilings (advanced mathematics 86.4 and graduate-level science 85.8 on a 0–100 index where 0 is the benchmark's entry-year frontier), while interactive, stateful ones lag (software engineering 52.6, search and terminal agents 56.8, tool agents 39.9) — so gains in bounded environments have not transferred uniformly to long, stateful workflows where errors propagate across the trajectory.
• observationsingle paper↑1↓0
  In a deterministic 365-day e-commerce business simulation of 1,000-4,000 agent steps, end-of-year assets do not predict the six underlying operating competences: across 18 frontier and open-weight models the top earner ranked 16th of 18 on fraud avoidance and below another model on profit per tool call, and the best negotiator finished mid-pack on assets.
• observationsingle paper↑1↓0
  In file-based sandbox agent tasks with distractor artefacts, scoring only whether the task goal is met overstates reliability, because agents frequently complete the target while modifying unrelated files: across nine LLM agents, pass rate exceeds a success-plus-preservation metric by roughly 10-17 points, with the gap widening as model capability falls.
• mechanismsingle paper↑1↓0
  For LLM agents on dependent multi-step tasks, success decays geometrically with the number of steps under a single per-step reliability parameter that rises with model scale but stays below 1, so long horizons eventually collapse even for the strongest deployed systems — and the decay tracks step count, not context length, since bounding the context window steepens the decay rather than easing it.
• mechanismsingle paper↑1↓0
  For LLM agents on long-horizon web penetration testing, holding the backbone model fixed and moving global state out of the context window onto a persistent graph of verified facts and fact-conditioned intent edges raises task success most on the hardest tasks, where success depends on facts obtained dozens of steps earlier (88.2% overall and 75.0% hard, vs 44.1% and 25.0% for the strongest shared-plain-text-memory baseline).
• observationsingle paper↑1↓0
  For LLM agents playing thousand-step StarCraft II matches through a high-level text interface, retrieving a handful of time-aligned episodes from a small bank of past winning games and gating between replaying a retrieved action and reasoning afresh over recent observations raises win rate over a summarization-only working-memory baseline at roughly a seventh of the token cost, with the margin widening at the harder difficulty level.
• mechanismsingle paper↑1↓0
  For long-horizon LLM agents in environments where an erroneous action changes world state, restoring an earlier checkpoint and carrying forward a structured reflection memory that excludes branch-local state claims beats either intervention alone, because reflection alone cannot undo environmental damage and rollback alone re-explores the same failure.
• observationsingle paper↑1↓0
  In long-horizon tool-mediated environments where strategically relevant state is available only on explicit query (300+ turn Civilization VI via MCP, 23 runs, four model families under a shared playbook), agents allocate only about 1-2% of non-infrastructure tool calls to strategic monitoring and execute only about 48-66% of the near-term commitments they state in their own planning diaries, even though the playbook told them to query victory progress every 20 turns.
• observationsingle paper↑1↓0
  For multi-turn agentic RL on sparse-outcome text environments (ALFWorld, WebShop) with Qwen2.5-1.5B/7B-Instruct, reading state and action values from two reserved vocabulary logits at the causal boundaries of the policy's own backbone, and assigning credit with turn-level GAE plus batch normalization, raises task success over PPO with a separate critic and over GRPO while sampling only one rollout per task.
• mechanismsingle paper↑1↓0
  For open-source LLM agents driving a ROS robot, having the model emit a pseudo-code tool-call plan up front, storing it in a persistent scratchpad, and then restricting each tool invocation to the plan's next step raises task success most on the longest multi-step tasks and for the smaller open models (GPT-OSS-20B on the hardest task rose from 35% with a persistent plan alone to 55% with the constraint), while a strong proprietary model gains little.
• observationsingle paper↑1↓0
  On repository-grounded LLM-infrastructure engineering tasks, frontier agents score far below ceiling (best 36.53% overall), and every one of eight models scores lower on long-horizon multi-file implementation than on single-file kernel completion, even though the full repository and test cases are visible.
• observationsingle paper↑1↓0
  On research-level theorem-proving tasks drawn from recent FOCS/STOC/SODA papers, wrapping a frontier model in a staged harness that explores strategies, gates when to decompose, solves section-level subproblems along a dependency graph, and routes verifier findings back to the affected section raises graded proof accuracy far above prompting the same model directly (30.3% to 54.0% for Gemini 3.1 Pro on TCS-Bench), and two such runs with different backbones make complementary errors, so selecting between them by sampled critiques reaches 71.0% against an oracle ceiling of 77.3%.
• mechanismsingle paper↑1↓0
  When self-reflection is used as a training signal for long-horizon agent and math tasks, resetting to the original prompt with a compact reflection patch prepended beats accumulating reflections inside the ongoing trajectory, and overly long reflections (more than ~10 points) hurt compared with 2-5 bullet points.
• mechanismsingle paper↑1↓0
  For single-agent procedural tasks where the relevant state fits a fixed schema known in advance, replacing the appended conversation transcript with a structured execution state that the model rewrites each step — discarding the reasoning trace once the state patch is validated — holds accuracy flat as the horizon grows while history-appending runtimes degrade, and the gain comes from the structure rather than the shorter prompt.
• observationsingle paper↑1↓0
  For small open-weight backbones (Qwen3-4B) driving long-horizon remote-sensing tool chains, inserting a step-level verifier that checks each intermediate tool output against schema, sub-goal and domain constraints and triggers replanning is the single largest architectural gain, while hierarchical decomposition without domain-aligned tool grouping adds little and can lower final accuracy relative to a flat agent with the same verifier.
• observationsingle paper↑1↓0
  When training 4B-8B Qwen3 agents with group-based RL (GRPO, DAPO, GSPO) on sparse-reward long-horizon tasks, injecting hints into the environment's feedback — procedural action guidance in early episode steps and early training, switching to enriched state observations in later steps and later training — raises success on the unmodified environments by a few points on average, while lowering performance on tasks that require questioning whether the request is answerable at all.
• mechanismsingle paper↑1↓0
  When training an LLM with policy-gradient RL on tasks that decompose into many dependent sub-problems, rewarding each trajectory segment by how many reference reasoning points it newly reaches beats a sparse final-answer reward by a margin that widens with task horizon, and still learns on problem sets where the base policy never samples a success and sparse reward gives no gradient at all.
• observationsingle paper↑1↓0
  In a year-long multi-agent social simulation driven by a single 4-bit quantized 8B model on a 12 GB consumer GPU with five agents, the run can be carried to roughly 50-52 simulated weeks before the accumulated prompt hits the context limit, while about 10% of finalized activity records are left with at least one missing field.

Techniques

• Action guidance early, observation enrichment latetraining
  Modify the training environment's feedback rather than the agent: give explicit next-action hints in the first steps of an episode and the first half of training, then switch to richer state information in later steps and later training, ke
• Budget reliability per step instead of reporting pass ratesprocess
  Fit a per-step reliability parameter from short-horizon runs and project it onto the target production horizon to predict success, instead of reporting aggregate benchmark pass rates.
• Distill chunk boundaries from trajectory-induced programmatitraining
  Induce two-level programmatic skills (composite skills made of ordered subskill calls) from successful trajectories, expand each subskill call into an action chunk whose boundary becomes an off-policy supervision target, and train the primi
• Externalize agent state onto a fact-intent DAGprocess
  Store verified environment observations as immutable fact nodes in a persistent graph and allow an action only as an intent edge whose preconditions are already-verified facts, replacing in-context state maintenance.
• Gate between replaying retrieved winning episodes and fresh retrieval
  The agent stores successful past gameplay episodes, retrieves the few most similar to the current time and observation, and a gating rule decides whether to execute the retrieved action directly or reason anew by fusing the retrieved episod
• Layered memory compression with four daily activity blocksprocess
  Replaces agent-managed scratchpads with a system-managed three-layer memory (full logs for recent weeks, ~200-word summaries, ~50-word milestones) and splits each simulated day into four plannable blocks.
• Measure proactive monitoring and reflection-action gap from process
  Compute, from an agent's tool-call log, the share of non-infrastructure calls spent querying global state (PMR) and the fraction of commitments stated in its planning diary that are executed within the next K turns (RAG@K).
• Persist a pseudo-code plan and restrict calls to itprocess
  Before any action, the agent generates an ordered pseudo-code sequence of tool calls, writes it to a scratchpad that persists across the whole episode, and the executor admits only the plan-specified tool at each step unless replanning is t
• Replace conversation history with a rewritten execution statarchitecture
  At each step the model sees only the immutable skill spec, the current structured state and the latest observation, emits a JSON state patch plus an action, and has its reasoning trace discarded after the runtime validates and merges the pa
• Require non-target files be byte-identical, not just task goprocess
  Score a task as successful only when all task-specific final-state evaluators pass and every pre-existing non-target file is unchanged between initial and final sandbox state.
• Reward segments by reference reasoning points reachedtraining
  Split each sampled trajectory into segments and give each segment a reward equal to the increase in the count of reference-solution reasoning points it has reached, with credit shortcut so that reaching the goal counts all points.
• Role-split physical agent loop with reflection feedbackprocess
  Splits long-horizon physical task management into deterministic perception agents, a synthesizing manager, a decision agent, an independent revision agent that may only adjust quantities, a deterministic safety guard, and a reflection agent
• Roll back the environment, keep the distilled reflectionprocess
  On a triggered review, an agent restores a coarse-to-fine-selected earlier checkpoint while carrying forward a memory of objective, environment facts, past milestones and failure analysis, deliberately omitting current-state claims.
• Route routine calls to a smaller modelprocess
  Send the short, structured, routine calls in an agent pipeline to a small cheap model and reserve a frontier model for the steps that need deep planning or sustained constraint tracking.
• Score long-horizon agents on separate capability axes, not oprocess
  Alongside the accumulated primary score, measure negotiation surplus, fraud spend share, drawdown over peak, profit per tool call, controllable return rate, and repeat-order price discipline as separate ranked axes.
• Stage the proof: explore, gate, decompose by dependency, re-process
  Allocates inference across explicit research stages — strategy exploration with a readiness gate, decomposition into a dependency graph of section-level subproblems solved in parallel, and global verification whose findings are routed back
• Verify each intermediate step against domain constraints, thprocess
  After every tool call, a manager role re-reads structured global memory and classifies the output as pass/uncertain/fail on schema validity, sub-goal consistency and domain plausibility, replanning or re-routing on failure.

Sources

Every paper, post or observation behind this capability. Ones cited by a claim show the claim and its stance; ones filed directly are about the capability but have no claim drafted from them yet. Open one to add support or contest it.

• Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents2026-09-16
  ↑ supports a claim here
• Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems2026-09-15
  about this capability · no claim drafted yet
• Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science2026-09-14
  ↑ supports a claim here
• DynSTEER: Dynamic Stage-wise Trajectory Evaluation and Execution-time Review for Agents2026-09-13
  about this capability · no claim drafted yet
• Toward Self-Adaptive Physical AI: Can LLM Agents Manage Long-Horizon Physical Tasks?2026-09-11
  ↑ supports a claim here
• Bridging Thought and Action: Taming Long-Horizon Instability in Open-Source LLM Agents with a MetaTool-Enhanced ROS Framework2026-09-11
  ↑ supports a claim here
• Tasks over Application Manuals: Revealing Gaps in Long-Horizon Procedural Reasoning for Language Models2026-09-11
  about this capability · no claim drafted yet
• The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement2026-09-10
  ↑ supports a claim here
• $Φ$-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?2026-09-09
  ↑ supports a claim here
• Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks2026-09-08
  ↑ supports a claim here
• Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing2026-09-07
  ↑ supports a claim here
• Long-Horizon Language Model Reinforcement Learning via Progressive Point Matching2026-09-07
  ↑ supports a claim here
• Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents2026-09-02
  ↑ supports a claim here
• CivBench: A Long-Horizon Benchmark for Tool-Mediated Agents in Civilization VI2026-09-02
  ↑ supports a claim here
• WorldBench: Culturally Grounded Benchmark for Multilingual Agents2026-09-01
  ↑ supports a claim here
• E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation2026-08-31
  ↑ supports a claim here
• How Fast Do Agents Rot? An Empirical Study of Long-Horizon Degradation in LLM Agents for Production Decision-Making2026-08-31
  ↑ supports a claim here
• HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving2026-08-31
  ↑ supports a claim here
• AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing2026-08-30
  about this capability · no claim drafted yet
• SKILL.state: Scalable Long-Horizon Agent Skills2026-08-26
  ↑ supports a claim here
• Agentopia on a Consumer GPU: A Reduced-Scale Long-Horizon Port with an 8B Model2026-08-25
  ↑ supports a claim here
• SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning2026-08-24
  ↑ supports a claim here
• SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning2026-08-20
  ↑ supports a claim here
• FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents2026-08-19
  ↑ supports a claim here · ↑ supports a claim here
• LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning2026-08-12
  ↑ supports a claim here
• Measuring AI Ability to Complete Long Software Tasks2025-03-18
  ↑ supports a claim here