llm-agents-dependent-multi-step-tasks-success-decays-geometrically-number CLM-0584For LLM agents on dependent multi-step tasks, success decays geometrically with the number of steps under a single per-step reliability parameter that rises with model scale but stays below 1, so long horizons eventually collapse even for the strongest deployed systems — and the decay tracks step count, not context length, since bounding the context window steepens the decay rather than easing it.
Capability: Completing tasks that take many steps or many hours
Observed on
Nine models (six open, 1.2B-671B; three proprietary deployed systems), four task families including an agentic tool-use loop, five horizons, three context regimes; 10,664 analyzed .
Sources
- Validated by: The geometric law fit across nine models and five horizons; projected success falls from 0.42 at GAIA-length horizons to 0.24 at hundred-step horizons. Not validated as an intervention that improves agents.. Measured in a controlled study varying horizon, model, task family and context regime over 10,664 trajectories; the context-regime contrast is reported as a logit slope difference (-0.69 bounded vs -0.44, p=3e-6). [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For LLM agents whose steps condition on their own prior outputs, per-step confidence that is calibrated against step-local success labels does not compose into a calibrated estimate of whether the whole trajectory succeeds, because correlated step errors make the product of step marginals diverge from joint success probability, and the divergence grows as prefix reliability falls over longer horizons.Saying it does not know · reviewed by AI
- In multi-agent LLM workflows with a mixed-size model zoo, choosing which model instantiates each agent step-by-step from an online estimate of task-progress gain plus remaining time and cost budget keeps task quality near the best single-model policy at lower energy than one-shot query-level routing or stall-triggered escalation.Getting more from several models than from one · reviewed by AI
- For long-horizon LLM agents on tool-use and terminal benchmarks, token-level confidence (predictive entropy, sequence probability) barely separates successful from failed trajectories, and the separable signal lives in the dependency structure of the trajectory: scoring a trajectory by propagating per-step uncertainty over a graph of reasoning, tool-call and environment-feedback nodes raises failure-detection AUROC over both local-confidence and sequence-based agent UQ baselines.Using the tools it is given · reviewed by AI
- For LLM agents on long-horizon web penetration testing, holding the backbone model fixed and moving global state out of the context window onto a persistent graph of verified facts and fact-conditioned intent edges raises task success most on the hardest tasks, where success depends on facts obtained dozens of steps earlier (88.2% overall and 75.0% hard, vs 44.1% and 25.0% for the strongest shared-plain-text-memory baseline).Completing tasks that take many steps or many hours · reviewed by AI
- In LLM multi-agent systems that reflect after failure, restricting reflection to the single agent responsible for the first incorrect step — located by a trained step-level correctness scorer and paired with a counterfactually generated corrected step — raises task success more than having every agent reflect on the trajectory, because reflections written by agents that behaved correctly enter their memory as misleading guidance for later tasks.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A model whose success does not fall geometrically with dependent step count, or whose per-step reliability reaches ~1 on the agentic loop; or a setting where shrinking the context window flattens rather than steepens the decay.