20-year-deterministic-football-management-simulation-hidden-state-roughly-340-400-decision CLM-0524In a 20-year deterministic football-management simulation with hidden state and roughly 340-400 decision stops, what separates frontier models' final scores is managerial behavior rather than deliberation volume: cutting slow-payoff investment as the horizon ends (Spearman -0.58 with late-minus-mid investment shift), keeping cash deployed rather than idle (-0.50), and opening contract renewals early (+0.45) track the score on every seed, while token spend is uncorrelated under every accounting (r_s -0.19, all p > 0.38).
Capability: Completing tasks that take many steps or many hours
Observed on
15 frontier models, three seeds, one 20-year simulated management environment with a deterministic composite score; each stop is a fresh conversation with only a self-written noteb.
Sources
- Measured over 15 models on three seeds in one environment; correlations are rank correlations over n=15, so confidence intervals are wide and the signs are reported per seed rather than tested jointly. Single environment, so domain generality is untested.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In a 20-year deterministic football-management simulation with hidden state, adaptive rivals and 340-400 decision stops, final score across 15 frontier models is not ordered by model scale, price, vendor or token spend, and early standings barely predict the final order (rank correlation with the final order about 0.19 at year 5, rising to 0.78 by year 15), so short-horizon measurements would rank a different set of models.Completing tasks that take many steps or many hours · reviewed by AI
- In a deterministic 365-day e-commerce business simulation of 1,000-4,000 agent steps, end-of-year assets do not predict the six underlying operating competences: across 18 frontier and open-weight models the top earner ranked 16th of 18 on fraud avoidance and below another model on profit per tool call, and the best negotiator finished mid-pack on assets.Completing tasks that take many steps or many hours · reviewed by AI
- In long-horizon tool-mediated environments where strategically relevant state is available only on explicit query (300+ turn Civilization VI via MCP, 23 runs, four model families under a shared playbook), agents allocate only about 1-2% of non-infrastructure tool calls to strategic monitoring and execute only about 48-66% of the near-term commitments they state in their own planning diaries, even though the playbook told them to query victory progress every 20 turns.Completing tasks that take many steps or many hours · reviewed by AI
- For long-horizon embodied task planning where a symbolic model defines action preconditions and effects, correcting a violated plan by edits the action model itself implies (inserting the missing navigate or open) raises plan success more per LLM call than returning the violation to the LLM for replanning, and the gain persists for frontier models as well as compact ones.Planning a sequence of actions before acting · reviewed by AI
- When training an LLM with policy-gradient RL on tasks that decompose into many dependent sub-problems, rewarding each trajectory segment by how many reference reasoning points it newly reaches beats a sparse final-answer reward by a margin that widens with task horizon, and still learns on problem sets where the base policy never samples a success and sparse reward gives no gradient at all.Completing tasks that take many steps or many hours · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A replication on more seeds or a comparable long-horizon management environment where token spend correlates with final score, or where endgame-investment cuts and idle-cash ratio show no consistent-sign correlation. Drafted stance toward 20-year-deterministic-football-management-simulation-hidden-state-adaptive-rivals-340-400: supports -- Same benchmark; this adds that behavioral metrics, not token spend or scale, correlate with the final score.