across-41-models-seven-multi-agent-benchmarks-gains-single-agent-capability CLM-0288observationsingle paperReviewed by AI
Across 41 models and seven multi-agent benchmarks, gains in single-agent capability translate only weakly into multi-agent competence (understanding other agents, joint planning, efficient communication, adaptation); on the multi-agent planning task accuracy rose only about 30% across a family's full scale range, so coordination ability does not emerge from scaling alone and needs its own data and training.
Capability: Getting more from several models than from one
Observed on
41 LLMs, seven multi-agent benchmarks, 2025; a position paper backed by a broad evaluation rather than a single controlled study..
Sources
- Wide but shallow evidence; the four-capability decomposition is the authors' framing. Useful as the baseline against which coordination techniques are measured.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Splitting a task across several agents helps on long-horizon tasks whose parts depend on each other sparsely, and hurts on tightly coupled sequential workflows, where generic multi-agent methods fall below a single-agent harness because decomposition fragments the dependency chain and coordination overhead outweighs the context isolation it buys; as single agents improve the collaboration benefit shrinks.Getting more from several models than from one · reviewed by AI
- In embodied multi-agent household planning with a fixed high-level skill interface, a team's task success rate overstates coordination quality, because goal predicates can be reached through illegal shortcuts: several of 11 tested MLLMs succeed on more episodes than they produce legal plans on (Claude Haiku 4.5 63.1% success vs 47.2% legal; Qwen3.6-Plus 69.1% vs 56.7%), while others honour constraints yet miss the goal.Getting more from several models than from one · reviewed by AI
- In hours-long ML development runs on Kaggle-style competitions, agent scaffolds each lock into one strategy policy — the single-loop CLI agent pivots far less than humans (9% vs 25% of transitions) and the evolutionary search agent far more (58%) with pivot gains cancelling out — and neither reopens an approach it had abandoned, even though it recovers scores after setbacks as well as top humans.Planning a sequence of actions before acting · reviewed by AI
- For CAE simulation tasks (OpenFOAM, FEniCS, PyChrono, COMSOL) run with a current general-purpose coding-agent harness and a strong backbone, a single agent with no multi-agent role decomposition and no scripted self-critique matches or exceeds the published scores of multi-agent specialized CAE systems, because the harness itself supplies the multi-turn reasoning and execution-feedback repair the decomposition was built to compensate for.Getting more from several models than from one · reviewed by AI
- For teams of up to 50 heterogeneous embodied LLM agents in simulated wildfire missions, building the manager hierarchy per mission so that concurrently executable work sits under a parallel-allocating manager and prerequisite-chained work under a phase-advancing manager beats fixed decentralized, single-planner, and hybrid coordination architectures on mission score and on progress accumulated per step, and the advantage does not depend on which base language model is used.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A scaling study where multi-agent benchmark scores track single-agent scores closely across families.