multi-agent-pruning-methods-agentprune-agentdropout-agp-dagp-gptswarm-re-run CLM-0447When multi-agent pruning methods (AgentPrune, AgentDropout, AGP, DAGP, GPTSwarm) are re-run under a fixed backbone, agent registry, role-to-node assignment and runtime, their accuracy-cost gains do not hold across starting topologies and mostly do not beat random pruning at the same structural budget — averaged over 20 cells the optimized systems lost 0.77 accuracy points against the original system versus 0.44 for matched random pruning, and only 12 of 25 method-topology pairs beat their random control.
Capability: Getting more from several models than from one
Observed on
Five compression-style MAS optimizers, four task families (GSM8K, MMLU-500, HumanEval, LiveCodeBench-150), small open-weight 4B-8B backbones, 6 agents, 2 rounds, mostly no tool acc.
Sources
- Validated by: Applied to 20 main-grid cells across five starting topologies and four tasks; the random control preserved accuracy about as well as the learned optimizers (-0.44 vs -0.77 points from the uncompressed system), exposing cells where the full multi-agent structure was not needed at all.. Measured on a 38-cell anchor-plus-one-axis grid with a pinned backbone per task family and a per-source observation filter, so deltas are attributable to the emitted topology. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When RL-training a central-agent multi-agent harness with 1.5B-3B Qwen2.5 backbones on multi-hop QA, optimizing only the coordinating central policy and leaving sub-agents frozen scores higher than jointly training central and sub-agent policies (average F1 47.79 vs 45.36, accuracy 42.88 vs 40.49 at 3B).Getting more from several models than from one · reviewed by AI
- When an agent harness is evolved from execution traces under a fixed interaction budget, verifying each candidate edit on tasks chosen for the behavior it targets and accepting it only when the gain is attributable to that behavior beats scoring every candidate on a shared fixed, random or coreset task split, and larger verification budgets do not compensate: fixed-split methods using up to 24x the rollouts fell below the unmodified harness in half of their harness-benchmark pairs.Improving its own scaffolding from failures · reviewed by AI
- In LLM multi-agent systems whose communication graph is produced by a learned topology generator, the generated graph carries edges that are not needed: scoring each edge by how much masking it changes the task outcome and the semantic entropy of the final answer, then keeping a top-ranked budgeted subgraph, cuts online tokens by about 20-25% while leaving average accuracy equal or slightly higher, and a small distilled predictor of those subgraphs transfers to generators and agent counts it was not calibrated on.Getting more from several models than from one · reviewed by AI
- In multi-agent LLM pipelines over reasoning and code tasks with a pool of 3B-32B open-weight backbones, conditioning each per-step routing decision on a learned filtered memory rather than the full accumulated trajectory leaves accuracy about the same while cutting inference cost by roughly 40-50%, so most of the accumulated history is cost without accuracy value.Getting more from several models than from one · reviewed by AI
- In hours-long ML development runs on Kaggle-style competitions, agent scaffolds each lock into one strategy policy — the single-loop CLI agent pivots far less than humans (9% vs 25% of transitions) and the evolutionary search agent far more (58%) with pivot gains cancelling out — and neither reopens an approach it had abandoned, even though it recovers scores after setbacks as well as top humans.Planning a sequence of actions before acting · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Re-running the same five methods under the same controlled harness and finding they beat matched random pruning in most topology-task cells, with gains that hold across starting topologies and exceed run-to-run variance.