multi-agent-llm-pipelines-over-reasoning-code-tasks-pool-3b-32b CLM-0446In multi-agent LLM pipelines over reasoning and code tasks with a pool of 3B-32B open-weight backbones, conditioning each per-step routing decision on a learned filtered memory rather than the full accumulated trajectory leaves accuracy about the same while cutting inference cost by roughly 40-50%, so most of the accumulated history is cost without accuracy value.
Capability: Getting more from several models than from one
Observed on
Closed-domain verifiable tasks (MATH, GSM-Hard, MBPP, HumanEval, MMLU-Pro), router trained with group-relative policy gradients, frozen 3B-32B open-weight backbones, max depth 6; n.
Sources
- supportsLearning What to Retain: Gated-Memory Routing for Efficient Collaboration in Multi-Agent LLM SystemsValidated by: Accuracy and token cost on five benchmarks against query-only routing (MASRouter), full-history routing (Puppeteer), fixed topologies (MacNet), AFlow; best average accuracy, +2.44 points over strongest baseline, -31.9% HumanEval cost; leave-one-out ablation of each module; three seeds reported in appendix.. Direct three-paradigm comparison on two benchmarks (GSM-Hard 70.27 vs 70.55; HumanEval 89.06 vs 89.84) with cost ~40-50% lower; own implementation of the full-history arm, cost measured in tokens with parameter-count price proxy; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In multi-agent LLM workflows with a mixed-size model zoo, choosing which model instantiates each agent step-by-step from an online estimate of task-progress gain plus remaining time and cost budget keeps task quality near the best single-model policy at lower energy than one-shot query-level routing or stall-triggered escalation.Getting more from several models than from one · reviewed by AI
- For LLM agents that must compose and then execute domain tool workflows, training separate adapters for the two roles splits the gains cleanly: supervised fine-tuning of the planner on reference tool sequences mainly improves workflow-order fidelity, while node-level RL on argument-match rewards mainly improves tool-argument grounding and final-answer quality, and combining both beats either on a 9B backbone.Planning a sequence of actions before acting · reviewed by AI
- For single-agent procedural tasks where the relevant state fits a fixed schema known in advance, replacing the appended conversation transcript with a structured execution state that the model rewrites each step — discarding the reasoning trace once the state patch is validated — holds accuracy flat as the horizon grows while history-appending runtimes degrade, and the gain comes from the structure rather than the shorter prompt.Completing tasks that take many steps or many hours · reviewed by AI
- For routing a video-production request to an ordered chain of registered sub-agents, an 8B open model fine-tuned on descriptor-conditioned routing plans (SFT on verified plans plus GRPO rewarding validity, sub-agent coverage F1 and dependency-consistent order) picks chains more accurately than much larger prompted planners, with the gap widest on long, structurally unfamiliar chains.Getting more from several models than from one · reviewed by AI
- For LLM agents on long-horizon web penetration testing, holding the backbone model fixed and moving global state out of the context window onto a persistent graph of verified facts and fact-conditioned intent edges raises task success most on the hardest tasks, where success depends on facts obtained dozens of steps earlier (88.2% overall and 75.0% hard, vs 44.1% and 25.0% for the strongest shared-plain-text-memory baseline).Completing tasks that take many steps or many hours · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A comparison in which full-history routing is meaningfully more accurate than gated-memory routing at matched depth, or in which gated memory's cost saving disappears once prompt lengths are measured properly.