multi-agent-pipelines-knowledge-intensive-qa-8b-backbone-deciding-collaboration-topology CLM-0615For multi-agent pipelines on knowledge-intensive QA with an 8B backbone, deciding the collaboration topology after retrieval and conditioning it on the retrieved evidence yields higher accuracy than generating the topology from query semantics alone with the same retrieved evidence available at execution.
Capability: Getting more from several models than from one
Observed on
Qwen-3-8B agents and topology generator, top-3 evidence from a Wikipedia-derived provenance knowledge graph, topologies bounded to 6 nodes; gains concentrated on knowledge-intensiv.
Sources
- Validated by: Controlled GPQA comparison holding backbone and top-3 retrieved evidence fixed with the verifier disabled: no topology 39.22%, query-conditioned G-Designer 45.34%, evidence-conditioned 50.00% (50.75% with the KG verifier); also 78.68% vs 70.02% average over seven benchmarks and over 50% fewer tokens than LLM-Debate on GPQA. Topology size grows from 2.02 to 2.66 nodes when retrieved evidence is conflicting or low-relevance (100 hand-labelled StrategyQA cases).. Measured on seven benchmarks with one backbone; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On knowledge, math and medical QA benchmarks with 8B open backbones (Llama-3.1-8B-Instruct, Qwen3-8B), the accuracy advantage of multi-agent debate pipelines over a single agent can be matched by a single-turn single agent whose hidden state is steered by a query-composed mixture of role vectors, at roughly a twentieth of the token cost, so the benefit of these pipelines comes from multi-perspective specialization rather than from inter-agent text exchange.Getting more from several models than from one · reviewed by AI
- For agentic retrieval-augmented multi-hop QA with 7B-class policy models, scoring each intermediate step for both logical validity and grounding in the retrieved passages — rather than scoring steps by the final answer — yields higher EM/F1 than outcome-reward and final-answer-based process-reward training, because it removes preference pairs where flawed retrieval happened to reach the right answer.Carrying a chain of inference through to its conclusion · reviewed by AI
- In multi-agent debate with 3 open-weight agents over 3 rounds, giving each agent a memory of past debates helps only if retrieval is conditioned on the current consensus ratio and peer responses are annotated with memory-derived reliability labels; randomly inserting past cases is the weakest retrieval policy and plain task-similarity retrieval trails every consensus-conditioned variant.Getting more from several models than from one · reviewed by AI
- For multi-hop QA over 28k-896k token haystacks with frozen Qwen2.5 backbones (7B-72B), summarizing the document in a parallel hierarchical tree — overlapping windows scanned ~5x, raw segments scored 'critical' passed straight to the final answer, and one summarization channel per decomposed sub-question — scores higher than sequential memory-agent and BM25 retrieval baselines without training, and beats an RL-trained memory agent on the out-of-distribution dataset while losing to it in-distribution.Losing information in long inputs · reviewed by AI
- In a hierarchical multi-agent system where a controller gates acceptance of worker outputs on semantic entropy and semantic density computed over resampled completions, the intervention helps only where failures come from semantic instability (vague or contradictory outputs) and hurts where they come from evidence selection or reasoning-chain errors \u2014 gaining on ambiguity/hallucination-prone benchmarks while losing 10-14 points on multi-hop QA.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: A controlled comparison on knowledge-intensive QA where, with backbone and retrieved passages held fixed, an evidence-conditioned topology generator performs no better than a query-conditioned one, or where the reported gap is reproduced by the extra training on execution-scored topologies rather than by conditioning on evidence.