eeg-spike-neuroscience-classification-benchmarks-multi-agent-llm-pipeline-planning CLM-0562On EEG and spike neuroscience classification benchmarks, a multi-agent LLM pipeline with planning, code generation and closed-loop retry does not by itself beat a single-shot LLM coding agent; the measured gain comes from making domain-specific pre-trained neural models addressable through one interface and choosing among them per dataset.
Capability: Competence in a specialist domain
Observed on
Three public datasets (Alzheimer's EEG, Parkinson's EEG, working-memory spikes), nine frozen pre-trained neuroscience models used as feature extractors, GPT-5.2 agents, 8 seeds per.
Sources
- Validated by: Primary metric (macro F1 or balanced accuracy) over 8 trials against general agents, scientific agents and two single-shot LLM coding baselines; plus a 'Vanilla' arm (same architecture, pre-trained models removed), whose initial round scored below Codex on AD (49.80 vs 51.01) and PD (64.52 vs 65.52), and a held-out fixed-model comparison on WM (POYO-1 +6.82 pp over handcrafted features, better on 7 of 8 seeds).. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For LLM data-preparation pipelines in specialist domains (math, finance, law, medicine), diagnosing a small pilot set of generated samples against domain-selected quality criteria and rewriting the individual pipeline components (chunking, instruction builder, response generator) before full-scale generation yields training data with higher downstream fine-tuning utility than post-hoc filtering or one-shot agent/skill generators using the same backbone — but the gain is non-monotonic in rounds, peaking at two and degrading after.Improving its own scaffolding from failures · reviewed by AI
- For LLM agents that must compose and then execute domain tool workflows, training separate adapters for the two roles splits the gains cleanly: supervised fine-tuning of the planner on reference tool sequences mainly improves workflow-order fidelity, while node-level RL on argument-match rewards mainly improves tool-argument grounding and final-answer quality, and combining both beats either on a 9B backbone.Planning a sequence of actions before acting · reviewed by AI
- In multi-agent LLM pipelines over reasoning and code tasks with a pool of 3B-32B open-weight backbones, conditioning each per-step routing decision on a learned filtered memory rather than the full accumulated trajectory leaves accuracy about the same while cutting inference cost by roughly 40-50%, so most of the accumulated history is cost without accuracy value.Getting more from several models than from one · reviewed by AI
- When a tool-using agent is trained with outcome-only RL and a surface prompt cue co-occurs with one tool in most training examples, the agent invokes that tool on tasks where it is useless, but only for tools it has already learned to use well and mainly when the cue is semantically related to the tool.Using the tools it is given · reviewed by AI
- In LLM multi-agent systems whose communication graph is produced by a learned topology generator, the generated graph carries edges that are not needed: scoring each edge by how much masking it changes the task outcome and the semantic entropy of the final answer, then keeping a top-ranked budgeted subgraph, cuts online tokens by about 20-25% while leaving average accuracy equal or slightly higher, and a small distilled predictor of those subgraphs transfers to generators and agent counts it was not calibrated on.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An orchestration-only arm (no domain pre-trained models in the tool registry) matching or beating single-shot LLM coding agents on the same benchmarks and seeds, or the per-dataset selection giving no gain over always using one fixed pre-trained model.