llm-agents-classifying-nodes-text-attributed-graphs-serializing-sampled-neighborhoods CLM-0414For LLM agents classifying nodes in text-attributed graphs, serializing sampled neighborhoods into prompt text makes the prediction depend on the arbitrary order of neighbors; replacing that text with a fixed-size, permutation-invariant vector of graph statistics (degree, PageRank, clustering, cross-community fraction, label entropy, majority-label fraction) plus only the top-k semantically relevant node texts raises accuracy and cuts per-message tokens.
Capability: Getting more from several models than from one
Observed on
Node classification on four text-attributed graphs (OGB-Arxiv, OGB-Products, Cora-full, Amazon-Computers) with GPT-4o-mini agents, training-free, one agent per detected community..
Sources
- Validated by: Accuracy and Macro-F1 against single-agent and orchestration-based AGL baselines on four datasets (best accuracy on all four); an ablation that restores ReaGAN-style verbalized neighbors while keeping the numeric signature for collaboration routing produced the largest accuracy drop of four ablations; tokens per cross-agent message compared against serialized 1-hop/2-hop neighborhoods.. Measured, one paper, one backbone (GPT-4o-mini), 1,000 test nodes per dataset. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In LLM multi-agent systems whose communication graph is produced by a learned topology generator, the generated graph carries edges that are not needed: scoring each edge by how much masking it changes the task outcome and the semantic entropy of the final answer, then keeping a top-ranked budgeted subgraph, cuts online tokens by about 20-25% while leaving average accuracy equal or slightly higher, and a small distilled predictor of those subgraphs transfers to generators and agent counts it was not calibrated on.Getting more from several models than from one · reviewed by AI
- In fully connected networks of LLM agents (Llama 3.2-8B, 50 agents, 10 rounds) where a fixed minority holds an unchanging extreme stance, neutral agents' textual justifications converge on the biased agents' vocabulary faster and more completely than their numeric stances do, so monitoring only the text overstates how much the group actually agrees.Getting more from several models than from one · reviewed by AI
- When a long chain of thought is verified by classifying a graph of its steps, the predictive signal comes almost entirely from the semantic content of the steps, not the graph shape: replacing step embeddings with purely structural metadata (child/parent counts, chain index, position) drops accuracy to near chance on math, science and multiple-choice reasoning, while remaining above chance only on code.Carrying a chain of inference through to its conclusion · reviewed by AI
- In simulated networks of LLM agents where one or two agents are instructed to advocate a position, the messages other agents write understate how far their elicited stance has moved: they mirror the persuader's rhetorical framing in most messages while explicit stance-change language is rare, so monitoring only generated text misses belief movement that a repeated token-probability probe detects.Getting more from several models than from one · reviewed by AI
- Verifying a chain of thought by decomposing it into nodes and asking binary consistency questions per node localizes errors better than judging the whole chain at once (F1 up 10-25% over CoT-based verifiers, error-localization accuracy far above an 8-vote end-to-end judge) while using 6-58 times fewer tokens, because whole-chain judging dilutes attention and multi-sampling pays for long generations.Carrying a chain of inference through to its conclusion · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: Show that shuffling neighbor order in verbalized graph evidence leaves predictions and accuracy unchanged for current models, or that the signature-based observation loses accuracy relative to full verbalization once the same total token budget is allowed.