two-agent-extract-then-answer-pipeline-multilingual-extractive-qa-routing-inter-agent-handoff CLM-0503In a two-agent extract-then-answer pipeline on multilingual extractive QA, routing the inter-agent handoff through English (forward-translate facts, answer in English, back-translate) lowers Exact Match relative to keeping handoffs in the user's language, and the loss grows with distance from English (13.0 points Spanish up to 30.6 points Hindi with Aya-23-8B).
Capability: Getting more from several models than from one
Observed on
One 8-bit quantized 8B multilingual model (Aya-23-8B), XQuAD, 300 items per language, four languages, zero-shot prompting, three-agent English-forced condition including back-trans.
Sources
- supportsTranslating the Translator: Decomposing the Cost of English-Forced Inter-Agent CommunicationValidated by: Exact Match and F1 on 300 XQuAD items per language for Spanish, Arabic, Chinese, Hindi, native-handoff condition versus an otherwise identical English-pivot condition; all four paired differences significant under exact McNemar with Bonferroni correction, discordant pairs strongly asymmetric.. Measured on one model, one dataset, four languages, 300 items each; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On paired solvable/unsolvable math problems translated into French and Greek, natively multilingual models (Gemma-4-31B, Qwen3-4B/30B) encode solvability most sharply in English yet agree least with that internal belief in their stated verdict in English, being more faithful in the lower-resource languages, while a weakly multilingual model (Llama-3.1-8B) shows the opposite ordering.Saying what actually drove the answer · reviewed by AI
- On 1,010 counterfactual edits rooted in Chinese cultural entities and aligned with English counterparts, an edit made in one language largely does not propagate to the other: edits succeed in the edited language (about 80% for Chinese at batch size 10, falling with batch size) while the other language keeps answering the old fact, so multilingual models store the same fact in language-specific ways that editing does not bridge.Absorbing and flexibly using new knowledge · reviewed by AI
- When instruction-tuned models (Qwen2.5 7B-32B) make function calls from non-English user requests, they often select the right tool and argument names but emit argument values in the wrong language, a failure that exact-match or AST tool-calling metrics record as a generic invocation error rather than a language mismatch.Using the tools it is given · reviewed by AI
- For agentic retrieval-augmented multi-hop QA with 7B-class policy models, scoring each intermediate step for both logical validity and grounding in the retrieved passages — rather than scoring steps by the final answer — yields higher EM/F1 than outcome-reward and final-answer-based process-reward training, because it removes preference pairs where flawed retrieval happened to reach the right answer.Carrying a chain of inference through to its conclusion · reviewed by AI
- For a domain-specific language essentially absent from pretraining corpora (FORM symbolic algebra), QLoRA fine-tuning an 8B model on a few thousand examples validated by executing them through the language's own interpreter beats frontier open-weight models of up to 756B parameters given syntax documentation in context, on execution rate and strict output match for the larger benchmarks, while general reasoning scores drop under 3 points.Generating and editing working code · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An equivalent controlled comparison on other multilingual models or tasks where English-pivot handoffs match or beat native-language handoffs, or where the gap does not widen with typological distance.