arxiv-2608-20853 SRC-0427 · paperMGAL: A Multilingual Granularity-Aware Long-Context Benchmark
Chunhan Li, Chenglin Xu, Zongyang Zhang, Jiale Liu, Zhuoxi Rao, Xudong Jia
Created: 2026-08-21 · Ingested: 2026-10-17
https://arxiv.org/abs/2608.20853(opens in a new tab)Evaluation of long-context Large Language Models (LLMs) has advanced rapidly. However, most existing benchmarks are limited to the document level and focus mainly on high-resource languages, leaving many fine-grained challenges insufficiently evaluated. To address this gap, we present MGAL, the first multilingual, granularity- and position-aware long-context benchmark. MGAL is constructed from United Nations (UN) reports spanning 8K to 128K tokens across the six official UN languages. It covers four coherent levels of linguistic granularity (word, sentence, paragraph, and document) and further
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
- supportsIn long-context tasks over 8K-128K-token documents, the middle-of-context penalty is granularity-dependent: word-level QA and paragraph-filling accuracy dip when the evidence sits mid-document, but sentence-cloze accuracy peaks in the middle, because mid-paragraph sentences carry concrete entity and date overlap that models match on instead of reasoning about discourse role.
- supportsOn long-context tasks over 8K-128K-token documents in the six official UN languages, models answer word-level span questions well (about 73% average accuracy across 12 models) but degrade sharply as the required unit coarsens to sentence selection (about 21%), paragraph filling and document-level summarization and translation, so long-context competence measured by needle-style word retrieval overstates comprehension of larger discourse units.