arxiv-2609-22851 SRC-0516 · paper

Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs

Created: 2026-09-19 · Ingested: 2026-10-17

Large Audio Language Models (LALMs) utilize either continuous features or discrete tokens, yet the optimal representation paradigm for general audio understanding remains debated. Existing benchmarks often focus on narrow domains or evaluate encoders outside LALM contexts. To address these gaps, we systematically evaluate continuous and discrete representations across speech, sound and music. Utilizing our UniARC framework with dual evaluation strategies across model scales from SmolLM2-135M to Llama-3-8B, we analyze the dynamic relationships of data volume, model capacity, and computational e

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsIn audio language models that feed encoder outputs into a text LLM, downstream understanding accuracy across speech, sound and music is set by how semantically text-aligned the encoder is rather than by whether its representation is discrete or continuous — reconstruction-oriented codecs with high acoustic fidelity underperform, semantically distilled tokenizers can beat their own continuous teacher, and scaling the LLM backbone does not recover the lost information.