arxiv-2609-00727 SRC-0466 · paper

Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models

Bhuvan Koduru, Dareen Safar B Alharthi, Rita Singh, Bhiksha Raj

Created: 2026-09-01 · Ingested: 2026-10-17

Audio language models are designed to understand speech, yet it remains unclear whether they capture how something is said beyond what is said. We present a mechanistic analysis of paralinguistic information in four open source models, Whisper-large-v2, Qwen2-Audio-7B Instruct, Qwen2.5-Omni-7B, and Chroma-4B, using the Expresso dataset with controlled speaking styles. We combine centered kernel alignment, linear probing with leave one speaker out evaluation, open ended tone prediction, and a content prosody leakage metric to trace how style information moves from the audio encoder to the final

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsIn encoder-projector-decoder audio-language models, speaking style is linearly decodable from late audio-encoder layers at about 82-85% on a 7-way style task, yet open-ended tone prediction at the output reaches only about 20-54%, and which end of that range a model lands on tracks its training objective rather than its encoder: a model trained to generate speech keeps acoustic sensitivity through the decoder while text-only-trained siblings with a nearly identical encoder answer from the words instead.