arxiv-2608-08569 SRC-0606 · paper

VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

Wenxu Jia, Dongjie Fu, Xize Cheng, Fangming Feng, Linjun Li, Wenshi Chen

Created: 2026-08-09 · Ingested: 2026-10-24

Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks. Despite this progress, their long-context inference remains severely bottlenecked by prohibitive KV cache memory demands. Existing text-centric compression methods struggle here, often disrupting speech continuity or discarding crucial semantic cues. To address this, we propose VoxZip, a train-free, two-stage semantic-anchored KV cache compression framework. The first stage uses automatic speech recognition (ASR) transcriptions as explicit semantic anchors to temp

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsFor speech LLMs on long audio, text-centric KV cache eviction methods (StreamingLLM, SnapKV, PyramidKV, ChunkKV) collapse because raw audio tokens have diffuse attention; anchoring compression on ASR transcripts — pooling each transcribed audio span down to its text-token length and adding the text embedding, while keeping untranscribed audio intact — retains over 90% of full-cache accuracy at a 5-10% cache budget where those baselines retain roughly 55-67% at a looser 25% budget.