audio-in-context-learning-per-episode-resampled-rules-current-large-audio CLM-0361In audio in-context learning with per-episode resampled rules, current large audio language models bind arbitrary non-speech sounds to new labels reasonably well (best model 58.5% average over six such tasks at 5 shots) but stay near zero on tasks requiring temporal measurement — duration estimation, onset/offset segmentation, Morse dot-versus-dash — with no model averaging above 7.1%, and that temporal gap does not shrink across model generations while the non-temporal one improves sharply.
Capability: Understanding speech and audio
Observed on
Five LALMs (DeSTA2.5-8B, Qwen2-Audio-7B, Qwen2.5-Omni-7B, Qwen3-Omni-30B, Gemini-3-flash-preview), nine artificial/one natural episode-based tasks, exact-match over 300 episodes, u.
Sources
- Measured: per-task exact-match with 95% bootstrap CIs across five models; A2 mean rose 15.6% (Qwen2) to 58.5% (Qwen3) while A3 mean moved 4.4% to 7.1%. A single-character Morse ablation removes compositional demand and Qwen3 still reaches only 11.5%, which isolates perception from decoding. Only five models and one benchmark; tasks are mostly artificial by design.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On audio temporal-localization tasks in 11 Southeast Asian languages, current audio LLMs return time spans whose coverage of the true span consistently exceeds their purity, and F1 falls steadily as the recording grows from under 30 seconds to 3 minutes, with several models unable to run at all on the longer bins.Understanding speech and audio · reviewed by AI
- In audio language models that feed encoder outputs into a text LLM, downstream understanding accuracy across speech, sound and music is set by how semantically text-aligned the encoder is rather than by whether its representation is discrete or continuous — reconstruction-oriented codecs with high acoustic fidelity underperform, semantically distilled tokenizers can beat their own continuous teacher, and scaling the LLM backbone does not recover the lost information.Understanding speech and audio · reviewed by AI
- In training-free video temporal grounding with frozen vision-language embedding backbones, matching a query only against frames sampled at a fixed rate leaves temporal context unused: adding clip-level (sliding-window) similarities projected onto the frame timeline raises localization accuracy even though clip-only similarity is worse than frame-only on its own.Reasoning about time in video · reviewed by AI
- When large audio language models are asked to return every time span matching a natural-language query, they almost always report fewer intervals than exist: across 21 systems on queries averaging about 4 ground-truth intervals, under-reporting rates run from 79% to 100% and no system exceeds 13.2% exact count accuracy, even for the models with the best overlap scores.Understanding speech and audio · reviewed by AI
- In large audio-language models answering binary "is sound X present" questions, the yes/no outcome is fixed at the first decoding step and the model's yes-minus-no margin is positive for most clips even with a silent input, so penalizing affirmative tokens at that single step whenever real audio fails to raise the margin above a small threshold increases correct rejection of absent events beyond fixed-strength contrastive decoding, at the cost of lower recall on genuinely present events and lower precision.Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An audio LLM that scores well above chance on duration estimation and single-character Morse discrimination from in-context demonstrations, while the same benchmark's non-speech binding tasks are unchanged, would show temporal perception is not a separate persistent boundary.