large-audio-language-models-answering-binary-sound-x-present-questions CLM-0204In large audio-language models answering binary "is sound X present" questions, the yes/no outcome is fixed at the first decoding step and the model's yes-minus-no margin is positive for most clips even with a silent input, so penalizing affirmative tokens at that single step whenever real audio fails to raise the margin above a small threshold increases correct rejection of absent events beyond fixed-strength contrastive decoding, at the cost of lower recall on genuinely present events and lower precision.
Capability: Stating false facts confidently
Observed on
Two instruction-tuned audio-language models (Qwen2-Audio-7B-Instruct, Gemma-3n-E4B-it) on binary yes/no audio QA (AudioCaps-Hallucination, Clotho-AQA); requires access to logits an.
Sources
- Measured on two models across three AudioCaps-Hallucination splits plus Clotho-AQA, against default decoding, AAD, and AVS baselines. F1 gains over AAD of 0.059-0.117 (Qwen2) and 0.025-0.064 (Gemma); on Clotho-AQA only 0.810 to 0.816 on Qwen2 and slightly below AAD on Gemma. Confusion matrices show true-No recall rising 0.266 to 0.858 while true-Yes recall falls 0.920 to 0.844, and precision drops on Gemma, so the gain is partly a shift toward answering no. Hyperparameters tau and gamma were not ablated; AVS still wins on some splits.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For large audio language models on multiple-choice audio reasoning benchmarks (MMAR, MMAU-mini), accuracy does not track whether the model's chain of thought is acoustically grounded or causally drives the answer: several models answer correctly from perception steps that assert unsupported labels, and for some models replacing the reasoning chain with vacuous or fabricated content leaves the final answer unchanged.Understanding speech and audio · reviewed by AI
- For spoken language models answering questions over long dialogue histories, converting the history to transcripts beats keeping it as audio when the answer is a lexical detail (exact slot values across dense turns: mean 71.3% vs 42.6%), but loses when the answer depends on how something was said, where audio history wins for six of seven models and by a larger margin once lexical emotion cues are removed.Understanding speech and audio · reviewed by AI
- When large audio language models are asked to return every time span matching a natural-language query, they almost always report fewer intervals than exist: across 21 systems on queries averaging about 4 ground-truth intervals, under-reporting rates run from 79% to 100% and no system exceeds 13.2% exact count accuracy, even for the models with the best overlap scores.Understanding speech and audio · reviewed by AI
- On multi-session dialogue questions whose answers depend only on how an utterance was delivered, a cascaded ASR-then-LLM pipeline (Whisper large-v3 into a frontier model) scores below 7B audio-native models on the same clips, and below even a transcript-only baseline, because the transcription step drops delivery cues entirely and adds content errors.Understanding speech and audio · reviewed by AI
- When vision-language models are given an explicit abstention option plus a stated payoff for correct, wrong and declined answers, quadrupling the penalty for a wrong answer barely changes how often they answer, so their abstention rate is largely insensitive to the stated cost of error across text, synthetic video and natural video recall tasks.Saying it does not know · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Show that the silent-reference yes/no margin is not label-agnostically positive, or that gating at the first step gives no F1 gain over fixed-strength contrastive decoding, or that the gain does not come with a drop in true-Yes recall. Proposed technique, not catalogued: first-step confidence-gated affirmative-token penalty against a silent-audio reference.