post-training-speech-llm-asr-grpo-adding-reward-term-separately CLM-0468When post-training a speech LLM for ASR with GRPO, adding a reward term from a separately pretrained, frozen CTC acoustic judge that scores each sampled transcription against the audio cuts insertion errors on out-of-distribution meeting speech (22-28% fewer on AMI) relative to a schedule-matched text-only -WER reward, without raising word error rate and without needing the judge at inference.
Capability: Stating false facts confidently
Observed on
One 7B speech LLM (Qwen2-Audio, LoRA) with a 0.3B character-level wav2vec2-CTC judge, trained on LibriSpeech with additive-noise augmentation, evaluated on one OOD corpus of 16 AMI.
Sources
- Validated by: Insertion counts and WER on 33,282 utterance-condition instances over six difficulty tiers against a schedule-matched mu=0 WER-GRPO baseline sharing identical data and schedule; meeting-level clustered bootstrap CIs, three seeds, leave-one-meeting-out, plus counterfactual probes on unintelligible audio, a 32-best CTC rescoring comparison, and insertion-AURC under policy-only confidence.. Measured, single paper, one policy and one judge. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On multi-session dialogue questions whose answers depend only on how an utterance was delivered, a cascaded ASR-then-LLM pipeline (Whisper large-v3 into a frontier model) scores below 7B audio-native models on the same clips, and below even a transcript-only baseline, because the transcription step drops delivery cues entirely and adds content errors.Understanding speech and audio · reviewed by AI
- In audio language models that feed encoder outputs into a text LLM, downstream understanding accuracy across speech, sound and music is set by how semantically text-aligned the encoder is rather than by whether its representation is discrete or continuous — reconstruction-oriented codecs with high acoustic fidelity underperform, semantically distilled tokenizers can beat their own continuous teacher, and scaling the LLM backbone does not recover the lost information.Understanding speech and audio · reviewed by AI
- For small instruction-tuned models (3-4B) answering closed-form true/false questions under a stated user opinion, scoring each sampled response by how much more common its answer is than the group itself predicted, and using that score as the GRPO reward, reduces answer flips under user pressure and raises accuracy without any labels, roughly matching label-supervised desycophancy fine-tuning at higher compute cost.Telling the user what they want to hear · reviewed by AI
- For spoken language models answering questions over long dialogue histories, converting the history to transcripts beats keeping it as audio when the answer is a lexical detail (exact slot values across dense turns: mean 71.3% vs 42.6%), but loses when the answer depends on how something was said, where audio history wins for six of seven models and by a larger margin once lexical emotion cues are removed.Understanding speech and audio · reviewed by AI
- In LLM-based ASR, steering decoding toward acoustic evidence only helps if candidates are first restricted to tokens within a small likelihood margin of the greedy token: unconstrained selection by an acoustic compatibility score removes hallucinated outputs but destroys transcription accuracy through unbounded insertions, while the constrained version removes a large share of hallucinations at near-zero WER/CER cost.Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A replication where adding the frozen acoustic reward term fails to reduce insertion rates on held-out far-field speech relative to a schedule-matched text-only reward, or where an equal-strength KL/regularization control produces the same insertion reduction without any acoustic signal.