rl-trained-multimodal-image-quality-models-supervising-reasoning-rating-reward-alone CLM-0478In RL-trained multimodal image-quality models, supervising reasoning with the rating reward alone raises rating alignment while making the stated reasoning less faithful — rating-only training reached an average PLCC/SRCC of 0.836/0.815 yet its reasoning-guided edits lowered judged image quality (mean gain -0.260, worse than the untrained baseline's -0.136).
Capability: Saying what actually drove the answer
Observed on
Blind image quality assessment with a 4B MLLM actor trained by GRPO on KonIQ-10k; faithfulness measured as the quality change a frozen editor+judge produce from the actor's stated .
Sources
- Validated by: Mean judge-scored quality gain from reasoning-guided edits on a 5,866-image subset: +0.789 for the trained actor vs +0.213 for Q-Insight and -0.137 for the untrained backbone, holding under a swapped editor (Mage-Flow) and two swapped judges including GPT-5.6 Sol (81.4% 3AFC win rate); rating alignment stays comparable to prior methods.. Single ablation table within one paper; the faithfulness metric is itself a model-judged edit outcome, so it measures whether the factors are actionable rather than whether they caused the model's own rating. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In RL-trained multimodal image-quality models, applying the reasoning reward only to reasoning tokens and the human-rating reward only to rating tokens keeps rating alignment at the level of rating-only training while turning reasoning-guided edit quality gain positive, and it prevents the collapse to a single universal edit that an unmasked shared reward produces despite that variant's larger raw gain.Saying what actually drove the answer · reviewed by AI
- For a same-family pair of reasoning and instruction-tuned 4B models, learning one interpolation ratio per module (attention block, FFN block, embedding, final norm) with a reward that scores correctness and ranks correct responses by length cuts average generated tokens by about 27% relative to the reasoning model while holding average accuracy, and beats fixed global-ratio merging and CMA-ES search over the same ratios.Cutting to the chase · reviewed by AI
- When a document image states multi-step selection constraints in narrative text and the supporting numbers sit in several charts, multimodal models degrade steadily as either the number of logical hops (2 to 5) or the number of charts (2 to 6) grows, with the best model reaching 62.83% overall against over 90% for human annotators.Carrying a chain of inference through to its conclusion · reviewed by AI
- For 4-8B multimodal models on long-video QA, RL training on questions authored from short scene segments but answered over the whole video improves benchmark accuracy more than RL on questions authored from the whole video, because segment-authored questions target details a whole-video view overlooks; including the segment timestamp in the training instruction adds a further gain by removing referential ambiguity, and is not needed at inference.Reasoning about time in video · reviewed by AI
- For 8B-14B generative reward models trained as judges, making the model generate input-adaptive evaluation criteria inside its own reasoning chain and rewarding the quality of those criteria with a stronger teacher model raises agreement with preference labels, and the gain scales with the teacher's capability rather than with the presence of the rubric step alone.Judging another's work as well as a careful reviewer would · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: A rating-only RL run whose stated quality factors, when used to edit the image, produce judged quality gains equal to or above those of reasoning-rewarded training, under independent editors and judges.