image-aesthetics-models-fine-tuned-critique-scoring-competence-producing-reference-aligned-aesthetic CLM-0403For image-aesthetics models fine-tuned for critique and scoring, competence at producing reference-aligned aesthetic critique does not carry over to judging whether an image suits a stated purpose, audience or cultural setting: all four aesthetic specialists tested scored below 30% accuracy on 301 expert-curated suitability cases (best 29.57%) versus 91.69% for the strongest general-purpose model, and they predicted 'suitable' in 59-77% of binary cases against a gold rate of 38.14%.
Capability: Judging another's work as well as a careful reviewer would
Observed on
Single-image closed-form suitability questions, no retrieval or tools, 448x448 inputs for locally run models; specialists are ArtQuant-APDD, ArtiMuse, Q-SiT, AesExpert..
Sources
- supportsAesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual SuitabilityMeasured on 301 expert-reviewed cases (291 binary, 10 three-way) with exact-match accuracy, plus a 100-case human grounding audit (ArtiMuse 21% accuracy to 3% evidence-grounded) and a 36-pair counterfactual audit where specialists showed weaker directional updating. Unparsable responses counted incorrect, which may penalize weak instruction-followers; base-vs-specialist comparison isolates tuning effect for only two of four models.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On 2,154 expert-curated multiple-choice questions across eight architectural categories, 11 current models all score above 90% zero-shot (best about 94%), with quality-attribute and pattern questions near ceiling, but accuracy varies by category and drops for smaller models on the harder tiers (to about 85-87%), so declarative architectural knowledge is largely present while the benchmark cannot show whether it is applied in design.Making design decisions that hold up as a system grows · reviewed by AI
- When a document image states multi-step selection constraints in narrative text and the supporting numbers sit in several charts, multimodal models degrade steadily as either the number of logical hops (2 to 5) or the number of charts (2 to 6) grows, with the best model reaching 62.83% overall against over 90% for human annotators.Carrying a chain of inference through to its conclusion · reviewed by AI
- In RL-trained multimodal image-quality models, supervising reasoning with the rating reward alone raises rating alignment while making the stated reasoning less faithful — rating-only training reached an average PLCC/SRCC of 0.836/0.815 yet its reasoning-guided edits lowered judged image quality (mean gain -0.260, worse than the untrained baseline's -0.136).Saying what actually drove the answer · reviewed by AI
- For multimodal LLMs, a behavioral perturbation signature (answer/confidence change under visual degradation, blank-image replacement, and grounding/relation probes) predicts which failure family an error belongs to far better than scalar confidence, yet folding the same signature into confidence as a direct penalty makes answer-versus-abstain ranking worse than scalar confidence alone.Saying it does not know · reviewed by AI
- General-purpose factuality verifiers (entailment-, consistency-, and RAG-trained) and zero-shot frontier LLM judges perform near chance at spotting unsupported claims in scientific peer reviews that must be checked against the full submitted paper, because they cannot separate paper-unsupported assertions from legitimate evaluative critique; small models fine-tuned on in-domain examples beat them by a wide margin.Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An aesthetics-specialized model that matches its base or a frontier general model on contextual-suitability accuracy without over-accepting, or evidence that the specialists' low scores come from answer-format parsing rather than judgment.