arxiv-2606-16262 SRC-0319 · paper

UXBench: Measuring the Actionability of LLM-Generated UX Critiques

Created: 2026-06-15 · Ingested: 2026-09-19

Large language models (LLMs) are increasingly deployed as UX judges that inspect interfaces, diagnose usability problems, and propose repairs. Yet no controlled benchmark measures whether the resulting critiques are reliable and actionable across heterogeneous product surfaces. We introduce UXBench, a benchmark for evaluating LLMs as interaction-grounded UX judges. UXBench comprises local-first runnable web fixtures spanning ten product-surface families, paired with coverage-gated browser exploration that forces models to collect interaction evidence before reporting. Each judge model produces

In brief

Frontier models write UX critiques that all help a downstream repair agent, but the amount of help they give differs by a small and surface-dependent margin rather than by a clean ranking. UXBench measures this by forcing a judge model to explore a running interface before reporting, then handing its report to a fixed repair agent.

The suite is 41 local-first HTML/CSS/JS fixtures (11 real-product anchors, 30 independently authored synthetic siblings) across 10 surface families from landing pages to chatbots and mobile micro-UIs. A coverage gate rejects premature termination, pushing the agent back to unexercised controls. Claude Code performs the repair under brand and interaction invariants; GPT-5.4-Mini scores the result on 7 rubric dimensions, 1–5.

All 8 models improved on the 3.27 baseline. GPT-5.4 gained +0.22, Gemini-3.1-Pro +0.14, an 0.08-point spread. Per-dimension leaders diverge: GPT-5.4 on error recovery, Kimi-K2.5 on feedback and trust, Qwen-3.6-Plus on flow and scanability. GPT-5.4, Kimi-K2.5 and GLM-5.1 each average about 0.57 pairwise win rate, with many head-to-head cells near 0.5.

Repair agent and scorer are held fixed, so variation is attributable to the reports. The scoring judge is an LLM from one of the evaluated families; blind human review by 6 UX-experienced reviewers backs the coarse ordering but groups GPT-5.4 and Claude-Sonnet-4.6 together instead.

Static fixtures only, 1 repair pipeline, no live sites or personalization. Treat UX-judge leaderboards as surface-conditioned, not global.

Written from the abstract by claude-opus-5 on 2026-10-24, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsWhen a critique's worth is measured by whether a fixed downstream repair agent can improve the interface from it, eight frontier models differ meaningfully in how actionable their UX critiques are, lead on different product-surface categories, and vary in reliability fixture by fixture; the best repair lift was about +0.22 on the benchmark's scale, so the task is unsaturated and not one-dimensional.