ui-to-code-generation-where-rendered-screenshot-gives-genuine-external-feedbackIn UI-to-code generation, where a rendered screenshot gives genuine external feedback, iterative self-critique still drifts below the initial draft over ten rounds, because a code edit changes the rendering non-locally and a fix for one region breaks regions that were already faithful; restricting each round to one typed, scoped repair target and carrying past targets forward as an avoid-list reduces that drift.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Generating and editing working code
Observed on
HTML/CSS webpage reproduction from a screenshot, three frontier and three open Qwen VLMs, ten refinement rounds, VLM-judge visual-fidelity scoring; effects on open models are mixed.
Sources
- Measured across 6 models x 3 benchmarks (18 settings): scoped rubrics beat naive self-evolution in 15/18 final-round and 14/18 best-round settings, average +1.20 overall judge points; naive refinement fell below the initial draft on Design2Code for both GPT models. A 60-sample human preference study on one benchmark for two models backs the main comparison. Scoring is by VLM judge, which the authors note is unstable on subtle differences; the coupling mechanism is argued and illustrated rather than directly measured.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- In open-ended editing tasks with no unique correct output (multi-slide design editing), a separate judge model that scores the structural diff between the original and edited artifact against the instruction, and returns its critique verbatim as the next-turn instruction, raises instruction-following over a single pass, but the gain is selective: most tasks pass on the first attempt, and a minority of the ones that enter the loop get worse unless an earlier iteration is restored when the judge's own score declines.Fixing its own mistakes · unreviewed
- Prompting a model to generate its own feedback on a draft output and revise accordingly improves quality on open-ended generation tasks (dialogue, code, review-writing) over a single-shot attempt.Fixing its own mistakes · contested
- Used zero-shot as step-level action critics in long-horizon tool-calling tasks, frontier models are over-pessimistic — flagging a large share of correct actions and pushing the actor into revision loops that lower success below no critic at all — whereas small models fine-tuned on action-level verification rationales flag far fewer valid actions and raise repeated-run reliability.Using the tools it is given · unreviewed
- Tool-augmented multimodal models trained only on how to call vision tools accept tool outputs uncritically; adding supervised trajectories where tool evidence is checked against the image and re-queried, plus a reward for correction that reaches a correct answer, raises accuracy on perception-heavy benchmarks (counting, depth, spatial relations) in 3B and 7B Qwen2.5-VL, while slightly lowering it on knowledge-heavy ones.Fixing its own mistakes · unreviewed
- When comparing a weight-shared recursive model against a depth-matched plain transformer on code generation, the ranking read from a single checkpoint can invert during training, because the larger-parameter arm passes its validation optimum and overfits while the parameter-lean recursive arm keeps improving — so a late-epoch "recursion fits better" reading is a degradation artifact, not a capability difference.Whether the measurement made the finding · unreviewed
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafter linked technique "external-feedback-repair", which does not list this capability in addresses -- recorded, not asserted. Drafted confidence: medium. Falsifier as drafted: Show that naive self-refinement with rendered feedback improves monotonically over rounds on these benchmarks, or that restricting each round to a single scoped repair target gives no advantage over free-form critique at equal round budget. Proposed technique, not catalogued: Rubric-scoped iterative repair with history.