law-of-triviality ADG-0016

Law of triviality

“People give disproportionate weight to trivial issues because those are the ones everyone can grasp.”

Also: Bikeshedding · Parkinson's law of triviality

Origin

C. Northcote Parkinson, 1957: a committee approves a nuclear reactor in minutes and argues for hours over the bicycle shed. About attention in groups, where the mechanism is that easy-to-evaluate features attract evaluation.

Why it should, or should not, apply to models

The mechanism transfers to a model used as a judge: surface features — which answer came first, which is longer, which sounds like the judge's own style — are easy to evaluate and get weight that substance does not. But it does not obviously transfer to a model as a producer: there the catalog finds the opposite, models satisfying hard domain constraints while missing trivial formatting ones. So the law splits on role. As evaluator, the model bikesheds; as author, it under-attends to the shed.

Standing: Holds

Standing counts claims a person has reviewed. Claims reviewed by AI alone are listed and marked, and do not move it.

Where it breaks

Claims showing the heuristic failing for models.

On scientific tasks where a domain-specific constraint and a general presentation constraint (letter case, output format, structure) are both stated in the same prompt, multimodal models satisfy the domain-specific constraint more often than the general formatting one, so domain competence does not imply full instruction compliance.
Following an unfamiliar procedureReviewed by AI
As a producer the model satisfied the hard domain constraint more often than the trivial presentation one: the reverse of bikeshedding.

Where it holds

Claims showing the heuristic applying to models.

Identifies position, verbosity, and self-enhancement biases in strong judges, while also showing high agreement with humans once mitigated.
Biased when judging other outputsReviewed by AI and Russ Miller
Judges weight position, verbosity and self-similarity — the trivially graspable features — until those are specifically mitigated.
As a raw pairwise judge with no order-swap mitigation, GPT-4 can have its verdict flipped simply by swapping which candidate answer is shown first; applying a position-swap protocol restores agreement with careful human raters to close to human-human agreement levels.
Biased when judging other outputsReviewed by AI and Russ Miller
A verdict flipped by swapping the order of the candidates is weight given to the most trivial feature possible.

Add a case where it holds or breaks

A break is worth more than a hold. If you have seen this fail for a model, with the setup written down, that is the most useful thing you can add here.