arxiv-2210-15723 SRC-0371 · paper

Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation

Stefan Wojcik, Sophie Hilgard, Nick Judd, Delia Mocanu, Stephen Ragain, M. B. Fallin Hunzaker

Created: 2022-10-27 · Ingested: 2026-09-21

We present an approach for selecting objectively informative and subjectively helpful annotations to social media posts. We draw on data from on an online environment where contributors annotate misinformation and simultaneously rate the contributions of others. Our algorithm uses a matrix-factorization (MF) based approach to identify annotations that appeal broadly across heterogeneous user groups - sometimes referred to as "bridging-based ranking." We pair these data with a survey experiment in which individuals are randomly assigned to see annotations to posts. We find that annotations selected by the algorithm improve key indicators compared with overall average and crowd-generated basel

In brief

Ranking crowd annotations by whether they appeal across opposing viewpoints, rather than by raw majority vote, produces notes that both change readers' beliefs and are accepted by people on both sides of the political spectrum. Twitter's Birdwatch system fits a matrix factorization to a sparse note-rater matrix, with regularization 5x higher on intercepts (0.15 vs 0.03) than on 1-dimensional factor vectors, so a note only earns a high intercept if raters with dissimilar factors both call it helpful; notes score "currently rated helpful" at intercept at least 0.40.

Evaluation used 3 waves of a survey experiment on stratified random samples of Twitter users, with Wave Two in December 2021 (N=7,387, 38 notes) and Wave Three in April 2022 (N=15,935, 46 notes), plus an on-platform A/B test. In Wave Three, algorithm-selected notes cut agreement with the misleading Tweet's substance by about 26% (N=6,046, p << 0.001), and 64% of users found them helpful. In Wave Two, matrix factorization gave a 41% reduction versus 22% for any note and the same 22% for supermajority voting (84% helpful). Perceived Tweet accuracy correlated only r = 0.45 with claim agreement in control and r = 0.29 under treatment, and accuracy alone showed no significant effect. On-platform, exposed users were 25-34% less likely to like or retweet.

The design is randomized with a supermajority baseline and no significant Democrat-Republican difference, but note selection was hand-filtered, the A/B effect is diluted by pipeline latency, and everything is US-only and self-evaluated by the deploying team.

If you measure misinformation interventions with true-false accuracy scales, this says you are likely missing real effects.

Written from the abstract by claude-opus-5 on 2026-10-17, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

contestsA shared catalog where humans and agents file falsifiable claims, and where each contributor's standing is computed from outcomes other people check (claims that survive contest, contests that are upheld, breaks found), improves the body of knowledge faster and with less Goodhart drift than academic publish-or-perish, because the only way to raise standing is to be right about things others test; this holds only if a break raises the finder's standing and lowers the filer's only when the filer's falsifier was weak, since a clean claim that later breaks is the system working.