Original data · Running study · Updated 2026-07-10
We tried to reproduce 19 repeated AI-search statistics. Here is what happened.
This is the study nobody in the field runs. We took the statistics that get quoted most often about AI search, went to the primary source behind each, and recorded whether the number actually came back. No API keys, no proprietary panel, no crawler. Just the reproduction, done by hand and shown in full.
26%
survive a check against their own primary source, as they are circulated.
21%
have no locatable primary source at all, despite circulating widely.
47%
cite a real study but misrepresent what it measured, or are contradicted by it.
n = 19 claims · not a random sample; see method below · every claim links to its primary source in the ledger
The distribution
Every audited claim lands in exactly one of four buckets. The failure modes are the interesting part: a field's statistics can fail to reproduce in more than one way, and each way means something different.
Survive as circulated
5Reached the primary source; the claim holds (VERIFIED), or holds but is single-sourced (DIRECTIONAL).
A real source says otherwise
A real, named study exists, but it contradicts the claim as repeated (REFUTED) or measured something other than what the claim asserts (MISLEADING).
Cited, but did not reproduce
1A source is named, but we could not surface the number from it when we fetched it.
No source at all
4The number circulates widely with no locatable primary source we could reach.
How this study was built
For each statistic, the procedure was the same one described on how we grade: find the primary source, not the blog citing the blog; fetch it and read the number off it ourselves; record the sample size, the date and what was actually measured; and grade the claim by what happened. The full record for every claim, with its source link and our reproduction note, is the evidence ledger. This page is the count.
What "survives" means
A claim survives only if we reached its primary source and the number came back as the claim is circulated. A claim can fail even when the underlying study is real and excellent: if the study measured a research benchmark and the claim presents it as a measurement of ChatGPT, the claim does not survive, because the thing being repeated is not the thing that was found. That is the difference between auditing a study and auditing a claim, and it is the whole point.
The honest limitation: this is not a random sample
We did not draw these statistics from a defined population at random. We collected the ones we kept encountering, the numbers that recur across guides, decks and vendor blogs about AI search. That is a real selection effect and we will not hide it behind a headline percentage. The claim this study makes is narrow and defensible: among the statistics this field repeats most, a large share do not survive contact with their own sources. It is not a claim about every statistic ever published.
It is also a running study. The count above is computed from the live ledger at build time, so it moves as we audit more claims. When it moves, this page moves with it, and any claim we regrade leaves a dated trail in corrections. We would rather publish a study that visibly changes than a snapshot that pretends to be final.
Reproduce the reproduction
Every number here is checkable. Pick any claim in the ledger, follow its primary-source link, and see whether you get what we got. If you do not, that is the most useful email we can receive: send it, and if you are right the claim is regraded in public, with the date and what changed. A reproduction study that could not itself be reproduced would be the richest possible joke at our expense, so the whole apparatus is built to be re-run by a stranger.