Original studies on AI search, using data I collected myself. Each one says how the data was gathered, what it can’t tell you, and links to the raw data.
Studies
- Which AI crawlers do the UK’s biggest news sites block? (October 2026). Nine of the UK’s ten biggest newsbrands block at least one AI crawler, but they block training bots far more often than the bots that fetch pages for AI answers.
Registered studies
These studies have a published method and haven’t been run yet. Results will be added to each page with the date they were collected.
- do news sites enforce their robots.txt?
- do news sites that block AI training still appear in AI answers?
- the Telegraph and the Daily Mail in AI answers
- link, mention or nothing
- established newsbrands and lower-credibility sites
How I run studies
- Public data only, collected by me, with the date it was collected.
- Method and limits written up alongside the findings.
- Raw data published so anyone can check it.
- Repeated on a schedule, so changes over time are visible.
Follow along on LinkedIn
I share new research, predictions and what I’m seeing in AI search on LinkedIn first.