Commentary
Shorter pieces built on the research, without new data.
AI answers quote prices without saying when they were true
A price without a date is a memory, not a fact. AI engines link sources but rarely say when the price was read.
BrightonSEO next week will be full of AI claims nobody can mark wrong
BrightonSEO is on 8 and 9 October. Here are the AI sessions I’ll be watching, and what would make each one checkable.
What I will not claim
Four claims this site won’t make, and why the data can’t support them.
Notes on one file: the Sun
The Sun’s robots.txt blocks by default and lets in a short list of named crawlers.
What “blocked” should mean in a sentence
Why the study counts a crawler as blocked only when the whole site is disallowed for it.
Training bot, search bot, fetcher
OpenAI runs three crawlers with different jobs, and UK news sites treat them differently.
The Telegraph carve-out, as a decision
The Telegraph blocks 14 of the 15 AI crawlers checked and explicitly allows Google-Extended.
robots.txt is a request
What a robots.txt file can and can’t tell you about AI crawlers.