Everything published on this site in one searchable list: 79 studies, experiments, datasets, analysis pieces, essays, guides, predictions and tools. Search by word, or filter by type and topic.
79 items, newest first
All items
Ask an AI when its figure is from and it tells you: 80 of 80
Run 14 November 2026. Asked "When is that figure from?", ChatGPT, Google AI Mode, Gemini and Claude gave a period in 80 of 80 chats. A neutral follow-up got one in 7 of 64. 3 periods…
Google answered 40 of 40 UK questions before the first link
Read 18 November 2026. On 40 everyday UK questions, Google showed an AI Overview or its own answer box above the first ordinary link every time. The first link started a median 769…
The Future of Search Report 2026: what can be checked
An annual report built only from dated readings on this site, with what each one can and cannot show.
AI citation tracker: is this site named or linked in AI answers?
A fixed set of questions asked of AI tools to see whether this site is named or linked. Baseline reading and data.
Research plans
Registered plans for studies that have not been run yet, each with its question, sample, measures and limits.
Search Console regex library: 44 tested patterns
44 regex patterns for Search Console and GA4, each tested, with copy buttons, a CSV and an in-browser tester.
Does publishing original research get you cited by AI?
No controlled study shows original research raises AI citations. Why it should help AI use your work, why credit is less certain, and the test this site will score on 2 Jan 2027.
The Jacob Whitmore Index
Every headline reading from the research in one table: the figure, its denominator, the date it was read and the study behind it.
UK AI Search Observatory
The home of the recurring research: which AI crawlers UK sites block by sector, what AI answers say, and who they recommend.
Which UK retailers do AI tools name? 36 answers from four AI tools
A pilot reading of which UK retailers four AI tools name when asked shopping questions. 36 answers, with the full answer file.
UK businesses in AI answers: the ten-sector plan
The plan for a quarterly reading of which UK businesses AI tools name, recommend and cite, sector by sector.
The Search Experiment Lab
Ten common questions about search and AI discovery, and whether each has been tested here, reviewed, or not tested at all.
How to say "search yes, AI training no" in one robots.txt line
One robots.txt line states whether your pages may be used for search, AI answers or AI training. Vinted is the one top-ten UK retail site using it. Copy the line and read the limits.
Will AI agents do your shopping? Three of the UK's biggest retail sites have said no in writing
Checked 12 November 2026. Amazon, eBay and Vinted tell AI shopping agents to stay out of checkout. Seven of the UK's ten biggest retail sites say nothing. What the files and one court…
Do the UK's biggest shopping sites block AI crawlers?
Read 12 November 2026. 4 of the UK's 10 most-visited retail sites block AI crawlers in robots.txt. Amazon blocks 12 of 15, eBay 9, Vinted 5, Temu 1. Six block none.
AI crawlers explained: what each one does and who blocks it
A plain-English reference to the 15 AI crawlers in the studies: who runs each one and what it is for.
Do the UK’s biggest supermarkets block AI crawlers?
None of the UK’s ten biggest supermarkets blocks any of 15 AI crawlers in robots.txt. The ten biggest newsbrands block 99 of the same 150 cells. Aldi names nine and allows them all.
The Data Room: every dataset, free to use
Every dataset behind the research as a plain CSV under CC BY 4.0, with an explorer to search, filter and download rows.
AI crawler benchmark: check your robots.txt
Paste a robots.txt and see how it compares with the UK sites in the studies, crawler by crawler.
How do ChatGPT, Claude, Gemini, Perplexity and Copilot choose their sources? What the companies actually say
Checked 8 Nov 2026: what OpenAI, Anthropic, Google, Perplexity and Microsoft say about how their AI answers find and pick sources. Only one explains how it chooses.
How many times should you run an AI prompt before you trust the number?
To pin an AI brand-mention rate to within ±10 points you need about 97 runs of one prompt. To show a change worked, about 388 per version. The maths and a free calculator.
Do AI companies block each other's crawlers? 17 AI company sites checked
On 6 November 2026, three of 17 AI company sites blocked rival AI crawlers in robots.txt. All three were chat apps. None of the 11 corporate sites blocked any. Cell-level data.
P37 missed: AI engines had the new Bank Rate within a day
I called that at least one of four AI engines would still give the old Bank Rate 24 hours after the 5 November 2026 decision. None did. Here is the data and what I got wrong.
What happens after the AI slop?
AI made publishing almost free. It didn't make trust cheap. Why proof, original data and open methods are the next edge in search and AI answers.
Nobody Wants to Read Your Book
Writing a book used to be hard, and that difficulty kept the supply of books in line with readers. AI has made writing almost free. Reading still takes ten hours. What happens when books…
UK news robots.txt, November 2026
0 of the 150 cells changed since 2 October 2026. The second monthly snapshot of AI crawler rules in the robots.txt files of the UK’s ten biggest news sites.
AI robots.txt generator
Build a robots.txt that blocks AI training bots and keeps AI search, with presets, comparisons and exports.
How to check a statistic before you use it: five questions
None of the 32 most-repeated AI search statistics passed five basic checks. Ask any statistic these five questions before you paste it.
Gartner said search volume would drop 25% by 2026. Can it be scored?
Gartner's 2024 call that search volume would fall 25% by 2026, checked in October 2026. It can't be scored as written. On the closest public data, it looks like a miss.
Do backlinks get you cited in AI answers? What 16 studies actually measured
Checked 30 October 2026. 16 studies on links, authority and rankings vs AI citations. All correlational, none with raw data, one naming a country. Brand mentions beat links.
Search Console's AI report shows impressions, not clicks. What it can and can't tell you
Checked 30 October 2026. Search Console's AI report counts impressions only: no clicks, no queries. What it can answer, what it can't, and how to report AI visibility anyway.
Do UK news sites and universities publish llms.txt?
Checked 30 October 2026. 5 of 33 big UK news and university sites publish llms.txt. The Daily Mail's opens AI access to selected URLs while its robots.txt blocks the rest.
What do AI visibility trackers say about how they measure? 13 tools checked
Checked 30 October 2026. 13 AI visibility trackers, nine questions about method. None publishes accuracy evidence. None says if it collects signed in or out.
AI search statistics, traced to the source
Checked 29 October 2026. The 32 AI search statistics repeated most across roundups, each traced to its source. None passed all five checks. None is about the UK.
Google is lying to your rank tracker on purpose
Google is deliberately serving fake results pages to suspected scrapers. It isn’t blocking them. It answers them, with complete, normal-looking pages whose organic results have been…
Meta’s Muse AI: what it means for SEO, Google and the future of search
Muse is Meta’s agent, not Facebook search. What Meta announced, how AI Mode differs, the UK position, and a six-layer stack for AI discovery. Dated.
SEO and AI search predictions for 2027
Nine calls for 2027, each with a confidence figure, a check date and a test.
Consumers will let AI fill the cart. They still won't let it pay.
Four 2025–26 surveys agree: shoppers let AI agents recommend and fill a basket, not pay. What agents check before they recommend you, and what merchants control.
Study plan: when AI engines repeat my findings, do they get them right and say where they came from?
A registered quarterly audit of whether AI engines repeat nine findings from this site correctly and credit them.
Study plan: does asking "when is that figure from?" fix an undated AI answer?
A registered experiment: the same ten UK figure questions on four engines, with a date follow-up against a neutral follow-up, 160 chats.
Study plan: which UK universities block AI crawlers, month by month?
A registered monthly check of which AI crawlers the 24 Russell Group universities block in robots.txt, using the same 15 crawlers and definition as the news series.
Ask an AI answer when its "current" figure is from
Google AI Mode and Gemini left the date off 7 "current" UK figures. Both errors I found were side figures next to the headline number.
Keep facts in body text, not in schema or meta tags
Four common HTML-to-text tools all kept body text and dropped JSON-LD, meta descriptions, aria-labels, data- attributes and comments.
Put the date in the same sentence as the figure
7 of 40 AI answers to "current" UK figure questions gave no date. If you publish figures, put the period in the sentence that carries the number.
Does naming a crawler in robots.txt override User-agent: *?
Yes. A crawler follows the group that names it and ignores the * group. The Sun's robots.txt uses this to let OpenAI in.
How to block AI training crawlers but still let ChatGPT-User in
Most of the UK's biggest news sites block AI training crawlers and leave the user-triggered fetchers open. Here's the robots.txt that does it.
Guides, each one backed by a dated test
Nine things to do today, each tied to a study on this site with a date, a sample and a CSV.
What text do machines actually read on a page?
A dated test of which parts of a web page survive when four common tools turn it into text.
Five labels every AI visibility number needs
Five labels any AI visibility number needs before anyone can check it: date, prompt, model, runs and one change at a time.
Crawled is not cited
An AI crawler in your logs proves a page was fetched. It does not prove the page was used, credited or clicked.
Study plan: does an AI crawler stop when my robots.txt asks it to?
A registered method for logging which AI crawlers fetch jacobwhitmore.com, then blocking one and checking whether it stops.
Study plan: do the sources AI engines cite contain the figure?
A registered method for opening every source AI engines cite for a UK market-size figure and checking whether the figure is in it.
Study plan: do personas change AI brand picks outside accounting software?
A registered method for testing whether the customer in the prompt changes AI brand recommendations across four UK business categories.
Study plan: undated current figures, every quarter
A registered method for rerunning the same ten "right now" UK figure questions on four AI engines every quarter.
Study plan: how fast do AI answers catch up with a Bank Rate decision?
A registered method for asking four AI engines the Bank Rate at fixed times before and after the 5 November 2026 decision.
Study plan: UK news robots.txt, month by month
A registered method for re-reading the same ten UK news robots.txt files every month and counting what changed.
Do AI answers say when a "current" figure is from?
On 10 October 2026, 7 of 40 AI answers to "current" UK figure questions didn't say when the figure was from. All 7 came from Google's two engines.
Does ChatGPT recommend different brands to different customers?
Checked 9 October 2026 Ask ChatGPT for the best accounting software in the UK and you get Xero, QuickBooks and one more. Tell it you’re the finance director of a 250-person manufacturer…
How much is the purple pound worth? Five figures, and AI doesn't say which year
Checked 9 October 2026 There’s no single figure. Published estimates of the purple pound, the spending power of disabled people and their households, run from £212bn to £446bn a year. On…
How to use the EntityMap validator, and what it can't tell you
I tested the EntityMap validator on 9 October 2026. It checks a file's shape, not whether it's true or whether anything reads it. None of nine UK newsbrands I checked publish one.
I said I'd check my World Cup search predictions in August. I can't.
In May I said I'd check my World Cup search predictions in August. I've tried. None of them can be marked, and here's why.
The future of search is a smaller click. That part can be checked.
Most future-of-search posts can't be marked wrong. This one makes three dated calls about links, mentions and the publisher split.
AI answers quote prices without saying when they were true
A price without a date is a memory, not a fact. AI engines link sources but rarely say when the price was read.
Ask AI for "the best" local tradesperson and you get a guess
"Who's the best plumber in Leeds?" has no checkable answer. AI engines answer it anyway, and I think they'll keep doing so.
Small business sites aren't letting AI in. They're not deciding at all.
Publishers write AI crawler rules. Most small business sites write nothing, so the platform's default decides for them.
BrightonSEO next week will be full of AI claims nobody can mark wrong
BrightonSEO is on 8 and 9 October. Here are the AI sessions I'll be watching, and what would make each one checkable.
Study plan: established newsbrands and lower-credibility sites
A registered method for comparing AI crawler block rates between established UK newsbrands and high-reach sites that fail a set credibility test.
Study plan: link, mention or nothing
A registered method for counting how often AI answers to UK news questions link to a news source.
Study plan: the Telegraph and the Daily Mail in AI answers
A registered comparison of two papers with almost identical robots.txt files and one difference.
Study plan: do news sites that block AI training still appear in AI answers?
A registered method and a fixed list of 30 UK news queries for checking which newsbrands appear in AI answers.
Study plan: do news sites enforce their robots.txt?
A registered method for checking whether the servers of ten UK news sites act on what their robots.txt files say.
Notes on one file: the Sun
The Sun's robots.txt blocks by default and lets in a short list of named crawlers.
How to write a claim you can mark wrong
The four things a claim needs before it can be checked.
robots.txt is a request
What a robots.txt file can and can't tell you about AI crawlers.
The Telegraph carve-out, as a decision
The Telegraph blocks 14 of the 15 AI crawlers checked and explicitly allows Google-Extended.
What “blocked” should mean in a sentence
Why the study counts a crawler as blocked only when the whole site is disallowed for it.
Which UK news sites block GPTBot and other AI crawlers?
Nine of the UK's ten biggest newsbrands block at least one AI crawler, but they block training bots far more often than the bots that fetch pages for AI answers.
Predictions ledger
Every dated prediction with its check date and the test written in advance. Right, wrong or could not score, and misses stay up.
Method
How the studies are run: sample rules, the 15 crawlers, what blocked means and the site robots.txt policy.
Nothing matches. Clear the search or pick another type.
How the library is organised
- Research is a dated reading of something, with a sample, a method and limits. Research plans are published before the data is collected.
- Experiments are tests I ran myself on AI tools or crawlers. The Search Experiment Lab shows which questions have been tested.
- Data publications come with a CSV. All of them are also in the Data Room.
- Analysis explains a development or someone else's study and what it does and doesn't show.
- Ideas & Perspectives are opinion essays. They are labelled so nobody reads them as research. See Ideas & Perspectives.
- Technical guides are short how-tos, each tied to a study.
- Predictions are dated calls and their results. The full record is the ledger.
- Resources are tools and reference pages. They are grouped on Resources.
Everything here is written by Jacob Whitmore. Dates are first publication dates; an updated date is shown when a page changed after that. New posts also go out on the RSS feed.