Study plan, registered on 11 October 2026. This study has not been run yet. I’m publishing the method before any results exist so it can’t be adjusted to fit them. Results will be added to this page, with the date they were collected.
On 9 October 2026 I found that the House of Commons Library briefing that Nationwide footnotes for a £446bn purple pound figure doesn’t contain that figure (the study). That was one figure. This study asks the same question of AI answers at scale: when an engine gives a UK market-size figure and cites a source, is the figure in that source?
The questions
20 questions, frozen as of 11 October 2026, each in the form How much is the UK [market] market worth? The markets are:
- pet food
- video games
- coffee shop
- gym and fitness
- plant-based food
- online grocery
- DIY and home improvement
- beauty and cosmetics
- tea
- craft beer
- cybersecurity
- legal services
- accountancy services
- wedding
- toy
- bicycle
- electric vehicle charging
- recruitment
- digital marketing
- takeaway food
Engines and runs
- ChatGPT, default model, signed in, temporary chat set to unpersonalised.
- Google AI Mode, signed in.
- Claude, web search on, incognito chat.
- Gemini, signed in, temporary chat.
One run per question per engine, in a fresh chat, with no follow-up questions, on 20 October 2026, from the UK. That’s 80 answers.
What gets checked
For each answer that gives a figure, I open the first source it cites: the first clickable link in the answer, or the first source card where the engine shows sources separately. I read it in full on the run day, PDFs included. Only the first source is checked, so the reading stays manageable and every answer is treated the same way.
Each checked source goes into one of four classes:
- Supported: the source gives the same figure in the same unit, allowing for the rounding the answer used (£274bn matches £274.1 billion).
- Different figure: the source gives a figure for that market, but not the one quoted.
- No figure: the source gives no figure for that market.
- Unreachable: paywall, error page or timeout. Retried once the same day, then published as unreachable. Never guessed.
Answers that give a figure but cite nothing are recorded as no citation and reported separately.
Dated call
- Claim: at least a quarter of the cited answers will point to a first source that doesn’t contain the figure.
- Called: 11 October 2026.
- Run date: 20 October 2026.
- Check date: 27 October 2026.
- Test: (different figure + no figure) ÷ (supported + different figure + no figure). Unreachable sources are left out of the sum and published.
- Hit if: 25% or more.
- Miss if: under 25%, or the study hasn’t been published by the check date.
- Void if: one of the four engines no longer exists on the run date.
- Baseline: not measured.
This call is P22 on the predictions page. If it misses, it stays up in red.
I set the 25% threshold before running anything.
What I will not claim
- That an engine made a figure up. A figure missing from the first source may be in another source the engine read.
- Which market size is right. The study checks whether the cited source contains the quoted figure, not whether either figure is accurate.
- Anything about markets outside the 20 or engines outside the four.
Data
ai-cited-sources-figures-2026-10-20.csv, one row per answer, with the columns run_date, engine, model_mode, session, question_id, question, figure_quoted, cited_url, cited_page_date, source_class, figure_in_source, location_in_source, read_in_full and note.