Free tools, reference pages, templates and checklists from the research on this site. Nothing needs a sign-up. Each one says what it does and what it can't tell you.
Tools
ToolAI robots.txt generatorBuild a robots.txt that blocks AI training bots and keeps AI search. Presets, comparisons and exports.Open the generator →
ToolAI crawler benchmarkPaste your robots.txt and compare it, crawler by crawler, with the UK sites in the studies.Check a file →
Tool and librarySearch Console regex library44 tested patterns for Search Console and GA4, with copy buttons and an in-browser tester.Open the library →
DataThe Data Room explorerSearch, filter and download rows from every dataset behind the research.Explore the data →
Templates to download
| Resource | What it does |
|---|---|
| Study plan template (TXT) | The ten-part plan used for the registered studies here: question, expectation, fixed sample, what gets recorded, limits, change log. Fill it in before you collect anything. |
| Prediction ledger template (CSV) | The same eight columns as the ledger on this site, with one example row to delete. Opens in Excel or Google Sheets. |
| Search Console regex patterns (CSV) | All 44 patterns from the regex library as a spreadsheet. |
| Zero-click extraction script (TXT) | The script used to measure where the first ordinary link sits on a Google results page. Explained in the study. |
Reference
| Resource | What it does |
|---|---|
| AI crawlers explained | The 15 AI crawlers in the studies: who runs each one, what it is for and who blocks it. |
| Training bot, search bot, fetcher | The three jobs an AI crawler can do, and why the difference matters in robots.txt. |
| What “blocked” should mean in a sentence | The definition used in every crawler study here. |
| Method | Sample rules, definitions and this site’s own robots.txt policy. |
| Reproduce the figures | The scripts that turn the saved files into the published numbers. |
| Tools and sources | The tools behind the research. |
Checklists and how-tos
| Resource | What it does |
|---|---|
| Five questions to ask of a statistic | A check to run before you quote a number in a deck or a post. |
| Five labels every AI visibility number needs | What must travel with an AI visibility figure for it to mean anything. |
| How to write a claim you can mark wrong | Turning a vague forecast into one with a date and a test. |
| How many times to run an AI prompt | How many runs you need before a share-of-answers number is worth quoting. |
| Block AI training crawlers, keep ChatGPT-User | The robots.txt lines, and what they do and don’t stop. |
| Content-Signal in robots.txt | One line that says “search yes, AI training no”. |
| All nine search hacks | Every short guide in one page, each tied to a dated test. |
Using these
The templates and datasets are free to use under CC BY 4.0: use them for anything and credit Jacob Whitmore with a link. If one is wrong or broken, tell me and the fix will be logged.
Looking for a study rather than a tool? Search the library.