check_date,group,site,domain_requested,requested_url,final_url,status_code,content_type,result,first_line,notes
2026-10-30,news,Sky News,news.sky.com,https://news.sky.com/llms.txt,https://news.sky.com/llms.txt,404,text/html,absent,,
2026-10-30,news,The Guardian,www.theguardian.com,https://www.theguardian.com/llms.txt,https://www.theguardian.com/llms.txt,404,text/html,absent,,
2026-10-30,news,Daily Mail,www.dailymail.co.uk,https://www.dailymail.co.uk/llms.txt,https://www.dailymail.com/llms.txt,200,text/plain,present,# Daily Mail,"Redirects to dailymail.com (query string added by the site, not recorded). File opens with an HTML comment line 'Updated: 2026-10-26' before the H1. 211 lines, 13 H2 sections. Says listed URLs (ending in an underscore before .html) are unblocked for AI access."
2026-10-30,news,The Independent,www.independent.co.uk,https://www.independent.co.uk/llms.txt,https://www.independent.co.uk/llms.txt,410,text/html,absent,,410 Gone
2026-10-30,news,BBC,www.bbc.co.uk,https://www.bbc.co.uk/llms.txt,https://www.bbc.co.uk/llms.txt,404,text/html,absent,,
2026-10-30,news,Mirror,www.mirror.co.uk,https://www.mirror.co.uk/llms.txt,https://www.mirror.co.uk/llms.txt,404,text/html,absent,,
2026-10-30,news,Yahoo News UK,uk.news.yahoo.com,https://uk.news.yahoo.com/llms.txt,https://uk.yahoo.com/,200,text/html,absent,,Redirects to the uk.yahoo.com homepage (query string not recorded)
2026-10-30,news,The Sun,www.thesun.co.uk,https://www.thesun.co.uk/llms.txt,https://www.thesun.co.uk/llms.txt,404,text/html,absent,,
2026-10-30,news,Daily Express,www.express.co.uk,https://www.express.co.uk/llms.txt,https://www.express.co.uk/llms.txt,404,text/html,absent,,
2026-10-30,news,The Telegraph,www.telegraph.co.uk,https://www.telegraph.co.uk/llms.txt,https://www.telegraph.co.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,University of Birmingham,www.birmingham.ac.uk,https://www.birmingham.ac.uk/llms.txt,https://www.birmingham.ac.uk/llms.txt,404,text/plain,absent,,
2026-10-30,university,University of Bristol,www.bristol.ac.uk,https://www.bristol.ac.uk/llms.txt,https://www.bristol.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,University of Cambridge,www.cam.ac.uk,https://www.cam.ac.uk/llms.txt,https://www.cam.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,Cardiff University,www.cardiff.ac.uk,https://www.cardiff.ac.uk/llms.txt,https://www.cardiff.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,Durham University,www.durham.ac.uk,https://www.durham.ac.uk/llms.txt,https://www.durham.ac.uk/llms.txt,200,text/plain,present,# Durham University,robots.txt names no AI crawler
2026-10-30,university,University of Edinburgh,www.ed.ac.uk,https://www.ed.ac.uk/llms.txt,https://www.ed.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,University of Exeter,www.exeter.ac.uk,https://www.exeter.ac.uk/llms.txt,https://www.exeter.ac.uk/llms.txt,200,text/plain,present,# University of Exeter,robots.txt names no AI crawler
2026-10-30,university,University of Glasgow,www.gla.ac.uk,https://www.gla.ac.uk/llms.txt,https://www.gla.ac.uk/schools/law/postgraduate/,200,text/html,absent,,Redirects to the School of Law postgraduate page (LLM degrees)
2026-10-30,university,Imperial College London,www.imperial.ac.uk,https://www.imperial.ac.uk/llms.txt,https://www.imperial.ac.uk/llms.txt,200,text/plain,present,# Imperial College London,"robots.txt names no AI crawler. An earlier read the same day through a different fetch tool showed the file opening with # comment lines, so the served version may vary"
2026-10-30,university,King's College London,www.kcl.ac.uk,https://www.kcl.ac.uk/llms.txt,https://www.kcl.ac.uk/llms.txt,404,text/plain,absent,,
2026-10-30,university,University of Leeds,www.leeds.ac.uk,https://www.leeds.ac.uk/llms.txt,https://www.leeds.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,University of Liverpool,www.liverpool.ac.uk,https://www.liverpool.ac.uk/llms.txt,https://www.liverpool.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,London School of Economics,www.lse.ac.uk,https://www.lse.ac.uk/llms.txt,https://www.lse.ac.uk/llms.txt,404,text/plain,absent,,
2026-10-30,university,University of Manchester,www.manchester.ac.uk,https://www.manchester.ac.uk/llms.txt,https://www.manchester.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,Newcastle University,www.ncl.ac.uk,https://www.ncl.ac.uk/llms.txt,https://www.ncl.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,University of Nottingham,www.nottingham.ac.uk,https://www.nottingham.ac.uk/llms.txt,https://www.nottingham.ac.uk/llms.txt,404,text/plain,absent,,
2026-10-30,university,University of Oxford,www.ox.ac.uk,https://www.ox.ac.uk/llms.txt,https://www.ox.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,Queen Mary University of London,www.qmul.ac.uk,https://www.qmul.ac.uk/llms.txt,https://www.qmul.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,Queen's University Belfast,www.qub.ac.uk,https://www.qub.ac.uk/llms.txt,https://www.qub.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,University of Sheffield,www.sheffield.ac.uk,https://www.sheffield.ac.uk/llms.txt,https://sheffield.ac.uk/llms.txt,404,text/html,absent,,Redirects www to bare domain first
2026-10-30,university,University of Southampton,www.southampton.ac.uk,https://www.southampton.ac.uk/llms.txt,https://www.southampton.ac.uk/llms.txt,404,text/html,absent,,
2026-10-30,university,University College London,www.ucl.ac.uk,https://www.ucl.ac.uk/llms.txt,https://www.ucl.ac.uk/llms.txt,502,text/html,gap,,502 Bad gateway on two reads the same day; not counted as absent
2026-10-30,university,University of Warwick,warwick.ac.uk,https://warwick.ac.uk/llms.txt,https://warwick.ac.uk/llms.txt,200,text/plain,present,# University of Warwick - AI Access Guide,robots.txt names no AI crawler
2026-10-30,university,University of York,www.york.ac.uk,https://www.york.ac.uk/llms.txt,https://www.york.ac.uk/llms.txt,404,text/html,absent,,