Checked 6 November 2026, 12:02 to 12:07 UK. 24 hours after the Bank of England raised Bank Rate to 4%, all eight answers from four AI engines gave 4%. None was stale. P37 said at least one would be. It is scored a miss, in red on the board.
It is the first call on this site to be scored, and it is wrong.
The call
- ID: P37. Called 11 October 2026, on the study plan. Check date: 6 November 2026.
- Claim, as written: “24 hours after the announcement, at least one of the four engines will give a stale answer in at least one of its two runs.”
- Hit: one or more of the eight 24-hour answers is stale.
- Miss: none is stale, or the round isn’t run and published by 13 November 2026.
- Stale means the answer names an earlier decision as the latest, or gives a rate the 5 November decision replaced.
What happened
0 of 8 answers at T+24h were stale. Six named the 5 November decision. Two gave 4% without naming any decision, which the plan classes as unreferenced, not stale.
| Engine | Chat 1 | Chat 2 |
|---|---|---|
| ChatGPT (free tier, temporary chat) | caught up | caught up |
| Google AI Mode (signed in) | unreferenced | unreferenced |
| Claude (incognito, web search on) | caught up | caught up |
| Gemini (temporary chat) | caught up | caught up |
The Bank’s own page at 12:02 UK: “Current Bank Rate 4%”, next due 17 December 2026.
Every round so far, from the same CSV:
| Round | Run (UK) | Caught up | Unreferenced | Stale | No figure |
|---|---|---|---|---|---|
| T+1h | 5 Nov, 13:20–13:26 | 7 | 0 | 1 | 0 |
| T+3h | 5 Nov, 15:03–15:08 | 2 | 3 | 2 | 1 |
| T+6h | 5 Nov, 18:02–18:08 | 5 | 3 | 0 | 0 |
| T+24h | 6 Nov, 12:02–12:07 | 6 | 2 | 0 | 0 |
The T+1h round ran 20 to 26 minutes late because the browser was disconnected at 13:00; that is logged in the CSV. A day-before round was meant to run on 4 November and didn’t. A substitute ran on the morning of 5 November, is marked late and excluded, and doesn’t affect P37.
Data: CSV · screenshots and method, including the 5 November clarification: study plan.
Why it missed
Fact: the test needed one stale answer in eight at T+24h. There were none. It wasn’t close in the sense that matters: no answer at T+24h gave 3.75% or named the September hold as the latest decision.
What I assumed
| Assumption | Held? |
|---|---|
| Some engine would lean on an older index or cached page for a day | Failed. Every engine was on 4% by T+6h at the latest. |
| Stale answers would cluster early and fade | Held. All three stale answers came in the first three hours. |
| Two chats per engine would be enough to catch a stale engine | Can’t tell. See below. |
| The engines would stay the same through the study | Failed for one. At T+3h, Claude’s model picker showed “Fable 5, High”. Every other round showed “Opus 5.5, Medium”. Both stale Claude answers came from that round. |
The third row matters most. Zero stale answers out of eight does not mean the true stale rate at 24 hours is zero. With eight answers, a 95% Wilson interval on 0 of 8 runs from 0% to about 32%. The miss is a fact about these eight answers. It is weak evidence about the engines.
The fourth row means I can’t separate time from model for Claude at T+3h. Both stale answers said the Bank had announced a hold that day. One also said the Bank cut rates on 30 July 2026, which is wrong. Whether that was the hour or the model, this run can’t say.
What the evidence says that I didn’t expect
Fact: Google AI Mode stopped dating the decision. From T+3h onwards, five of its six answers gave the right rate and never said which decision set it. The date was in the sources panel, not in the answer. That matches what this site found on 10 October: the undated answers then also all came from Google’s engines.
Fact: wrong pages were still being cited after the answers had moved. At T+3h, AI Mode’s sources panel included a bank’s page still saying “Currently 3.75%”. At T+24h, one Claude answer said “one page also still says the rate was held at 3.75%” and set it aside.
Fact: the engines disagreed on a detail nobody asked about. Answers gave the vote as 8–1 or 9–0, and two Claude answers said reports disagreed. The vote is not part of the classes and is not scored here.
Opinion: the interesting window for live figures is hours, not a day. A call set at T+24h was testing the wrong part of the curve.
What this does not show: why any engine lagged, anything about other figures or other engines, anything about traffic, or anything about what a signed-out user would see.
What I’d change
- In how the call was written: the window. I’d set the check at T+3h, where the stale answers actually were.
- In the sample: two chats per engine per round is too few to say anything about a single engine. Eight answers can rule out a high stale rate, not a low one.
- In the method: record the model shown at every round and treat a change as an event. It is now a column in the CSV; it should be a rule in the plan.
- Next call on the same subject (suggested, not yet on the board): “At the 17 December 2026 decision, at least one of the eight T+3h answers will be stale.” If I register it, it gets its own ID, test and check date, and P37 stays as it is.
The 48-hour and 7-day rounds still run on 7 and 12 November. They will be added to the plan page. They don’t change P37.
Board line
P37 · called 11 Oct 2026 · checked 6 Nov 2026 · MISS · 0 of 8 T+24h answers stale
Correction log (already live): 6 Nov 2026 · P37 · Scored as a miss. In the 24-hour round (12:02 to 12:07 UK), six answers named the 5 November decision, and two gave 4% without naming it. The claim is unchanged.