Does ChatGPT give the same answer twice? Ask before you trust a shortlist

Written 14 October 2026.

Not always. Before acting on an AI shortlist, ask again in a new chat with the same wording. What appears both times is the steadier part of the answer.

What I measured

On 9 October 2026 every accounting software prompt ran twice on ChatGPT and twice on Gemini, each in a fresh temporary chat. Sage Accounting was named in one generic ChatGPT run and not in the other. What held: ChatGPT picked FreeAgent for the self-employed plumber in both runs. Study and CSV.

How to do it

1. Open a new chat.
2. Paste the same wording.
3. Keep what appears both times. Treat the rest as unsettled.

If you track AI visibility, the same goes for your numbers: one run shows what’s possible, not what’s typical (Five labels every AI visibility number needs).

What I will not claim

  • How often answers change. Two runs per prompt is enough to see answers move, not to measure it.
  • That what appears twice is right. Repeated isn’t the same as correct.
  • Anything beyond two engines, three prompts and one evening.

This is hack 8 on the search hacks page.