1–2 minutes

Does ChatGPT give the same answer twice? Ask before you trust a shortlist

Same prompt, fresh chat, different answer. Run an AI shortlist twice and keep only what appears both times.

Written 14 October 2026.

Not always. Before acting on an AI shortlist, ask again in a new chat with the same wording. What appears both times is the steadier part of the answer.

What I measured

On 9 October 2026 every accounting software prompt ran twice on ChatGPT and twice on Gemini, each in a fresh temporary chat. Sage Accounting was named in one generic ChatGPT run and not in the other. What held: ChatGPT picked FreeAgent for the self-employed plumber in both runs. Study and CSV.

How to do it

1. Open a new chat.
2. Paste the same wording.
3. Keep what appears both times. Treat the rest as unsettled.

If you track AI visibility, the same goes for your numbers: one run shows what’s possible, not what’s typical (Five labels every AI visibility number needs).

What I will not claim

  • How often answers change. Two runs per prompt is enough to see answers move, not to measure it.
  • That what appears twice is right. Repeated isn’t the same as correct.
  • Anything beyond two engines, three prompts and one evening.

This is hack 8 on the search hacks page.

Implement with AI

Describe what you want the AI to implement or change for this post. It opens with the article and your request already filled in.

Implement with: ChatGPT Perplexity Claude Grok