How to use AI Answer Stability
Ask an AI engine the same question twice and you can get two different answers. AI Answer Stability asks ChatGPT, Gemini, Perplexity, Claude and Google AI Mode the same question several times each, then counts: how often each one names you, and how sure that count is.
Why one answer isn't enough
Most AI visibility scores rest on a single answer per engine. But the brands an engine offers change from run to run, often in the middle and tail of the list. One answer that names you, or doesn't, is a coin toss you've seen land once.
So the tool runs the question several times and reports a rate with an interval. “Named in 7 of 10 answers, somewhere between 40% and 89%” tells you both where you stand and whether the number is firm enough to act on.
Running a check
- Question. Write it the way a customer would ask an AI assistant, e.g. What is the best server-side tagging platform for Google Tag Manager? Between 8 and 200 characters.
- Your brand, and optionally competitors, separated by commas. These are the brands you want tracked in every answer, even when they're only named in passing.
- Runs per engine (Pro). More runs, tighter interval.
- Press Run check. Progress shows per engine as answers land, and partial results appear as they come in. A check usually takes one to three minutes, and a little longer when an engine is slow to answer. You can leave the page: the address carries the check, so reloading picks it back up.
Reading the results
The four figures
- Named at all: the share of answers, across every engine that answered, that named your brand.
- 95% interval: the range that share very likely sits in (a Wilson score interval). With few runs it's wide; that's honesty, not a bug.
- Named first: how often your brand was the first option offered.
- Answer stability: how much the list of brands overlaps from one answer to the next, from 0 (different every time) to 1 (the same every time).
When the interval is wider than 30 points, the tile says it's too wide to report. Run more times per engine to narrow it.
Mentions by run
The map has brands down the side and one column per answer, grouped by engine. A shaded cell means the brand was named, and the number is its position in that answer: the lightest blue is named first. A blank cell means it wasn't named. A dashed cell means that run returned no answer; it's left out of every figure rather than counted as “not named”, and it isn't charged. Hover it to see why: the engine was busy (tried again after a short wait), took too long, or the day's check limit was reached. An engine with fewer than three answers shows “too few answers” instead of a stability score, because two answers are only one comparison.
Brands you didn't enter appear too, when the answers offered them as options. That's usually where the surprises are: the competitor you didn't know the engines rate.
The map works from the keyboard: Tab into it once, then move with the arrow keys.
Mention rate by engine
Each engine's rate for your brand, with its own interval. This is where engines disagree; it's common to see one name you almost every time and another barely at all.
Cited sources
The sites each engine cited, and in how many of its answers. Sites cited in most runs are the ones shaping the answer; worth knowing whether you're among them.
History and Run again
Once a check is a day old, Run again asks the same question with the same brands again (it costs the same as a new check). Every run of a question is kept together, and History shows them one row each, newest first, with each run's 95% interval. Read it by the width of the bars: a move smaller than the bars are wide is likely noise. Click a row to open that run; an older run says so at the top. Run again waits a day because the same question asked twice in one day reuses that day's answers.
Free and Pro
- Free account: three checks a month, at 5 runs per engine, tracking your brand and up to 3 competitors.
- Pro: more checks each month, 10 or 20 runs per engine for a tighter interval, and up to 10 competitors. See plans for the current allowances.
A check that fails doesn't count, and a question someone already checked today is shared rather than run again (see below), so it costs nothing.
Method and limits
- These are API answers, collected through each engine's API with web search on. They can differ from what someone sees in the ChatGPT, Gemini, Perplexity or Claude apps, which add their own settings and memory. Google AI Mode is its results page.
- Cited sources are those found by the web search run for these API answers. The apps may search differently.
- Runs are set to the UK, except Gemini, which can't be pinned to a country this way.
- Your brands are matched in the answer text directly. Other brands are the ones an answer offers as options for the question, as judged by a language model. Brands named only in passing, and brands named in the question itself, are left out.
- Answers are shared for 24 hours: if the same question was checked today, you see those answers and the time they were collected. Your brand names are yours; nobody else sees them.
What leaves your browser
Your question goes to the AI engines to be answered, and each answer goes to a language model to pick out the brands it offers. Your brand and competitor names stay on our servers: they're matched against the answers there. Keep personal details out of the question. The details are in the privacy policy, section 4a.