Your competitor appears in a ChatGPT answer. By lunch, you have three tabs open for software that will tell you how often it happens. Visibility reporting is the measuring instrument, not the treatment. A chart can show you the missing citation; it cannot change the answer.

In To the Founder Who Saw a Rival in ChatGPT: Don’t Buy the Dashboard, I argued for paying for work that changes citations rather than a prettier view of the problem. That advice is easy to agree with and hard to check. So here is the second look: a 30-day test you can run on a tool, an agency or your own team.

Keep a fixed set of buyer prompts. Split it into prompts you will work on and prompts you will leave alone. Record the answers on a schedule, log the work, then compare the two groups. If the treated prompts improve while the controls do not, you have a reason to keep paying for the work. If all you get is a better chart, you have your answer too.

Question: did anything change besides the report?

Every pitch in this category blurs two jobs into one price. The first is counting: did your brand get cited, which URL appeared, and who displaced you? The second is changing the conditions behind that answer: fixing an inaccessible page, writing something an engine can quote, or earning an accurate mention in a source it already cites.

I used to tell clients a new report would sharpen their decisions. I was wrong. A report helps only when someone acts on what it finds. This test grades that action, not the ruler.

Before Day 1, write down what would make you continue paying. If a hypothetical treated group starts with citations on 3 of 20 prompts, you might require at least 8 of those 20 at Day 30 while the control group moves by no more than 2. Those are decision thresholds for your test, not a promised result. Pick a bar that matters to your business and keep it fixed. Otherwise, Day 30 becomes an exercise in moving the goalposts.

Setup: fix the prompts before you touch the pages

Choose 20 to 40 prompts a buyer would actually type, not 100 versions of your brand name. Build the set from three buckets:

  • Brand prompts: your name against a competitor. Start with five.
  • Category prompts: questions such as best X for Y, X pricing comparisons and X alternatives. Use ten to fifteen.
  • Problem prompts: questions a buyer asks before they know your category exists. Use ten to fifteen.

Adjust the buckets to fit your total, then freeze the wording. The mix builds on a baseline protocol that checks 10 to 30 buyer prompts on a fixed weekly schedule. If you only track your own name, you can pass a test that never risked failing.

In week zero, run each prompt once per engine you intend to track. For every answer, record whether you were cited, the cited domain and URL, where your brand appeared, and the sentence carrying the claim. Keep results for ChatGPT, Perplexity and Google AI Overviews separate. Semrush tracked 230,000 prompts and found different citation mixes by engine. A prompt earns its comparison only against the same engine and the same wording next week.

Now split the prompts into treated and control groups. Keep the groups reasonably similar across your three buckets. More important, check for overlap before you begin: if two prompts point to the same page or the same recommending source, an edit aimed at one may affect both. Move overlapping prompts into the same group or note the overlap as a limitation. A control prompt is not a control if you quietly treat its source.

Instrument: make the weekly readings repeatable

One screenshot proves very little. A study of 2,961 runs found the same brand list in the same order only about 1 in 1,000 times. Run the full set on the same day each week, using the same wording and engines, and record the individual answers as well as the trend.

You can buy that measuring instrument or build a cheap one. The low-cost version reads prompts from a Sheet, sends them through APIs and writes brand presence back to the Sheet. The baseline protocol estimates low single-digit monthly costs for 25 weekly prompts, with Perplexity Sonar at $5 to $14 per 1,000 requests as its main variable cost. If you want the build spelled out, use this weekly AI brand monitor with Google Sheets and three APIs. No budget for that? Run the prompts by hand on Tuesday mornings and log them the same way.

Whichever instrument you choose, keep it consistent for the month. Do not compare an API result from week one with a casually reworded browser query in week four and call the difference progress. Never grade the scale as if it were the treatment.

A bathroom scale beside a lab notebook, illustrating measurement rather than treatment

Treatment: check access, then change what can be cited

Work only on sources associated with your treated prompts. Record what you plan to touch before you touch it; leave control-associated sources alone for 30 days. If a change could affect both groups, mark it in the log rather than pretending the separation held.

Start with access. I have burned a week fixing copy on a blocked page, so I check this before editing sentences now. Inspect robots.txt for accidental blocks on /blog/ or /resources/. Confirm the relevant sitemaps are submitted in Google Search Console and Bing Webmaster Tools. A technical walkthrough of AI search mentions makes the case for checking the indexes and sources an engine may use; a Google ranking alone does not settle whether the page is available where you need it.

Separate training access from search access while you are there. Cloudflare distinguishes GPTBot for training from OAI-SearchBot for search, with controls by bot type. If you change a bot rule, write down which rule changed. Blocking training is not the same decision as blocking access that may support search citations.

Once the page is reachable, give the engine something useful to quote. For a treated prompt cluster, put a direct 40- to 60-word answer near the top of the relevant page. Add specifications and limits in readable text. Where a buyer expects proof, use a number or a named source you can stand behind. A peer-reviewed GEO study reported visibility lifts of up to 40% from techniques including statistics, citations and quotations; that is a reason to test quotable material, not a forecast for your month.

Rewrite the answer block before polishing its markup. FAQ, Article and Organization schema, alongside entity listings, can help describe a page. None makes an empty answer worth citing. Fix access first, then fix the answer.

Off-site treatment: inspect the rival citation

Your own page may not be the source doing the recommending. Different engines draw on different indexes and sources, and Semrush’s citation analysis found different mixes across them. Do not assume a homepage rewrite will displace a community thread or directory listing.

Pick two treated prompts where a rival appears. Open the cited URLs behind those answers and write down who is making the recommendation. If a thread or directory is doing the work, pursue an accurate mention or a listing where vendors are already compared. Do not fake it. One honest spec sheet in the right thread beats ten paragraphs of homepage adjectives.

Keep this work inside the treated set as far as you can. If the same third-party page also serves a control prompt, record the spillover. The log is there to keep you honest, not to make the experiment look tidier than it is.

Change log: no dated work, no credit

Give every intervention one row: date, URL or source touched, what changed in plain language, and which treated prompts it targets. A useful row looks like Oct 18 / /pricing / added 55-word answer block with limits and starting price / prompts 12 to 15. Worked on SEO does not.

Keep this log beside the weekly answers. On Day 30, you need to see whether movement followed a specific change, not reconstruct a month of work from someone’s memory. If a vendor is running the test, ask for this format before Day 1. For a longer look at what execution work can include, see Tools That Help You Get Featured in Google AI Overviews, Not Just Monitored. A reporting-only tool has observations to log, not interventions.

Weekly readings: compare treated prompts with controls

On the same day each week, run the full set and record:

  1. Citation share for treated prompts and controls: how many prompts in each group cited your brand.
  2. Your cited URLs: which page appeared for each prompt and engine.
  3. The displaced or competing sources: which domains and pages appeared where you did not.
  4. Any changed conditions: edits, bot rules, listings or obvious overlap between groups.

That third item is your next fix list. Tracking that shows the domains and pages appearing instead of yours turns a red dot into a URL you can read.

Expect wobble. Identical prompts can return different sources on different runs. If you repeat a prompt within a week and it flips, record both readings; for your running tally, count that prompt as half credit and keep the raw answers. Do not rewrite a page because Tuesday looked bad and Thursday looked good. Look for a treated line that rises across the month while the control line stays comparatively flat. One answer is weather.

Day 30: decide what the result changes

Start with the pass rule you wrote before Day 1. Suppose, purely as an example, that treated prompts go from 3 of 20 cited to 9 of 20 while controls go from 3 of 20 to 4 of 20. If your log also shows relevant edits before the movement, you have evidence worth acting on. You do not have proof that a single edit caused every new citation; prompts and sources can change outside your control. Keep funding the work that appears to be helping, and keep measuring it.

Other outcomes call for different decisions:

  • Both groups climb: something beyond your targeted work may have shifted the citation pool, or your treatment may have spilled into the controls. Check the raw answers and log before claiming a win. Semrush documented a sharp change in ChatGPT’s Reddit citations over a matter of weeks; no single brand edit explains a platform-wide swing. Run another month before making an annual commitment.
  • Neither group climbs: the treatment may have missed the sources behind the answers, or it may need longer to register. Reopen the rival’s cited URLs. If your edited pages never appear among the relevant sources, aim the next round at that mismatch instead of buying more reporting.
  • The controls climb but treated prompts do not: revisit your chosen pages, access checks and off-site sources. Do not call the month a pass because the dashboard contains more green.

Thirty days gives you a spending decision, not a durable lead or a revenue forecast. Twenty to 40 prompts are a small sample, answers wobble, and the engines can change their source mix without telling you. Use the test as a filter, then keep checking the work that clears it.

Put the same protocol in front of every candidate

Before Day 1, ask a tool provider, an agency and your own team the same three questions:

  1. What will you change in weeks one and two for the treated prompts?
  2. Who owns the page edits, access checks and off-site work?
  3. What will the weekly change log show?

Judge the answers against the same pass rule. A tracker may give you an excellent chart but no edits. An agency’s strategy document is not a changed URL. Your own team’s good intentions are not a dated row either. If nobody can show what changed and which prompts it targeted, you bought measurement.

This is where groas makes a stronger case than a reporting-only tool or a retainer built around periodic reviews: its specialized models execute content and search optimization work continuously, a named human strategist sets direction, and actions are logged. That is a mechanism to test, not a result I can claim on your behalf. Demand the same prompt split, the same log and the same Day 30 decision from groas as from anyone else.

If treated prompts move and controls do not, keep paying for the work and stop mistaking reporting for the win. If neither moves, stop polishing the instrument and inspect the sources the rival answer actually cites. If both move, keep the test running before you commit further. Control is not a dashboard full of sightings. It is a dated log of changes beside a citation line that gives you a reason to make the next decision.