AI Mentions Are Not Recommendations: Measure the Difference

Classify what the answer actually did

A brand appearing in an AI answer is evidence that the answer named it. It does not automatically mean the answer recommended the brand, cited its website or helped produce a qualified enquiry. Measure those outcomes separately. Read the surrounding language, identify the user's actual requirement and record whether the answer presents the business as a suitable choice. A useful report preserves those distinctions instead of combining every appearance into one success count.

Start with the sentence, not the logo. In the fictional examples used in this guide, “Apex supplies commercial equipment” is a neutral mention. “Consider Apex for your requirement” expresses a recommendation, although the recommendation may still be unsuitable. Neither sentence is an observed model output. They demonstrate a coding decision that a reviewer can repeat on actual answers without guessing what the brand appearance was supposed to mean.

Use four independent fields: brand named, website cited, recommendation present and recommendation accurate for the requirement. A citation is a reference to a source; it can support background information without endorsing that source's business. A recommendation can appear without a link to the recommended company's website. Keeping the fields separate allows you to notice combinations that a single visibility score would hide, including a highly visible brand attached to an inaccurate description.

Context can change the meaning completely. A negative comparison, historical reference or warning can include a brand name without presenting a buying opportunity. An answer listing available suppliers may create a shortlist, while an answer explaining why one supplier fits the stated constraints makes a stronger suitability claim. Keep the excerpt supporting your classification. The purpose of the rubric is to make the reasoning auditable, not to give every favorable adjective an inflated commercial value.

Before counting, agree which brand spellings, subsidiaries and products belong in the inventory. Record ambiguous matches for review rather than silently adding them. Two businesses can share a short name. A product reference may identify a manufacturer correctly while failing to mention the service business you are measuring. Resolve those identity questions in the rules so changes in the reviewer do not become apparent changes in visibility.

Set denominators before counting

A rate is only interpretable when its denominator is clear. For a defined set of eligible answer observations, mention rate is the number naming the brand divided by that eligible set. Citation rate counts observations citing the defined website. Suitable recommendation rate should use relevant recommendation opportunities, with eligibility decided before you see whether the brand appears. Those measures answer different questions and should not be compared as if they shared the same population.

For a deliberately hypothetical example, suppose a panel contains 20 successful observations. Eight name the brand and four cite its website, producing a 40% mention rate and 20% citation rate within that panel. If only ten observations genuinely ask for a suitable provider recommendation and three accurately recommend the offer, the suitable recommendation rate is 30% of those ten opportunities. These invented numbers illustrate the arithmetic. They are not Fuel results, market benchmarks or evidence about any AI product.

Decide how failed runs, refusals and unsupported requests will be handled before collection. An interface error is not evidence of a recommendation failure, but dropping every inconvenient run can make a panel look better than it is. Retain scheduled runs and their outcomes, then report the valid-answer denominator alongside the number of failures. If a tool does not expose a response for a particular request, say so and keep that limitation visible.

Separate neutral discovery prompts from branded accuracy prompts. “Which suppliers fit these requirements?” tests discovery within the stated conditions. “What does Apex sell?” tests representation of a named entity. Combining them makes the mention rate almost meaningless because the second question already supplies the brand. Both can be useful; they should have separate labels, purposes and reporting rows. A campaign can improve brand accuracy without demonstrating broader discovery.

Freeze the prompt wording, market and observation conditions sufficiently to make comparisons understandable. Record when you deliberately add a new service or buyer segment. An expanded panel may be commercially better, but its average should not be presented as directly comparable with the old panel. Preserve both the original cohort and the new one until you can explain how coverage and results differ.

Code recommendation fit and accuracy

A recommendation must pass a fit test before you treat it as useful. Read the request's constraints: service, geography, customer type, relevant capability and any stated limits. Then compare the answer with verified business information. A provider can be recommended confidently for a location it does not serve, a product it does not sell or an engagement it would not accept. Positive language does not repair that mismatch.

Create an accuracy record with the claim, the supporting business source and the outcome. For the fictional Apex example, the answer might suggest the supplier for installation even though the verified offer covers equipment supply only. Code the recommendation as present and the service-fit condition as failed. Do not erase the recommendation or award full suitable-recommendation credit. The two fields preserve a real distinction: the brand was considered, but the consideration was inaccurate.

Allow an “insufficient information” state. If the answer says a supplier is suitable for a specialized requirement and your business sources do not confirm or contradict the capability, the right conclusion is unresolved. Ask the responsible business owner for an accurate description before deciding. A visibility analyst should not invent a credential or expand service availability simply to make an answer look correct. Update the verified fact inventory when the underlying business changes.

The accompanying conceptual diagram separates mention and citation signals from recommendation classification. Its shapes do not represent measured frequencies. Use the visible rubric below to make the decision explicit: preserve the answer excerpt, identify the relevant requirement and explain why the proposed choice does or does not fit. This is more useful than a positive/negative sentiment label that cannot distinguish a flattering error from an accurate commercial introduction.

For consequential classifications, have a second reviewer inspect uncertain cases. Record the disagreement and the final rule rather than quietly editing the old count. If the rules change materially, recode a comparable earlier sample or mark the series break. A stable measurement process needs consistent definitions more than it needs a complicated score. The audit trail lets an executive understand why a number changed.

Separate citations from referral visits

A cited link, a human visit and a qualified enquiry are separate events. An answer can cite a page without anyone opening it. A visitor can arrive to research a definition and leave with the answer. An enquiry can be irrelevant or duplicate. Build a path from visible reference to landing behavior to business qualification, and identify which stages you can actually observe. Missing attribution should remain unknown rather than being assigned to AI because the timing looks convenient.

Start with the landing page and the visitor's task. Does the page provide the information the answer appears to promise? Can an interested buyer find the relevant service, availability and next action? If the cited source is an educational guide, a contextual route to the offer can help the reader continue when appropriate. A forced enquiry gate can undermine the reason the page was useful in the first place. Evaluate the actual journey before changing its purpose.

Reconcile analytics definitions before combining reports. Google's explanation of key events and conversions distinguishes important on-site actions from the conversions used to evaluate advertising. For this analysis, document the event counted, attribution scope and business qualification rule. A form submission can be a useful operational signal without being a sales opportunity. Keep CRM outcomes, duplicates and unsuitable enquiries visible when assessing commercial value.

Suppose a hypothetical supplier receives three enquiries after an answer cites its installation guide. Without source evidence linking those people to that answer, three is not a measured AI conversion total. Even if referral information identifies the visits, a before-and-after comparison still needs to consider campaigns, seasonality and changes in the offer. Attribution describes a recorded path under a reporting model; it does not by itself establish what would have happened without the exposure.

Use a practical evidence ladder. First verify the cited destination and its content. Next inspect observed referral sessions where identifiable. Then reconcile meaningful actions and qualification with the business record. Finally consider a suitable experiment if you need to estimate additional business caused by an intervention. Each step can inform a decision without pretending it proves every later stage.

Investigate missing or inaccurate recommendations

When suitable recommendations are missing, diagnose the missing condition instead of ordering a generic “AI content” rewrite. Check whether the business is clearly described, the relevant service has a useful public page, availability is accurate and important evidence is understandable in visible text. Those are controllable assets. They can improve the information available to readers and retrieval systems without guaranteeing that an external answer will use them.

For Google specifically, its generative AI optimization guide describes eligibility and ordinary Search foundations rather than a special AI schema requirement. The guidance also rejects the idea that adding llms.txt is required for these features. Treat that as Google-specific documentation. It does not establish how another provider selects sources, and it does not justify promising that a file, format or repeated keyword will secure recommendations everywhere.

Check the relevant inclusion settings before interpreting a missing citation as an editorial problem. Google's Search generative AI control documentation explains inclusion, exclusion and inherited property settings for its covered features. The default is not proof of a particular site's current state. Record the actual property and inheritance where authorized access exists. Keep a search-inclusion decision separate from an organization's AI-training preferences and from the question of whether any particular answer appears.

For an inaccurate recommendation, identify the exact misleading fact and its likely evidence sources. Correct outdated owned descriptions and conflicting service boundaries. Where third-party material is involved, record the discrepancy and pursue only authorized, truthful corrections. Do not manufacture reviews, customer outcomes or location pages to give the preferred answer more apparent support. The repair should improve the available facts even if the next sampled answer does not change.

After a bounded correction, repeat the same relevant observations and preserve the old answers. If the answer improves, describe the sequence and limitations. If multiple sources, prompts and site sections changed simultaneously, you cannot isolate the cause from the before-and-after result alone. Keeping a small change log makes that uncertainty manageable and prevents every favorable response from becoming an invented case study.

Build a decision-ready report

Make the report easy to challenge constructively. Its first page should state the buyer question, prompt-panel coverage, products or interfaces tested, dates, market, observation conditions and eligible counts. Show mention, citation, suitable recommendation and accuracy outcomes separately. Include failures and unresolved cases. A leader should be able to see whether the work concerns discoverability, factual representation, referral behavior or business results without decoding a proprietary headline score.

Attach examples with purpose. Use one accurately coded answer to demonstrate the rule, one unsuitable recommendation to explain the commercial risk, and an ambiguous case if it affects the decision. Preserve the full underlying evidence privately when needed; the report excerpt must not strip away qualifications that change its meaning. An impressive screenshot can illustrate a result, but it cannot represent an entire product, market or period.

Give each finding an owner and an appropriate action. A wrong service description belongs with the content or business-facts owner. An inaccessible cited page may need technical work. A weak conversion route needs an actual user-journey check. An unrepresentative prompt panel needs measurement repair. This separates work the team can complete from external outcomes it can only observe, and prevents the report from becoming a long list of vaguely worded visibility wishes.

If you need implementation support, compare the scope of an AI SEO engagement against those concrete needs. Bring the prompt set, actual answer examples, verified offer boundaries and available referral evidence. Useful scope connects technical eligibility, credible content, accurate business information and measurement. It should also say which findings remain unknown and which outcomes cannot be guaranteed. A service proposal is easier to evaluate when the diagnostic record already identifies the decision.

Close each reporting period with a short change log and the next test. Retain the original panel where continuity matters, label new coverage and explain material product changes. Spend more time on discrepancies that affect suitability or customer decisions than on cosmetic movement in a blended score. The value of reporting lies in choosing a better next action, not in making every line trend upward.

What this measurement cannot prove

A prompt panel is a defined sample of interactions. It is not a direct measure of all user impressions, universal market share or revenue influenced across the internet. Even carefully repeated observations remain tied to the selected prompts, products, conditions and observation window. Report those boundaries next to the findings so readers do not have to discover them in a footnote after a broad claim has already shaped the decision.

An answer naming the brand does not establish that the website caused the mention. The model may draw on different sources or prior information, and the visible citation set may not reveal every input to its output. Similarly, a recommendation appearing after a site edit does not prove the edit caused it. Use the result to refine a hypothesis, preserve the evidence and choose a bounded next step when further measurement is worthwhile.

The distinction between AI mentions and recommendations remains useful even when data is incomplete. You can identify inaccurate service claims, improve a misleading landing page or clarify measurement rules before estimating business impact. These are concrete improvements with observable acceptance criteria. Keep them separate from future answer behavior and commercial outcomes, then report each stage honestly when the evidence arrives.

Turn these definitions into a repeatable collection process with the AI answer study protocol. If an outside provider will run the work, compare the proposal’s scope and evidence requirements so a mention count is not sold as a suitable recommendation or a qualified enquiry.