A new customer types “ChatGPT” into a signup survey. That is useful evidence: the customer remembers an AI assistant as part of the story. It is not a recovered clickstream, and it does not establish that the purchase would have disappeared without that interaction.
Self-reported attribution becomes more useful when the report names exactly what was asked, who answered, and which decision the answer can support. “AI was mentioned by 18 respondents” is a defensible observation. “AI generated 18 customers” introduces a causal conclusion the questionnaire did not test.
Three questions hiding inside one field
First discovery, assistance during evaluation, and the final reason to buy are different events. A buyer might hear about a service from a colleague, use an assistant to compare alternatives, and return through a saved link. A single mandatory channel dropdown has no place to preserve that sequence.
This also explains why a survey and an analytics report can disagree without either being broken. Google defines direct / none as traffic without a clear referral source, with examples including untagged links and visits from offline documents. [4] The channel label describes available traffic-source information. It does not say what the visitor remembers influencing the decision.
Treat the survey as a second measurement with a different target. Retain its answer beside the recorded source. Do not overwrite the analytics channel to make the two systems agree.
Wording changes the observation
Pew Research Center documents how response choices and question order can change answers. In a 2008 election example, 58% selected the economy when offered it as a choice, while 35% volunteered it in an open question. That finding is about a political survey; it is not an estimated bias correction for AI attribution. It shows why aided selection and unprompted recall should not be treated as interchangeable measures. [1]
If a team adds “ChatGPT” to the top of a dropdown, a subsequent increase is ambiguous. It could reflect more AI use, more recognition of an available option, or both. The earlier and later numbers no longer come from the same instrument.
AAPOR recommends short, specific, neutral questions, careful response options, and pretesting. It also cautions that changing wording or mode can mimic a change over time. [2] These principles support a better collection process. They do not supply a universal accuracy percentage for customer memory.
A short survey worth piloting
Begin with this open prompt: “Where do you remember first learning about us?” Add “It's fine if you don't remember” and allow the field to remain blank. Do not precede it with a question about AI; that would introduce the concept before asking for unprompted recall.
Follow with a separate evaluation question: “What did you use to research or compare options before signing up? Select all that apply.” A pilot could include search engines, AI assistants, people the respondent knows, reviews, social content, events, an other field, none, and don't remember. These are proposed options to test with your customers, not a validated universal instrument.
For someone who selects an AI assistant, ask an optional follow-up: “Which assistant did you use, and what did it help you do?” A short explanation can distinguish finding the brand from checking a feature after the shortlist was already made. Avoid asking people to provide private conversation histories when a description is sufficient.
Test whether respondents interpret “first learning” as intended. Retain the exact wording, order, options, timing, and version for each response. If you need to change the form, run old and new versions concurrently on randomly assigned respondents before joining their trends. AAPOR describes this split-ballot approach for evaluating measurement changes. [2]
Keep the denominator attached
Here is a hypothetical month: 1,000 new customers were eligible for a survey, 600 saw it, 200 answered, and 18 named an AI assistant in the discovery field. The same 18 answers produce three percentages. Each tells a different story.
| Calculation | Result | Accurate reading |
|---|---|---|
| 18 / 200 respondents | 9% | Share of respondents naming AI |
| 18 / 600 shown the survey | 3% | Recorded AI mentions per exposed customer |
| 18 / 1,000 eligible customers | 1.8% | Recorded AI mentions per eligible customer |
The unanswered cases are unknown. Treating every nonrespondent as a non-AI customer would be an assumption; treating respondents as representative of all customers would be another. Neither follows from the counts.
AAPOR's standard definitions distinguish coverage, measurement, and nonresponse problems. It explicitly notes that a response rate alone does not establish how much nonresponse error exists, or even whether it exists. [3] Report survey exposure and completion, then investigate which customers are missing.
For an operational check, compare respondents and nonrespondents on information already collected appropriately: plan, country, customer type, signup week, or purchase value. A survey shown only after a desktop checkout may miss a different mix of customers than one shown across purchase paths. A good completion percentage cannot repair a group that was never invited.
Calibration means looking for disagreements
Keep original text alongside coded categories. “Google AI” may mean an AI Overview, an assistant, or an imprecise memory of search. Preserve an ambiguous category until the answer supports something more specific. Have a second reviewer check a small sample of classifications and document the rules for disagreements.
Then compare survey answers with available referral and campaign records for the same cohort. Look at cases with an AI referral but no AI mention, and AI mentions without an AI referral. These are informative mismatches, not entries to delete. A source record can be incomplete; memory can also be incomplete.
This comparison can identify collection problems and generate hypotheses. It cannot establish a single truth score when neither measurement observes the whole journey. If the business question is incremental revenue, a separate causal design is needed; survey mentions alone do not provide the missing counterfactual.
Use the result to learn customers' language, identify discovery paths worth investigating, and decide what to test next. Publish the wording, dates, counts, coding rules, and limits alongside the trend. A modest claim with a visible denominator is more useful than a precise-looking AI revenue share built on an unexplained form.