Reading Post Purchase Surveys Without Fooling Yourself
How did you hear about us is cheap, fast, and biased in at least four known ways. Used with those biases named, it still beats platform attribution for some questions. Here is the honest way to run it.
Post purchase surveys get quoted like census data and collected like a comment card. Response rates are often a small slice of orders, the respondents are not typical, memory fills in a tidy story, and the first channel a customer remembers is rarely the whole path. And still, for the question of where demand actually starts, a consistent survey will correct platform attribution in ways that save real money. Both things are true. The skill is holding them at once.
We have seen surveys kill a channel that a holdout later proved was working, and surveys expose a podcast and word of mouth engine that no platform could see at all. The difference was never the tool. It was whether the team named the biases in the same breath as the result.
Related reading on this site: Attribution When Customers Switch Devices Mid Journey, Attribution for Podcast Sponsorships and Host Read Codes, Attribution Windows: The Setting That Quietly Rewrites Results.
Name the four biases before you chart anything
Selection: happy, engaged buyers answer at higher rates, so channels that create fans over report. Recall: the memorable touch gets credit, the forgettable ad that started the path does not. Recency: the last thing seen before purchase crowds out the first. Suggestion: your answer list teaches people what to say, and any channel missing from the list will be undercounted to zero.
None of that makes the data useless. It makes it directional. A channel doubling in survey share over two quarters, on a stable question, is signal. A channel at 11 percent versus 13 percent this month is weather. Report the sample size and response rate on the same chart as the shares, every time, no exceptions.
Keep the question frozen, and the options honest
The value compounds from consistency. Changing the wording or the option order resets your baseline, so freeze the core question, put the options in a sensible order with an other and a free text field, and resist seasonal edits. Free text is where new channels first appear: a creator, a community, a retail encounter that no platform will ever report.
Include the uncomfortable options. Direct, typed it in, and a friend told me are real answers for many stores, and omitting them forces that demand into channels that did not earn it. A survey designed to flatter the media plan will succeed at exactly that.
Use surveys to aim tests, not to replace them
The strongest pattern: survey suggests a channel is under credited, you run a holdout or a spend change in one region, orders answer the question. Survey suggests a channel is over credited, same move in the other direction. The survey is the hypothesis generator with a fast feedback loop, and experiments remain the judge.
Publish the misses too. When a survey led test proved the survey wrong, say so in the readout. A team that only hears survey wins will over trust the next result, and the survey's real value, cheap correction of platform blindness, depends on it staying a tool with a known error rate, not an oracle.
Frequently asked questions
What response rate makes a post purchase survey usable?
There is no magic threshold. A consistent question with a stable, reported response rate, even a modest one, supports directional readings over time. What makes it unusable is quoting shares without the sample size, or changing the question while comparing across the change.
Why do surveys over credit word of mouth and creators?
Because they are memorable. Selection and recall bias favour channels customers can name and feel good about, while forgettable early touches, like a plain search ad weeks earlier, fade. That is why survey results aim experiments rather than settle budget questions alone.
Should the survey ask for the first or the last touch?
Ask how the customer first heard of you if the goal is demand origin, and phrase it exactly that way. Mixing first touch answers with platform last click reports without naming the difference produces most of the arguments teams have about this data.
How should survey results be reported to leadership?
Shares with sample size and response rate attached, trended over quarters, with the known biases stated in one line, and any experiment the survey prompted reported alongside. That format keeps the tool credible and prevents single month noise driving spend.