Impact Lab

Subscribe Now to Our Free Email Newsletter
October 29th, 2007 at 8:46 am


Seth Godin:  Thanks to the internet, surveys are a lot cheaper than they used to be.
And the prevalence of roll-your-own amateur surveys means that we all
have a lot to learn.

A survey can teach your customers or it can help you learn from them.

And it might be a real survey, or it could be a census.

The traditional understanding of a survey is that the goal is to
LEARN from your population and that you will ask a scientific sampling,
not everyone.

You can TEACH people with a survey, though, simply by asking them
questions that help them notice things they never noticed before. "Do
your prefer option A or option B," might just be a way of getting
people to notice that you even have an option B. 

The very act of asking a question may change the experience for the
customer. One small firm I know shows prospects a book of testimonials.
Then they say, "I hope that when we’ve completed our job for you,
you’ll be willing to write one too." That seed increases the likelihood
that people are going to be looking for something good to say, which
increases the likelihood that they’ll enjoy the event.

Of course, this can spiral out of control pretty quickly. Push polling,
in which faux pollsters call people up and ask them questions with
patently false assumptions about competing candidates, for example, is
just wrong.


But don’t forget the hybrid solution, which I call a Trident survey.
"4 out of 5 dentists surveyed recommend sugarless gum…" Hardly
scientific, but publishing the results made dentists feel better about
recommending the gum and made people with teeth happier about chewing

Which leads to the question of how many people you’re going to ask.
Professional surveyors almost never ask everyone. They carefully select
a representative sample (not so easy) and invest in each interview,
thus scaling the results for the whole population. That’s how Nielsen works.

There are plenty of inexpensive ways to ask EVERYONE your question,
though. That turns your survey into a census. A census only works, of
course, when the response rate is close to 100%, because uneven
response rates are going to skew your results.

Analytics is a form of census survey. You can track how everyone who visits your website behaves. Focus groups,
on the other hand, are a poor use of just about anyone’s resources,
because they are inherently not surveys at all. Without a skilled
moderator, all you get is useless (but extremely vivid) data.

So, I guess I’d summarize the survey question by identifying four kinds of surveys that are worth doing:

  • Census surveys designed to teach your market, not you. The act of asking the question is a marketing tactic.
  • Public non-scientific surveys (or census surveys) in which
    publishing your results to the group helps change the group’s behavior.
  • Professional surveys designed to extract really meaningful data from a small group.
  • Census-based analytics in which you are extracting data about behavior from the entire group.

One last warning: round off. 98.2% is a bogus result. "Most" is a
lot more accurate. The ultimate purpose of most traditional surveys is
to make decisions. Alas, your audience is often the very worst group to
help you make a decision. When you let a survey be presented as
accurate, it becomes the silent decision maker in the room and leads,
often, to mediocre products for the middle of the market.

Via Seth Godin


You must be logged in to post a comment.

Are certificates more valuable than college degrees?