Explosion of Really Bad Surveys


Share on LinkedIn

Local newspaper columnist James Lileks takes some well-deserved (and hilarious) potshots today at bad surveys.

He also reveals that, back in college, he did a turn as a telephone interviewer. So he at least has some sympathy for what it’s like to be in the survey biz.

It does seem like there’s been an explosion of really bad surveys over the past several years. Personally I blame the confluence of several factors:

  1. Online surveys have gotten really cheap and easy. This means organizations do more surveys but at the same time put less care and effort into designing the survey. Gone are the days when doing anything but the smallest survey meant hiring a market research company (for a minimum of $50K). It’s distressingly common to see surveys riddled with typos, nonsensical questions, and other problems which make it clear nobody could be bothered to do a good job.
  2. Yet the long-form survey style somehow persists. When surveys were rare and expensive it made sense to ask every imaginable question because you needed to squeeze every possible insight from each participant. Today this mindset continues, even though surveys are cheap and common, and it’s not unusual for a consumer to be asked to respond to literally hundreds of questions about a single three-day trip.
  3. And consumers are refusing to respond to bad surveys. Across the industry you hear people complaining that response rates are down on email surveys. But instead of asking the sensible question (“Why don’t people want to take our survey?”), many companies respond by simply sending more survey invitations. To the same badly-designed, overly-long survey that 98% (or more!) of their customers won’t fill out.

These problems won’t be easy to solve, mostly because the root cause is that most organizations don’t care as much about the customer experience as they say they do. This has always been the case–when it comes to customer service most companies talk the talk much better than they walk the walk–but the difference is that today it’s easy to just do a survey instead of doing something.

Republished with author's permission from original post.

Peter Leppik
Peter U. Leppik is president and CEO of Vocalabs. He founded Vocal Laboratories Inc. in 2001 to apply scientific principles of data collection and analysis to the problem of improving customer service. Leppik has led efforts to measure, compare and publish customer service quality through third party, independent research. At Vocalabs, Leppik has assembled a team of professionals with deep expertise in survey methodology, data communications and data visualization to provide clients with best-in-class tools for improving customer service through real-time customer feedback.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here