Local newspaper columnist James Lileks takes some well-deserved (and hilarious) potshots today at bad surveys.
He also reveals that, back in college, he did a turn as a telephone interviewer. So he at least has some sympathy for what it’s like to be in the survey biz.
It does seem like there’s been an explosion of really bad surveys over the past several years. Personally I blame the confluence of several factors:
- Online surveys have gotten really cheap and easy. This means organizations do more surveys but at the same time put less care and effort into designing the survey. Gone are the days when doing anything but the smallest survey meant hiring a market research company (for a minimum of $50K). It’s distressingly common to see surveys riddled with typos, nonsensical questions, and other problems which make it clear nobody could be bothered to do a good job.
- Yet the long-form survey style somehow persists. When surveys were rare and expensive it made sense to ask every imaginable question because you needed to squeeze every possible insight from each participant. Today this mindset continues, even though surveys are cheap and common, and it’s not unusual for a consumer to be asked to respond to literally hundreds of questions about a single three-day trip.
- And consumers are refusing to respond to bad surveys. Across the industry you hear people complaining that response rates are down on email surveys. But instead of asking the sensible question (“Why don’t people want to take our survey?”), many companies respond by simply sending more survey invitations. To the same badly-designed, overly-long survey that 98% (or more!) of their customers won’t fill out.
These problems won’t be easy to solve, mostly because the root cause is that most organizations don’t care as much about the customer experience as they say they do. This has always been the case–when it comes to customer service most companies talk the talk much better than they walk the walk–but the difference is that today it’s easy to just do a survey instead of doing something.
