What to Do with Free Responses?


Share on LinkedIn

Free response questions (aka open-ended questions, aka verbatims) are a powerful tool for finding out what’s top-of-mind with customers. The traditional method for dealing with them is to have an intern perform the mind-numbing task of reading each comment and categorizing it, in order to do some statistical analysis. This is what we did for the recent NCSS report on customer complaints, though I and one of my managers played the roles of interns.

In the past I’ve played around with some text analysis software to try to reduce some of the burden of this chore, and found that it didn’t help much for the size of data set we’re working with (usually under 10,000 free responses). It took a little too much time and effort to train the software to give good results. On the other hand, if we had hundreds of thousands (or even millions) of comments I could definitely see the value.

For surveys in an interview format it makes sense to have the inteviewer provide a preliminary categorization of the customer’s comment. The interviewer won’t know as much about the company’s internal business processes as an insider, but will probably get it right 90% of the time. The interviewer can also flag ambiguous cases for further review.

There’s also a limit to what you can get from a free response question. When you ask for comments, most customers will give you just the one thing most important to them at that moment. So this is not a very good way to track the actual incidence of specific problems, since you miss customers who may have experienced the issue but didn’t happen to mention it.

So when something interesting develops in the comments, it’s a good idea to start asking about it specifically in the survey. You will get a much better read on whether there’s a problem with (for example) language skills or the IVR if you ask every customer whether they experienced that issue or not. If it turns out you don’t have a problem then the question can be removed.

Republished with author's permission from original post.

Peter Leppik
Peter U. Leppik is president and CEO of Vocalabs. He founded Vocal Laboratories Inc. in 2001 to apply scientific principles of data collection and analysis to the problem of improving customer service. Leppik has led efforts to measure, compare and publish customer service quality through third party, independent research. At Vocalabs, Leppik has assembled a team of professionals with deep expertise in survey methodology, data communications and data visualization to provide clients with best-in-class tools for improving customer service through real-time customer feedback.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here