Open-ended questions have long been a part of customer experience surveys. However, there is a massive amount of inconsistency in the way these kinds of questions are used. Some companies have a single open-ended question in their survey and keep the overall survey quite short. Some, at the other end of the spectrum, use multiple open-ended questions as part of a lengthy survey.
Regardless of the number of open-ended questions, these questions can be a source of frustration and a cause for early termination of the survey for customers. The input provided by customers to these questions can be a source of frustration for companies and the people who need to use the information provided. Unhelpful responses can be short, nonspecific, or lacking in sufficient detail.
Despite the frustration that open-ended questions in CX surveys often cause for both customers and companies, customers want to be able to tell their stories, and companies need the context available from open-ended feedback. It’s time for companies to follow some best practices in this area and take advantage of tools now available that will enable a better experience for both customers and companies.
MaritzCX has created SmartProbe™, a new way of getting unstructured feedback within electronic surveys. In some ways, SmartProbe harks back to the days of telephone interviews, when interviewers asked open-ended questions and were trained to follow up and probe more deeply in response to customers’ answers.
SmartProbe uses a patented approach allowing for follow-up probes that are in context, both in terms of the content and the length of the comment. In many ways this is an improvement over not just the way open-ended questions are done as part of electronic surveys today, but also compared to the way that probing has been done during live interviews as well.
Work that we at MaritzCX have done for clients has shown the power of SmartProbe. SmartProbe significantly increases both the number of words in customers’ responses and the depth of the comments that are provided. As the graphic below shows, without SmartProbe we get an average of just 7.9 words per response. With SmartProbe, the number of words jumps by 23 words. Without SmartProbe, 34% of people gave no comment. With SmartProbe, the number of customers with no comment declines to 11%.
Source: Client Ongoing Project
It’s not just the number of words that change; more importantly, SmartProbe increases your ability to understand your customers’ experience and take action. Note the examples below, and the difference between what was initially provided and the additional commentary provided after the SmartProbe. The difference is plain as day.
Pre-SmartProbe: Open more windows to serve the people
Post-SmartProbe: Open more windows to serve the people. I waited probably more than one hour from the minute I walked in until the time I reached the window.
Pre-SmartProbe: N/A
Post-SmartProbe: I had several boxes. It would have been nice to have a cart available instead of making three trips.
Pre-SmartProbe: Not be incredibly rude, nasty, unprofessional as (name) was this afternoon.
Post-SmartProbe: Not be incredibly rude, nasty, unprofessional as (name) was this afternoon. She was uncaring from the moment I got to the counter. I was making a shipment for work and would not accept our business credit card. She made me pay with my own personal card and then said my signature on my debit card did not match that on my ID even though it was the same name and the picture on my ID was clearly me. She then walked to the back … with my debit card and ID without telling me where she was going. She came back and said she spoke with the manager, but then when I said I wanted to speak with the manager she said he was not there. She was very disrespectful, and talked down to me. She cost me money out of my own pocket when I was doing work for business. She wasted my time, my company’s time, and the time of the people behind me in line. I was very very unsatisfied with the way I was treated, and the service I received.
Here are some best practices to consider when using unstructured questions in surveys:
One of the concerns that researchers have about open-ended questions is that they can increase the length of the survey or cause a higher break-off rate. Following the best practices above will help you optimize value of your unstructured response from open ended questions, while minimizing impact on survey length, respondent fatigue and early breakoffs from the survey.
After all, would you rather have responses from four open-ended questions that were potentially misleading or not at all useful, or have a response from just one open ended question that is much more detailed, specific and most importantly actionable? The choice seems pretty obvious
In CX measurement and management, context is everything. It’s key to understanding customers’ needs and responding effectively, it’s crucial for effective coaching of frontline employees, and it enables you to take more precise action to improve CX.
Don’t settle for less. Maximize your customer understanding with open-ended questions. Have more impact.
