I recently discussed the importance of getting Voice of the Customer (VOC) feedback and common methods, such as surveys, to understand customer perceptions and expectations across different touch points. To be effective and acquire actionable insights, questions must be designed with best practices applied. I also recommend a “test & learn” approach.
Writing and launching surveys has become a science and an art. I have seen cases where surveys have gone wrong because of undefined goals or wrong objectives. For example, if you consider the Net Promoter (NPS) question, you would not survey customers for the sole purpose of knowing IF they would recommend a company or its products. Instead, you should focus on understanding the reasons WHY customers would or would not refer. Also, you want to avoid asking leading questions that influence responder opinions, otherwise, you will not obtain “voice of the customer” feedback that is authentic and useful. While these are obvious tips, some things are not so clear like how much does survey question order matter?
When asking responders the NPS question, is it better to ask “how likely they would recommend” at the beginning or at the end of a survey? Does the question order create a bias? The jury is still out. I have tried both ways and believe there are pros and cons to each scenario. For instance, if you ask the Net Promoter question at the end and the survey is too long, responders may quit, and you won’t get the quantitative results needed. On the other hand, if you ask the NPS question first, the answer may not reflect the person’s actual views.
I have been researching the answer to this question besides learning from my own survey testing. I like Zontziry (Z) Johnson’s perspective about survey placement and related advantages and disadvantages in her article. She explains:
The Argument for Placing It First
Putting the overall question first does a few things. First, according to the research, it can actually act as an anchor by which a respondent answers the rest of the questions. So, if we ask an overall question first and get that knee-jerk reaction from a respondent that might be given if asked about their experience by a friend, as they take the rest of the survey, they tend to adjust their following answers to match the answer to the overall question. This is great for the researcher if the answer to that overall question is positive; this isn’t great if that answer is negative.
The Argument for Placing It Last
According to a research article from 2003, “Order Effects in Customer Satisfaction Modelling,” from the Journal of Marketing Management, “…customers’ overall evaluations are more extreme and better explained when provided after attribute evaluations.” By asking the specifics before asking the overall question, respondents will be thinking more about how the specifics add up to their overall level of satisfaction. Arguably, this could mean they are giving a more thought-out answer to that overall question. When should you put the overall question last? When you want your audience to remember various aspects of their experience and give a more thoughtful response. You may also want to consider placing it last if your respondents will be answering awhile after the experience you’re asking them about. That way, you can remind them of the particulars about the experience before asking them to give their overall impression.
I know CX professionals who place important questions at the beginning AND at the end to capture BOTH the initial opinion and impressions they are left with after being reminded of the details. While I am not a fan of duplication in the same survey, I am a big believer in testing all scenarios and letting the data guide decisions.
What has been your experiences with survey design and the order of survey questions? Do you ask CX measurement type of questions, like NPS, Satisfaction, and Level of Effort, at the beginning of your survey, at the end or both, and why? We would love to hear your perspective and lessons learned on this topic. Contact us any time, and share on Twitter to benefit others.
*All opinions expressed on the DoingCXRight Blog and site pages are the authors’alone and do not reflect the opinions of or imply the endorsement of employers or other organizations.
