I first heard of the Net Promoter Score from Fred Reichheld, one of its co-developers, in a talk he gave at a vendor conference. When he stated that “the NPS is the best predictor of business growth,” my interest was piqued. Why? I have never found evidence in my 20+ years of experience to support that statement. Since his talk and the release of his book on the same topic, I have conducted many studies to examine the merits of the NPS claims. Additionally, other researchers, from both industry and academia, have conducted research on the NPS. Our basic conclusions: the NPS claims are not true and there are a lot of problems with their research claims.
My research addressed several issues surrounding the NPS claims, not just the original claim. Based on mine and others’ research, I present my list of 5 NPS myths. Additionally, I present evidence that debunks each myth as well as what you can do to improve how you measure customer loyalty. You’ll see that the NPS is not all that it’s cracked up to be.
5 NPS Myths
Myth 1: NPS is the best predictor of growth
Fact: The NPS is not the best predictor of growth. Keiningham et al. (2007), looking at the personal computer industry, found that satisfaction is just as good as the NPS at predicting growth. They also found the same pattern of results in other industries (e.g., insurance, airline, ISP). In all cases, satisfaction and NPS were comparable in predicting growth.
What you can do: Use other loyalty questions. The use of the other two common loyalty questions, “Overall sat” and “Buy again,” result in the same conclusions as when you use the NPS.
Myth 2: Net scores are easy to understand
Fact: Net scores are ambiguous; You can arrive at the same NPS score using different Promoter and Detractor scores. Net score creates a metric on an entirely new scale (from -100 to 100) that is different from the original customer rating (0 to 10). Additionally, CX professionals exhibit bias when interpreting summary metrics. Also, the segments used to calculate the NPS are arbitrary. Specifically, while the NPS formally defines Promoters as customers who gave ratings of either 9 or 10, we found that Promoters could also include respondents who gave a rating of 8, 9 or 10.
What you can do: Use mean or top-box scores as summary metrics instead of net scores. The use of the mean keeps the group metric on the same scale of measurement as the original ratings (0 to 10). Train your employee on statistics and statistical thinking (long-term).
Myth 3: The NPS is the only loyalty question you need in your survey
Fact: Customer loyalty is multidimensional; it is not a single thing and you simply can’t measure “loyalty” with a single question. There are really three different types of customer loyalty. The three types of loyalty are: Retention, Advocacy and Purchasing. In a nationwide study asking over 1000 customers about their current network operator, different loyalty questions were predictive of different types of objective business growth metrics:
- Retention loyalty questions were the best predictor of future churn rate
- Advocacy loyalty questions (including the recommend question) were a good predictor of new customer growth
- Purchasing loyalty questions were the best predictor of Average Revenue per User (ARPU) growth
What you can do: Measure all types of customer loyalty that are relevant to your business. Consider the different ways your customers can contribute to the bottom line and ask about those types of loyalty behaviors. Here is a more comprehensive list of customer loyalty questions you can use in your next customer survey.
Myth 4: High correlation between NPS and customer experience (CX) illustrates importance of CX in driving recommendations
Fact: The correlation between NPS and CX is artificially inflated because both are measured using the same method (a survey asking for self-reports; both use similar rating scales). The true correlation between CX satisfaction and customer loyalty is much lower than what is indicated when self-reported metrics are used to measure both variables. As your loyalty metric becomes more objective (asking about number of people to whom you recommended, for example), the correlation between “recommendations” and CX decreases significantly.
What you can do: Use objective loyalty measures in your analysis. Integrate your disparate data silos (e.g., CX system, CRM, Mixpanel) to link objective loyalty metrics (e.g., number products purchased, amount spent) with CX metrics (satisfaction with product, service). When you analyze these data, the results will be a better reflection of reality.
Myth 5: The claim that “NPS is the best predictor of growth” is widely accepted by CX professionals
Fact: In a survey of CX professionals, Only 25% of CX professionals said they believe the claim that the NPS is the best predictor of growth. This finding is more remarkable for CX professionals from loyalty leading companies. Of CX professionals from loyalty leading companies, only 14% agree with the NPS claim. Of CX professionals from loyalty lagging companies, 42% agree.
What you can do: Share your concerns about the NPS (along with the supporting research) with your CEO and CMO. Sharing the alternatives of the NPS with them to help move your loyalty measurement program forward.
Summary
This post summarizes scientific evidence that debunks the popular beliefs about the NPS. I presented 5 popular beliefs about the NPS and show why those beliefs are not supported by evidence. The evidence shows that the NPS: 1) is not the best predictor of business growth, 2) is difficult to understand, 3) is insufficient in capturing all the components of customer loyalty, 4) inflates the importance of CX on loyalty and 5) is not widely accepted by most CX professionals. These myths are presented in Table 1.
Your job may require you to use the NPS. Your livelihood might depend on selling the NPS. If you want to keep using, supporting, loving the NPS, go ahead. I really don’t care what loyalty metric you choose to use. What I do care about is sharing evidence-based conclusions and helping companies optimize the use of their data. The quality of your customer analytics is only as good as the metrics you analyze. In this post, I shared the empirical evidence regarding the myths of the NPS and what you can do to overcome them.
References
Hayes, B. E. (2008). Net promoter score debate: The measurement and meaning of customer loyalty. Business Over Broadway.
Hayes, B. E. (2008). Customer feedback programs best practices: An empirical investigation. Business Over Broadway.
Keiningham, T. L., Cooil, B., Andreassen, T.W., & Aksoy, L. (2007). A longitudinal examination of net promoter and firm revenue growth. Journal of Marketing, 71 (July), 39-51.
Morgan, N.A. & Rego, L.L. (2006). The value of different customer satisfaction and loyalty metrics in predicting business performance. Marketing Science, 25(5), 426-439.
Netpromoter.com (2007). Homepage.
Reichheld, F. F. (2003). The One Number You Need to Grow. Harvard Business Review, 81 (December), 46-54.
Reichheld, F. F. (2006). The ultimate question: driving good profits and true growth. Harvard Business School Press. Boston.
Excellent, comprehensive, contemporary, real-world. Thank you!!
You’re welcome, Michael! Thanks for the kind words.
if i could stand up and cheer I would. Well done sir!
An example of myopic, slavish corporate dependence on NPS as KPI: http://customerthink.com/comcasts-nps-gamble-can-the-metric-help-fix-the-customer-experience-culture/
Great article and very helpful: reflects our experience as advisors in this area.
Great article Bob,
Happy someone is finally brave enough to burst the NPS bubble many corporations have been blindly following.
As you also mention, whatever metrics you use, it’s the analysis and recommendations made, and actions taken to address any issues highlighted, that truly matter. So many companies continue to collect data but do nothing with it.
Dave, Guy, and Denyse, thanks for the compliments! I’m just trying to inject some science and critical thinking into the conversation.
Great post Bob! Sound arguments. Needed to be said again. I don’t mind using the NPS question but I don’t think it’s better than others. In fact, other questions make more sense in certain contexts. And besides, as it’s been said many times, it’s not about the number: It’s about what you do with it. To me the biggest fallacy is the claim that NPS is the only number needed. No single customer sentiment metric, whether NPS, satisfaction or other alone is sufficient. Customer feedback must be captured from multiple other sources to complete the picture and really make continuous improvement or transformation.
Thanks for this work.
Isn’t it amazing how a high level of self promotion can turn a marketing campaign into an industry standard? Shame on all of us for believing without first confirming.
Glad to see Sam’s response to this important post. Having been a dedicated naysayer about the institutionalization of NPS since the concept first surfaced (devoted an entire chapter to this in my 2012 book, The Customer Advocate and The Customer Saboteur), I take no particular pride for my perspectives – – but do feel shame and disappointment for the myopia and laziness that I’ve too often observed from market research practitioners and the many organizations relying on NPS as a universal, even sole, KPI.
For example, NPS believes that its ‘annexation’ of advocacy behavior, as a surrogate for recommendation, is justified. In other words, from their point of view recommendation = advocacy. It isn’t and it doesn’t: http://customerthink.com/is-there-a-single-most-actionable-contemporary-and-real-world-metric-for-managing-optimizing-and-leveraging-customer-experience-and-behavior/
By the way, my friend Olaf Hermans wrote a LinkedIn post recommending asking about Goo0dwill instead of intent to recommend. Read it here: https://www.linkedin.com/pulse/only-one-question-you-want-ask-your-customer-goodwill-olaf-hermans?trk=prof-post
Let me chime in as a consumer who reads these columns because he’s interested in seeing what the “enemy” is thinking. (My goal is to get the best value for the money I spend and to understand how others are trying to manipulate me.) I also love to take surveys and hate when I get to would-you-recommend questions. I’m philosophically opposed to recommending almost anything to my family, friends, and colleagues. I may have had a good experience, but I know that I can’t guarantee the same for them.
To give a concrete example, after spending decades without a financial analyst, I finally found one that I like because he thinks outside the box and goes beyond investment platitudes. He is unfortunately employed by a company that I don’t like at all. I doubt that many other analysts in the company are like him. I’m certainly not going to say that I’ll recommend the company.
I also have a strange view on life. I prefer to live frugally and would rather drive a Honda Accent rather than a Mercedes because all I need to do is to get from point A to point B and would worry about theft or damage to the expensive vehicle. I also don’t like luxury hotels because I consider their special customer perks to be intrusive and much prefer to be left alone in an adequate room in a cheaper hotel. I know that most other people don’t think like me so that I don’t believe it worthwhile for me to recommend what I like because the average person wouldn’t like it.
While Bob Holley’s story is unique, the part that refers to the NPS question is not surprising. I do mostly telephone surveys and in one of my first surveys I was recording very high CSAT scores and then a 0 for the NPS question. After I comkpleted the survey I circled back and ask the participant why he was unlikely to recommend the company when he appeared to be very satisfied with their performance. He said “I work for the government and am not permitted to recommend any vendor”. The light bulb went on and I now ask:
“Assuming you are permitted, how likely….”
Good survey design includes a choice of not making a choice for each question. Choices like No Opinion, No Answer, Pass. THe NPS question does not include that option so the only choice is 0 or a very low score.
Bob and Sam,
Thanks for your comments. I guess what I glean from your words is that you need to be cognizant of practical matters (not just research standards) that will impact your interpretation of ratings as well as generalization of your findings.
Bob,
As Tip O’Neil said – “All politics is local”. I guess I am saying that all opinions are individual and we should design our survey instruments to allow individuals with different situations and biases to complete the survey and tell us what they mean, not what come closest to what they mean.