You are probably already familiar with the Net Promoter Score (NPS), a metric used to gauge the health of the customer relationship. Although it is widely used by companies, most people don’t know that it actually has three serious problems. First, the “research” behind the NPS claims is flawed. Second, the calculation of the metric (a difference score) results in an ambiguous score that is difficult to interpret. Third, the NPS is insufficient in measuring the multidimensional nature of customer loyalty.
NPS Backgrounder
In 2003, the Net Promoter Score (NPS) was formally introduced by Fred Reichheld, and, today, it is used by many of today’s top businesses to monitor and manage customer relationships. Fred Reichheld and his co-developers of the NPS say that a single survey question, “How likely are you to recommend Company Name to a friend or colleague?”, is the only loyalty metric companies need to grow their company.
NPS Methodology
The NPS is calculated from a single loyalty question, “How likely are you to recommend us to your friends/colleagues?” Based on their rating of this question using a 0 (Not at all likely) to 10 (Extremely likely) scale, customers are segmented into three groups: 1) Detractors (ratings of 0-6), 2) Passives (ratings of 7-8) and 3) Promoters (ratings of 9-10). A company can calculate its Net Promoter Score by simply subtracting the proportion of Detractors from the proportion of Promoters:
NPS = prop(Promoters) – prop(Detractors)
NPS Claims
Fred Reichheld, the co-developer of the NPS (along with Satmetrix and Bain & Company) has made very strong claims about the advantage of the NPS over other commonly used loyalty metrics. Specifically, they have said:
- The NPS is “the best predictor of growth,” (Reichheld, 2003)
- The NPS is “the single most reliable indicator of a company’s ability to grow” (Netpromoter.com, 2007)
- “Satisfaction lacks a consistently demonstrable connection to… growth” (Reichheld, 2003)
Reichheld supports these claims with research displaying the relationship of NPS to revenue growth. In a compelling graph, Reichheld (2006) illustrates that companies with higher Net Promoter Scores show better revenue growth compared to companies with lower Net Promoter Scores. However, Reichheld sites only one study conducted by Bain & Associates (co-developers of the NPS) showing the relationship between satisfaction and growth to be 0.00. [1]
NPS Problem #1: NPS is Not the Best Predictor of Growth
Independent researchers have conducted rigorous scientific research regarding the NPS and found that the original NPS claims are not substantiated. For example, Keiningham et al. (2007), using the same technique employed by Reichheld to show the relationship between NPS and growth, used survey results from the American Customer Satisfaction Index (ACSI) to create scatterplots to show the relationship between satisfaction and growth. Looking at the personal computer industry, they found that satisfaction is just as good as the NPS at predicting growth. Keiningham et al. (2007) found the same pattern of results in other industries (e.g., insurance, airline, ISP). In all cases, satisfaction and NPS were comparable in predicting growth.
Still, other researchers (Morgan & Rego, 2006) have shown that other conventional loyalty measures (e.g., overall satisfaction, likelihood to repurchase) are comparable to the NPS in predicting business performance measures like market share and cash flow. Contrary to Reichheld, other researchers, in fact, have found that customer satisfaction is consistently correlated with growth (Anderson, et al., 2004; Fornell, et al., 2006; Gruca & Rego, 2005).
Bottom line: NPS (i.e., recommend question) is not the best predictor of growth. The satisfaction question is just as good at predicting growth.
NPS Problem #2: Difference (Net) Scores are Unnecessary
Customer metrics are numerical scores or indices that summarize customer feedback results. The NPS metric is a difference score (% of Promoters – % of Detractors). But there are other, better approaches to summarizing data, including:
- Mean Scores: This is the arithmetic average of the set of responses. The mean is calculated by summing all responses and dividing by the number of responses. Possible scores can range from 0 to 10.
- Top Box Scores: The top box score represents the percentage of respondents who gave the best responses (either a 9 and 10 on a 0-10 scale). Possible percentage scores can range from 0 to 100.
- Bottom Box Scores: The bottom box score represents the percentage of respondents who gave the worst responses (0 through 6 on a 0-10 scale). Possible percentage scores can range from 0 to 100.
Why the NPS camp uses a difference score as their summary metric is beyond me. It makes absolutely no sense. If you wanted to summarize the typical height of a group of people, would you calculate a summary metric that subtracts the % of people under 5’ from the percent of people over 6’ to get a “Net Height Score?” No. That’s ridiculous. Instead, you would calculate the average height (or median) as your summary metric.
And for those of you who believe that the NPS tells you something different than the mean score, you’re wrong. In a prior study, I compared the Net Promoter Score and the Mean Recommend Score across 48 companies. I found that the correlation between the Mean Score and the NPS across the 48 brands was .97! That is, the NPS does not provide any additional insight beyond what we know from the Mean Score; both metrics are telling us essentially the same thing about how the brands are ranked relative to each other.
Additionally, because the NPS is a difference score, it lacks meaningful scale of measurement. While the original ratings of the recommend question can vary from 0 to 10, the NPS values can range from a low of -100 to a high of 100. So, what does an NPS of zero (0) indicate? Given that zero (0) falls exactly between the two ends of the NPS continuum, you would think that a zero value would indicate half of the respondents would recommend and the other half would not. But that is not the case (see Figure 1). An NPS score of 0.0 equates roughly to a Mean Score of 7.1.
Bottom line: The NPS (a difference score) obfuscates the true underlying meaning of the customer responses. Don’t use difference scores to summarize your survey data. Instead, stick with simple summary metrics like mean or top/bottom box scores.
NPS Problem #3: NPS Represents Only One Dimension of Customer Loyalty
Customer loyalty is multidimensional; it is not a single thing. There are really three different types of customer loyalty. The three types of loyalty (with specific survey questions) are:
1. Retention Loyalty is the extent to which customers remain customers and/or do not use a competitor and leads to long-term customers who are easier to service.
- How likely are you to switch to another provider?
- How likely are you to renew your service contract?
2. Advocacy Loyalty is the extent to which customers advocate your product and/or brand and leads to the acquisition of new customers.
- How likely are you to recommend us to your friends/colleagues?
- Overall, how satisfied are you with our performance?
3. Purchasing Loyalty is the extent to which customers increase their purchasing behavior and leads to increases in the average revenue per customer.
- How likely are you to purchase different solutions from us in the future?
- How likely are you to expand the use of our products throughout your company?
These three types of customer loyalty questions were predictive of different types of business growth (see Figure 2). In a nationwide study asking over 1000 customers about their current network operator, different loyalty questions were predictive of different types of objective business growth metrics:
- Retention loyalty questions were the best predictor of future churn rate
- Advocacy loyalty questions (including the recommend question) were a good predictor of new customer growth
- Purchasing loyalty questions were the best predictor of Average Revenue per User (ARPU) growth
By using only the NPS question, you are limiting your understanding of the health of your customers. Selecting the right customer loyalty questions for your survey requires careful thought about your customers and your business. Think about how your customers are able to show their loyalty toward your company and include loyalty questions that reflect those loyalty behaviors you want to manage and change.
Additionally, consider your business growth strategy and current business challenges and select loyalty questions that will help you address those challenges. For example, if you have a high churn rate, you might consider using a retention loyalty question to more effectively identify solutions to increase customer retention. Additionally, if you are interested in increasing ARPU (average revenue per customer), you might consider including a purchasing loyalty question.
Bottom Line: The NPS an insufficient loyalty measure. Customers can engage in a variety of loyalty behaviors. Think about customer loyalty more broadly and include different types of loyalty questions that meet your specific business needs.
Summary
The NPS has three major problems. First, the claim that the NPS is the best predictor of growth is flat out wrong. The preponderance of the scientific evidence shows that “overall satisfaction” and “buy again” questions are comparable to the “recommend” question in predicting new customer growth.
Second, the NPS is calculated using a difference score that, more likely, confuses people than enlightens them. Mean scores keep the metric on the same scale of measurement (0 to 10) and makes the results easier to interpret.
Third, the NPS measures only one of the three types of customer loyalty. Using a comprehensive set of loyalty questions will help you target solutions to optimize different types of customer loyalty of existing customers and, consequently, improve business growth. Including a “likelihood to quit” question and a “likelihood to buy different” question can help you understand why customer are leaving and identify ways to increase customers’ purchasing behavior, respectively.
If the NPS has these problems, why is it being used in its current form? Remember that, in the 15th century, everyone believed that the sun revolved around the earth. It made sense, after all. From people’s perspective, it appeared that the earth was the center of the universe. With heavy mathematical gymnastics, you could even explain the retrograde motion of the planets. However, based on observational evidence via a telescope, Galileo Galilei realized that, in fact, the earth revolves around the sun. The evidence-based picture of the world eventually won out. So, stop listening to dogma and follow the evidence. NPS = Not Predictive Science.
References
Anderson, E. W., Fornell, C., & Mazvancheryl, S. K. (2004). Customer satisfaction and shareholder value. Journal of Marketing, 68 (October), 72-185.
Fornell, C., Mithas, S., Morgensen, F. V., Krishan, M. S. (2006). Customer satisfaction and stock prices: High returns, low risk. Journal of Marketing, 70 (January), 1-14.
Gruca, T. S., & Rego, L. L. (2005). Customer satisfaction, cash flow, and shareholder value. Journal of Marketing, 69 (July), 115-130.
Hayes, B. E. (1997). Measuring Customer Satisfaction (2nd Ed.). Quality Press. Milwaukee, WI.
Ironson, G.H., Smith, P.C., Brannick, M.T., Gibson W.M. & Paul, K.B. (1989). Construction of a “Job in General” scale: A comparison of global, composite, and specific measures. Journal of Applied Psychology, 74, 193-200.
Keiningham, T. L., Cooil, B., Andreassen, T.W., & Aksoy, L. (2007). A longitudinal examination of net promoter and firm revenue growth. Journal of Marketing, 71 (July), 39-51.
Morgan, N.A. & Rego, L.L. (2006). The value of different customer satisfaction and loyalty metrics in predicting business performance. Marketing Science, 25(5), 426-439.
Netpromoter.com (2007). Homepage.
Reichheld, F. F. (2003). The One Number You Need to Grow. Harvard Business Review, 81 (December), 46-54.
Reichheld, F. F. (2006). The ultimate question: driving good profits and true growth. Harvard Business School Press. Boston.
[1] http://resultsbrief.bain.com/videos/0402/main.html
Article originally appeared on Appuri.
Excellent analysis! You explain in detail why NPS does not provide the complete picture of the customer experience and you provide information on the missing elements. So very well researched and presented. Your work clearly presents the argument that NPS does not give full and clear feedback to be used as a ligitimate metric. Thank you.
Betsy, I always try to convey complex ideas in simple, understandable terms. Your kind words makes me feel I hit the mark this time. Thanks!
NPS = Not Predictive Science. A great lecture on Net Promoter Score. Simple, crystal clear and convincing. Thank you, Bob Hayes.
Indeed, NPS is not the only metric to follow and reading details among detractors is often the best way to know what to improve, similarly to satisfaction question.
Thanks Bob, great article. I agree companies have gone too far in their adoption of NPS, often not understanding it’s pros and cons. Not that I am a huge advocate of it, but there is a couple of points I’d like to raise for a discussion.
1) NPS does correlate with revenue for some businesses which suggests that companies should just pick a KPI that’s right for them – NPS, CSAT, CES, netCNS all can work
2) NPS only works if followed up by the ‘why?’ question and more importantly someone actually takes the trouble to act on the verbatim feedback
Measures are important but not without a proper improvement programme driving them
Thank you for sharing this Bob. You’ve done a great job in revealing the weaknesses of NPS
Interesting arguments. My own objection is that ‘loyalty’ is a tricky noun, encompassing lots of nuance and not accessible to simplistic questioning.
Did I miss a viable alternative somewhere in the article?
Thanks for all the recent comments and questions!
Sampson, thanks for sharing the article on LinkedIn and giving the blog post the Sampson Lee bump!
Olga, Yes to number 1. Recommend, Sat and Buy again are all the same. Pick one of them and use it. My problem with the NPS camp is that they say recommend is BETTER than Sat and Buy again. It’s not. They are the same. Yes to number 2.
Marie, that is why a clear definition of whatever you’re measuring is important. That definition will guide how you operationalize (how you measure) the construct. As for loyalty, you have two measurement options: 1) self-report and 2) objective behaviors.
Bartek, I’ve included other questions you could use. Check out the other types of customer loyalty (e.g., retention, purchasing) and use those questions if they are appropriate to your situation. Also, I created a new way to measure attitudes via a survey using an open-ended questions that requires only a one-word response. Here’s the blog post in which I compare a traditional survey (rated questions) with a two-question survey (both open-ended questions): http://businessoverbroadway.com/what-must-you-ask-comparing-two-customer-survey-approaches I would love to hear this group’s comment on that post.
Insightful artcile, but maybe implicitly comparing yourself to Galileo Galieo is a bit much, no?
Other than that, I think what a lot of the studies in favor or against NPS neglect is that the industries surveyed here have very different customers with very different purchase motives and mindsets. The items presented here all seem to come from a B2B background, but B2B growth is not the same as B2C and you can’t compare purchasing your favorite yoghurt with purchasing IT solutions or life insurance – so assuming that some finding from the personal computer market can be applied to all other industries is difficult.
All in all, while I know from my background as a market researcher that word of mouth is indeed a very relevant (maybe even the most relevant) touchpoint for a number of products from various sectors, it does seem wild to limit your customer studies to just one question when you could just use, say, 6 as you propose here.