The Final Word on the Pros and Cons of NPS

1
1,068 views

Share on LinkedIn

I can’t take seeing yet another article extolling the virtues/lambasting the shortcomings of NPS. So I want to lay it all on the table – the pros and the cons – and try to offer a balanced assessment that neither canonizes nor demonizes Fred Reichheld, NPS, or its adherents or critics. Here are my ground rules.

  • Objective: move beyond the near-religious arguments between the apostles and cynics and present facts, experiences and conclusions regarding NPS
  • Approach: describe, analyze and discuss the relative merits and demerits of NPS
  • Planned outcome: a balanced presentation to provide researchers and marketers with meaningful, thoughtful context for evaluating NPS and deciding whether and when to use or not use NPS
  • Prayer: pre-empt some of the seemingly never ending back-and-forth on this issue and glib arguments pro and con

Audacious? You bet, but not any more audacious than staking claim to “the ultimate question.” No other issue in the domain of Voice of the Customer (VoC) – including both customer loyalty and customer experience – is as pervasive or divisive, so it is worth the effort. Let’s jump in with the pros and cons..

Pro: Simplicity. Gone are the days of black-box, convoluted metrics. Virtually all loyalty metrics rely on basic computations and present loyalty scores as a single number. In this sense, most metrics are “simple.” NPS is simpler in that it is single variable. Any multivariate measure is more complex than a univariate measure. Promoters sounds much better than “top 2-box” and there never is even an attempt to explain the bottom 7-box classification of Detractors. What can be simpler than answering the only question a company needs to ask and classifying people as for, against and in-between?

Pro: C-Suite Credibility & Support. In a stroke of strategic marketing genius, Reichheld and Bain sold NPS at the C-suite and board level. I have never had a client say my CEO, senior leadership or the board likes a particular loyalty metric – except for NPS, which regularly is adopted specifically because of an edict from above. This acceptance at the C-suite and board level has prompted increased budgets for all types of VoC work. It is easier to get funding for projects embraced by senior execs, and linking compensation to NPS (as some companies do) has further solidified resources for VoC projects. While NPS may have stolen the thunder from other metrics and may or may not be better for research and marketing than other approaches, NPS has proven to be a lightning rod for galvanizing the involvement and funding support of executive leadership.

Pro: Popularity. With the exception of “overall satisfaction,” there is no loyalty metric used and cited more often than NPS. There are a number of research firms with no IP of their own that focus exclusively NPS, and there is a mini-industry centered on NPS, with its own training, certification and conferences. As a by-product of this popularity, there always is NPS data against which firms can quasi-benchmark their performance. While benchmarks often are misleading, misused and over-rated, they also are inherently appealing.

Con: Predicting Customer Behavior and Business Outcomes. The validity of any metric depends on the extent to which it explains customer behavior and business results, the outcomes that matter. While every company would prefer to have happy customers (however defined) than unhappy customers, the rationale for loyalty is that it is good for business: a company captures a greater share of the lifetime value from more loyal customers than from less loyal customers. The “best” loyalty metric, as such, is that which can be empirically demonstrated to be the best predictor of loyal customer behaviors and associated business outcomes.

Here the evidence for NPS is mixed. Having conducted and read any number of analyses comparing NPS with other loyalty metrics, I can categorically state that in no study I have ever read, seen or conducted did NPS ever outperform all of the other metrics. In no instance did NPS beat GfK’s LoyaltyPlus or any of the other leading loyalty metrics; in a number of instances NPS proved a better predictor than simple satisfaction and other single variable measures, but not always. If better research and marketing means more valid measures that are more predictive of more desirable customer behaviors, then NPS gets at best a fair to middling grade on this criterion.

This isn’t surprising, as the underlying question behind NPS (likelihood to recommend) is embedded in one form or another in most of the multivariate loyalty metrics: combining willingness to recommend with any other relevant variables almost inevitably will have more explanatory power than recommend by itself. Even if NPS did capture the single most important question, combining recommend with any other relevant variables almost inevitably will have more predictive power than recommend by itself.

Talk about setting the bar high. I guess “the ultimate question” sounds more compelling than offering “another darn good question,” but NPS fails to live up to the ultimate hype. NPS holds up fairly well against overall satisfaction and other univariate metrics, such as Customer Effort, but falls far short of being distinctively more powerful, let alone attaining “ultimate” status.

Neutral: Moving the Needle. Part of the allure of NPS for senior executives, no doubt, is the promise that all you need to do is ask one question, which sounds like a great way to streamline research (and associated budgets and staff). This claim was a bit of a bait-and-switch. Even if NPS is the ultimate measurement of loyalty, any metric by itself is anemic, as it begs the question of why the score is what it is and how it can be improved. Knowing the temperature of the patient may be important, but the objective is to improve the patient’s health.

For this reason, every serious NPS program typically uses some form of key driver analysis to identify the performance criteria “driving” the metric. In reality the approach to the driver analysis may be more important than the actual loyalty metric, as the driver analysis — which identifies, quantifies and prioritizes the areas on which companies need to improve performance to boost their overall loyalty scores – is the actionable component that brings the metric to life. The “ultimate” metric without an understanding of the key drivers is far less actionable than a less-perfect metric linked to an insightful driver analysis.

Mixed: What Defines “Best?” A compass points to “magnetic north,” the North Star illuminates “celestial north.” This distinction may be immensely important for some technical applications, but both will lead north, both are “good enough” to provide direction. If the “true north” in loyalty measurement is determined by the metric that best predicts customer behaviors and business outcomes, then NPS misses the mark. But this doesn’t mean that NPS isn’t a useful metric. NPS points in the right direction, and it is positively correlated with customer profitability and other loyalty metrics. Improving NPS scores will never be bad for a company.

All thing being equal, it’s difficult to argue for “good enough” when better metrics are available. But all things aren’t equal. If the CEO prefers and the CFO will fund NPS work, if there is deep organizational commitment to and a history with NPS, judgment might dictate that continuing to use NPS is more prudent than starting new trend lines, having to educate leadership and employees, and spending resources fighting over the issue. In such circumstances, an ingrained good metric may make more sense than a better new metric.

Mixed: Industry and Business Model Matters. The appropriateness of NPS depends on the sources of customer value and varies by industry and constituency. Is value generated more by retention (ongoing purchases), cross-selling (buying other products or product lines) and upselling (buying more expensive products and services) or by recommendations to others? NPS is far less appropriate for a supermarket – characterized by frequent purchases and a wide variety of products and product lines – than for mortgage banks and real estate companies, where the lifetime value of a customer has more to do with potential referrals and recommendations than their next house purchase in eight or ten years. In industries such as these, NPS might be a better indicator of loyalty than for businesses where lifetime value is more about retention and cross-selling, such as retail banking, mobile phones or cable/Interne services.

While the value of a recommendation almost always is exceptionally difficult to monetize, recommendations are a component of the lifetime value of customers in virtually every industry. In some instances – such as businesses based on what-ever-of-the-month clubs, sales “parties,” membership models or affinity groups – recommendations are especially important to fueling growth.

There are, of course, other metrics that capture recommendations – such as Advocacy and measures of actual recommendations rather than likeliness to recommend – and recommendations are a component of most other major loyalty metrics. The more a company relies on word-of-mouth, the more important it is to have an NPS-like metric or NPS-like component in a larger loyalty metric.

Transactional vs. Relationship Measures. Loyalty is a relationship concept, and NPS, like other loyalty metrics, is a relationship measure. All too often, however, NPS is used in conjunction with surveys of various types of customer interactions where the metric simply doesn’t fit especially well. While there may be some instances where NPS might work as a measure of a customer experience or interaction, in most instances the concept of “transactional NPS” is an effort to measure the whole based on an individual part (or a component of a part). In research-speak, this is a “level of analysis” problem and involves relying on an inductive approach that is far better suited to qualitative inquiry than quantitative research.

Yes, every experience, every interaction is part of the overall customer relationship. The whole (or the relationship), however, is more than the simple sum of the parts (or experiences and interactions). Asking a customer if they are likely to recommend a brand because of a particular inquiry to a call center, a specific visit to a branch or a store or a website, or any other fleeting interaction usually is a major mistake. The proof of this erroneous approach is apparent from the key driver models and validations of transactional NPS projects: the resulting key driver models of transactional NPS almost always are far weaker than those derived from loyalty or relationship models, and NPS based on transactional surveys almost always are less predictive of customer behavior than scores based on relationships.

Weakness: The “Net” in Net Promoter. Nets are easy to calculate and appealing in simplicity, but they are statistically troubling. There are any number of combinations of Promoters, Detractors and Passives that yield the same NPS. It’s hard to treat the results as equal when comparing an NPS of 20 based on 40% Promoters, 20% Detractors and 40% Passives with an NPS of 20 based on 60% Promoters and 40% Detractors.

Because of the net calculation approach, moreover, technically the margin of error (MOE) can’t be calculated in the same manner as simple percentages, and univariate measures are inherently more likely to be unstable and have larger MOEs than multivariate metrics. Loyalty modeling, moreover, is a customer-level exercise. But NPS is an aggregate measure. An individual respondent doesn’t have an NPS score. Drivers or stated reasons can be used to explain what prompts someone to be a Promoter or a Detractor, but the modeling doesn’t explain NPS per se. While this may seem like the splitting of a technical hair, it is important for researchers and marketers to understand the nuance. Customer loyalty modeling is conducted at the customer level, while NPS is not a customer-level metric; key drivers explain the motivations behind individual loyalty, not overall NPS.

The Net-Net

So what is a serious researcher and marketer to do?

Weigh the pros and cons of NPS and other approaches with a critical eye. Carefully consider your company and business model. Keep in mind the ultimate business objective: boosting customer loyalty as a path to improving customer profitability. I always like to test multiple metrics to determine which measure best explains or predicts customer behavior and business outcomes.

If your focus is transactional, NPS almost certainly is not appropriate. If your target is the customer relationship and loyalty, NPS always will be a viable approach for consideration – it unquestionably will be “good enough” and may well be the most practical solution. Just look for the North Star and decide.

For more information, you can contact Howard Lax at [email protected]

1 COMMENT

  1. Though the effort toward NPS impartiality is appreciated, along with a recognition that NPS universality is based on good marketing, as well as corporate researcher tendencies toward complacency and risk aversion, let me offer my own perspective on customer advocacy measurement. Your observations about advocacy – “There are, of course, other metrics that capture recommendations – such as Advocacy and measures of actual recommendations rather than likeliness to recommend – and recommendations are a component of most other major loyalty metrics. The more a company relies on word-of-mouth, the more important it is to have an NPS-like metric or NPS-like component in a larger loyalty metric.” – are almost, but not quite, accurate.

    In the customer advocacy framework you’ve experienced me incorporating in b2b and b2c client studies when we worked together, recommendation was never an included element on results reported to a client. In fact, in using paired variations of framework application for testing purposes, i.e. one with and one without recommendation, the variation with recommendation always produced less actionable results. It was level and frequency of positive and negative word of mouth that proved to be more indicative of actual downstream customer behavior. For additional insight, please refer to Tables 6 and 9 in http://www.harrisinteractive.com/vault/Harris-Interactive-Poll-Research-Word-of-Mouth-2009-06.pdf

    Further, in multiple studies where we did incorporate independent NPS scoring (both when you and I worked together and with my later employer), a respondent had to give a score of 10 to match up with positive advocacy and granular, swing voter (DFA) results. In my experience, any score of 9 or 8 was sufficient to render the recommendation score virtually useless.

    Although some of the analytical results are proprietary, rest assured that ample proof of this can be provided. This, I hope you’ll agree, is meaningful, thoughtful context.

LEAVE A REPLY

Please enter your comment!
Please enter your name here