Find the “Ultimate” Loyalty Metric to Grow Your Business

15
8,357 views

Share on LinkedIn

Managing a business is like flying an airplane. You need a good pilot and equipment to take off and keep it flying to the right destination. During flight, the pilot has a dashboard of instruments to indicate speed, height, direction, rate of ascent/descent, fuel, engine status and so on.

In business, the equivalent of that cockpit is the Balanced Scorecard, developed in the late 1990s by Robert Kaplan (Harvard Business School) and David Norton. The idea was to provide a balanced measurement of company performance using a combination of financial, customer, process and employee perspectives.

Measuring customer satisfaction and loyalty is a bit more complicated than counting revenue, however. In fact, some experts I interviewed for this article said that one of the problems with loyalty research is that it’s too much like “quantum physics.” Actually, I think quantum physics is easier.

Relationship Health Indicators

Loyalty gurus generally agree that customer loyalty is more than just repeat-buying behavior. Michael Lowenstein of Harris Interactive, for example, says that loyalty is a complex combination of a “state of mind” and a “set of behaviors.” OK, so how are we going to measure that?



In 2006, Fred Reichheld’s book, The Ultimate Question (Harvard Business School Press, March 2006), proposed a simple method to measure loyalty, called the Net Promoter® Score. (Note: Net Promoter is a registered trademark of Satmetrix, Bain and Reichheld). Based on responses on a zero- to 10-point scale, customers are grouped into promoters (9 or 10), passives (7 or 8 ) and detractors (0 to 6). Subtract the percentage of detractors from promoters, and, voilà, you’ve got a Net Promoter Score (NPS).

With that, Reichheld asserts, you can get rid of those long surveys and expensive loyalty researchers. Just focus on improving your NPS, and your company will grow. Prominent brands like Amex, GE and Intuit have embraced the method, which has created a bandwagon effect the likes of which we haven’t seen since TQM, BPR and CRM.

The trouble is, I’ve found no solid evidence that NPS is the single best customer feedback metric for all companies. Or, that it’s even a valid metric in all situations. But I suppose a book entitled, One Pretty Good Question That Might Help You Grow, wouldn’t sell too many copies.

Simple, or Simplistic?

Make everything as simple as possible, but not simpler.
—Albert Einstein

A growing cast of NPS critics says Reichheld has gone too far, dumbing down customer loyalty into an overly simplistic measurement. Jim Barnes of BMAI Strategy reviewed 15 loyalty models used by major corporations and found three groups of factors: functional, experiential and emotional. “Reducing loyalty to a single number is laughable,” he said.

Recent independent research suggests that, at best, NPS is not the “best” metric and can’t be applied in all industries. At worst, it’s just plain wrong and will lead managers astray. While Reichheld points out that his critics are protecting their business interests, it’s also fair to say that the hype surrounding NPS has been good for sales of Reichheld’s books and keynote speeches, and probably helps Satmetrix sell more customer feedback solutions, too.

Timothy Keiningham of IPSOS Loyalty seems to be leading the charge to put Reichheld’s claims up for public inspection. In particular, Keiningham and his academic colleagues attempted to replicate Reichheld’s comparison of NPS and ACSI (American Customer Satisfaction Index), where he claimed that there was no correlation between ACSI and growth. The new study found that the ACSI and NPS offer similar predictive capability using Reichheld’s own data. Keiningham says this finding suggests bias in Reichheld’s research, attempting to discredit ACSI, a competing loyalty metric.

Laura Brooks of Satmetrix, who conducted the original research with Reichheld in the 2000-2001 time frame, flatly denies any bias. Brooks claims that Satmetrix “has tangible evidence in enterprises we’ve worked with.” Maybe so, but I’d suggest this is like a software vendor saying it has a few successful customers, and then proclaiming all other customers will receive the same benefits. That’s called marketing, not research.

Larry Freed, president and CEO of ForeSee Results, which supports the ACSI, found in an online retail study of 8,400 consumers that NPS doesn’t cause growth but, rather, is “spuriously” correlated to growth, while the real growth “driver” was good old-fashioned customer satisfaction. He also says, as other researchers have told me, that NPS has statistical shortcomings. Why go to the trouble of asking people for a zero-to-10 rating, then “throw away” information by categorizing into three buckets? A simple mean (average) score would be more reliable (less statistical error). But perhaps not as memorable as “net promoter.”

Respected academics, who shouldn’t have any ax to grind in this debate, find little support for NPS in independent research. In the paper, The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Business Performance, Neil Morgan, a professor at Indiana University, and Lop Leotte do Rego, a professor at the University of Iowa, found that “metrics based on recommendation intentions (net promoters) and behaviors (average number of recommendations) have little or no predictive value.” The most effective measurement? Average satisfaction scores.

For his part, Reichheld won’t respond to his critics directly. And he declined to answer my questions by phone or email. (For the record, CustomerThink has no vested interest in any specific loyalty methodology.) His stock response is that “the value of NPS will not be determined in academic debating circles but in the real world of customers and employees.” To be fair, in July 2006 he noted in a blog that NPS “does not provide proof of a causal connection between NPS and growth” and that some of the original collected data was “imperfect.” You think?

So it’s a standoff. The NPS camp (Bain, Reichheld and Satmetrix) has a simple metric with proprietary data they say proves it works. Academics and loyalty researchers often make their data public, at least for peer review, but keep their loyalty models locked up in their own black boxes.

Managing Loyalty in the Real World

Yes, NPS can work for individual companies. Intuit, one of the most customer-centric companies around, is a Net Promoter advocate. To understand why customers are promoters versus detractors, Intuit uses a text analytics solution from Clarabridge to mine write-in comments from TurboTax users. Now you’ve got the best of both worlds: a very short survey with diagnostic capability. (See Use Text Analytics To Listen to Customers on Their Terms.)



FileNet, a content management vendor acquired by IBM in 2006, has more than five years of experience with customer loyalty programs, under the direction of Suhail Khan, head of Worldwide Customer Loyalty. FileNet used to calculate a “loyalty index” but switched to NPS a couple of years ago and found it to be “easier to digest” for the organization and a “great tool to drive change.” What’s more, Khan says FileNet can link NPS improvements to revenue growth. Technology provider: Satmetrix.

But some companies, initially drawn to the simplicity of NPS, are changing it to suit their unique business needs. In short, they don’t use the “just one question” methodology that Reichheld recommends.

Model Metrics, a salesforce.com consulting partner based in Chicago, found inspiration in the NPS concept but adapted the methodology. The firm’s founder, Adam Kaplan, and principal consultant, Kevin Turner, say they also ask the customer to rate the quality of service and likelihood of future business. Three questions along with comments keep the survey short and response rates high, while still providing enough information to take action. (See Turner’s CustomerThink article, Does the Ultimate Question Work for Loyalty? My Experience Says Yes.)

Thermo Fisher Scientific spent several years refining its loyalty management approach. Tricia Rakiey, Global Customer Allegiance leader, says that the company was initially drawn to the NPS concept but, like Model Metrics, wanted more questions to add diagnostic abilities. Surveys can run 20 to 30 questions long, with critical questions covering overall customer satisfaction, ease of doing business, likelihood to repurchase and likelihood to recommend. While the jury is still out in terms of statistical proof, Rakiey says that the company’s composite Customer Allegiance Score has helped to rally people to work on customer issues. Technology provider: CustomerSat.

And, of course, many companies don’t rely on NPS at all. A recent CustomerThink member survey found that just 18 percent have implemented NPS. Jill Griffin of The Griffin Group says that about one-third of her clients have expressed some interest in NPS. Still, the most popular implemented metrics for CustomerThink members are overall customer satisfaction (69 percent), likelihood to recommend (56 percent) and likelihood to buy again (43 percent).

Egg, a European provider of digital banking and financial services, doesn’t use NPS, and hasn’t found the underlying likelihood-to-recommend question to be very useful, either. According to John Jennick, head of Customer Experience and Action, Egg has used a “test and learn” approach to find the best questions.

While willingness to recommend is part of the mix, it’s not the “one question” that drives Egg’s growth. In fact, Egg found that asking if the interaction with Egg “made me feel my time was well spent” provided results that helped managers make better decisions on corrective action needed. Technology provider: Confirmit.

NPS Bubble Will Burst

Let’s tip our hat to Reichheld for making customer loyalty a board room discussion. NPS popularity has also been an overdue wake-up call to loyalty researchers that they should be more practical. Especially for smaller businesses that can’t afford their services.

NPS popularity has also been an overdue wake-up call to loyalty researchers that they should be more practical.

But I’m dismayed at the finger-pointing I’ve seen in the loyalty research industry. It’s unprofessional, reminding me of software executive flame wars. I don’t know whom to trust. Certainly not Reichheld, anymore.

Furthermore, I’m very disappointed that Reichheld has not provided any third-party supported research that NPS truly is “the one number you need to grow.” Cases studies and stories aren’t proof. The fact that GE likes NPS doesn’t mean it will work for everyone, or anyone, else. Remember Six Sigma?

Also keep in mind that revenue growth isn’t the only important business objective. Improving gross margins, net operating cash flow, shareholder return and market share were also ranked high in our recent CustomerThink survey. Moreover, Jeff Marr of Walker Information points out that in some business models, the key to growth may hinge on prospects, not necessarily existing customers.

I predict that NPS hype will peak in the next year, then decline as more business leaders realize it’s not the quick fix they thought it was. Then it’s back to the book store to find the next fad diet.

Recommendations



In the meantime, what can you do? For starters, have an open mind. Unless you want to bet your career on one magic number, invest some time to understand both sides of the debate.

  1. Remember that any loyalty metric doesn’t cause growth; it’s just an indicator. Make sure it’s one of the key “dials” on your business management dashboard, in balance with other key business measurements. Odds are, revenue growth isn’t the only thing that matters.
  2. Keep surveys as short as possible but not too short. Depending on the situation, long surveys can work quite well. But for transactional surveys, a few questions (five to 10) is the limit. Make sure they include the right questions—those that link to improved business performance and provide adequate diagnostic capability on underlying loyalty drivers.
  3. If you decide to rely solely on NPS, validate it with common sense and some history. Not so sure? Then hedge your bets by also asking questions about overall satisfaction and likelihood to repurchase. Randy Brandt of Maritz Research says the right questions will “depend on market conditions, rules of competition and buying patterns.”
  4. Brand your metric and make it your own. Develop an internal marketing campaign to explain what the metric means (no statistical arguments, please) and how it helps the organization succeed.
  5. Reward managers and employees for improving customer loyalty, but don’t overdo it. Management judgment has not gone out of style. If you make the rewards or penalties too strong, you’ll stimulate unwanted “gaming” behavior. Remember, counsels Jeanne Bliss of CustomerBLISS, the point is to drive change within the organization to “get people to work better together” serving customers.
  6. Plan to evolve and refine your measurements and rewards. You’ll need to make adjustments over several years to keep it working just right.

Which brings us back to the key question: Is there one “ultimate question” that will effectively measure customer loyalty for all businesses? The short answer: No.

But the “ultimate answer” is that you must figure out the right metrics for your business. What’s at stake is your customers’ loyalty—and your future success. Do you have anything better to do with your time?

15 COMMENTS

  1. Bob –

    Your excellent article was both timely and even-handed on the subject of Net Promoter. While I support the professional and academic research that finds little or no correlation between NPS scores and customer behavior (recommendation being more of an outcome than the behavior itself), I think that understanding loyalty behavior is a lot simpler than many would believe. In my latest CustomerThink article, I’ve identified desired customer behavior as commitment, a combination of emotional and functional motivation. We define advocacy, moreover, as the active expression of commitment, because it takes into account informal peer-to-peer communication, and other related customer behaviors.

    Michael Lowenstein, PhD CMC
    Vice President and Senior Consultant
    Harris Interactive Loyalty

  2. Bob, I am sure that Reichheld will not be sending you a Christmas card this year, but I thought that your article was fair and thought provoking.

    Your article was that much more meaningful to me since I happen to be involved with designing enhancements to our Feedback Management module to include surveying, which will include elements from NPS.

    NPS certainly should not be the only measurement of consumer loyalty, but it does include elements that are relevant and leveragable in any assessment of consumer satisfaction.

    Whatever a company elects to use to measure loyalty, I believe it must be:

    • Easy and fast for the consumer to provide input.
    • Extended at every touch point.
    • Associated directly with particular activity and staff whenever possible.
    • Facilitates immediate action on individual feedback as appropriate.
    • Is reviewed in aggregate over time and results in real change.

    Make it simple, measure it often and do some good with it.

  3. The good thing about NPS is that it’s simple and easy to understand.

    And it’s based on a fundamental willingness to recommend (WTR) question that is one of the three key questions that has come up over and over again as I’ve interviewed loyalty researchers and read papers on the subject. (The other two: overall satisfaction, and willingness to buy again.)

    The danger is assuming that WTR (or NPS, which is just a derivative), or any one customer feedback measurement, is the only one worth asking. Researchers have not been able to prove that WTR is a good predictor of business performance for all industries. Even the conventional wisdom that it’s better for B2C, has been disputed by some academics.

    In B2B, where the person surveyed may not be the person who signs the PO, or the executive sponsor, WTR may be simply irrelevant.

    So, I’d say that NPS may be relevant in some assessments of customer loyalty, but shouldn’t be blindly applied.

    To your closing comment: “Make it simple, measure it often and do some good with it” I would add: make sure it’s meaningful, and that you survey the right people.

    Bob Thompson, CustomerThink Corp.
    Blog: Unconventional Wisdom

  4. Thanks Bob nice article.

    Picking up on your aeroplane analogy in the first paragraph. If I was flying an aircraft which single indicator is most important? Is the fuel indicator more important than the landing gear indicator? If there was just one instrument that pulled everything together we could all fly planes – or run large successful companies!

    Malcolm Wicks
    Three Step Consulting

  5. NPS is a logical and objective measure of what a particular group of people feel about your business/products in an overall sense. And as it’s framed in terms of likely future purchasing action, it clearly provides intelligence of commercial importance. Finally, based on a concept that’s both easy to grasp and implement, it’s hardly surprising the methodology has taken off. Being a specifically overarching indicator of a particular group’s feelings, NPS doesn’t also, of course, dig more deeply into the reasons why. It’s thus a complementary research tool – and a good one. Why all the fuss?

  6. Bob,

    Isn’t the ultimate goal of measuring customer satisfaction and loyalty to improve the customer experience? I think so.

    The NPS score cannot be the ONLY number you need to grow because it fails a litmus test for a complete customer satisfaction measurement system: is it actionable? Not at all.

    Operators need more than an “outcome” score like NPS. They need customer feedback on specific elements of the customer experience itself so they can fix specific problems and make the right improvements.

    Jack Mackey
    Vice President
    Service Management Group (SMG)

  7. Bob,

    I would use NPS as an “early warning” metric, one that is akin to these: The manager of a steel mill will use tonnage sold per week as a rough estimate of whether the business is on track to make their quarter’s “goal.” A movie theater owner will look at the number of tickets sold per night to see if the business is on its sales target. In other words, they are early warnings that apprise the manager of any issues that may need attention.

    (Generally, the standard they are measuring against takes into account current economic conditions, new internal market initiatives, those started by or distracting their customers, or product popularity, among other things.)

    Each one has a goal – “sales per period of time”, a question – “How do I know I’m on target?”, and a metric – “units sold per time period.”

    NPS is a (slightly?) more sophisticated measure like these: It is a measure that gives management an “early warning” about the goal, be it loyalty or growth, they are trying to attain, but, just like the measures above, management has to dig deeper (ask lots more questions) to find out what factors are really affecting the value of the metric, as you and others have said.

    The main problem I have with a metric like this is that it can lull the business into not looking for things to do better. If the business keeps getting a score that meets their standard then things are fine? I doubt it.

    Gotta look for problems and advantages even when things are going well.

    Jonathan Narducci

    CornerStone Cubed
    Building Customer Powered Value

  8. Good analogy, Jonathan! Ideally a loyalty metric should be a leading indicator. Obviously metrics like sales revenue happen after the customer’s action. Waiting for revenue to fall just means it will take longer to turn things around.

    If a company does “experience” survey immediately after interactions with customers, they can take action to address issues with unhappy customers, before they have time to fester. I think this is one of the most powerful features of Enterprise Feedback Management (EFM) systems.

    Unhappy customers are more likely to leave, and even if they don’t, they aren’t shy about spreading the bad news.

    For less frequent “relationship” surveys, the same is true. If a customer has endured a pattern of poor support or other problems over a period of time, a low score should stimulate some action.

    Unfortunately, the same canary doesn’t work in all coal mines. The only thing worse than no canary, is one that doesn’t sing at the right time. So, a company still has to invest some time to find the right metric.

    Bob Thompson, CustomerThink Corp.
    Blog: Unconventional Wisdom

  9. John, I couldn’t agree with you more when you ask why all the fuss. As Bob pointed out, NPS is a simple and easy to understand tool but it shouldn’t be blindly applied. He’s absolutely right. NPS is an excellent tool for creating a customer-centric culture across a company but it’s only as good as the organizational discipline put in place to take action.

    I echo Chuck’s comments about covering all the touch points – you need to take a holistic approach by getting the people, process and technology properly aligned to make the right operational changes. But it doesn’t end there – you must act upon the data and get your employees engaged to create customer loyalty. It’s what we at Satmetrix call our loyalty discipline – see http://www.satmetrix.com/company/discipline.htm.

    Deborah Eastman
    Chief Marketing Offficer
    Satmetrix

  10. Deborah

    The Net Promoter Score is not a simple and easy to understand tool. It would be if it were truly a reliable predictor of customer behaviour. But as recent research has shown, it is not. Without a much better understanding of what drives NPS and whether NPS drives business results, it is just another simplistic and misleading metric. Just like customer satisfaction was when it was introduced with a fanfare all those years ago. And service quality before that.

    I note in writing this comment that Satmetrix is a co-developer of NPS along with Fred Reicheld.

    Graham Hill
    Independent CRM Consultant
    Interim CRM Manager

  11. Surely the allure and simplicity of asking only a single question or two to determine customer satisfaction and/or loyalty is appealing. On the surface it sounds so inviting. But in practice, the concept of The Ultimate Question has a major utility flaw, and that is a lack of actionable information that clients can actually use to improve their business.

    To paraphrase H.L. Mencken…

    “For every complex problem there is an answer that is simple, neat, and wrong.”

    NPS has proven useful as an aggressive scoring method that we at Mindshare use in specific situations to help differentiate tightly grouped units – to spread out the distribution. But the morale problems caused by it significantly dimish its usefullness in many corporate cultures. (Employees who have grown up since grade school thinking that 90 is an “A,” 80 is a “B,” etc. are not all that excited to see their performance metrics in the 30’s and 40’s.)

    Further, the concept of an “Ultimate Question” is simply not supportable as a tool of improvement. It might be useful to some as a historical scorecard, perhaps. But I’ve always thought the purpose of customer feedback is not simply measurement of history but diagnosis of areas of improvement. Imagine telling your doctor you feel ill, having him ask you “Why?” and then, after you answer, that’s the end of the conversation!

    I was such a fan of “The Loyalty Effect.” I wish I could say the same of “The Ultimate Question.” Common sense tells us that a single question methodology cannot possibly be robust enough to provide the actionable information required to drive improvement.

    Richard D. Hanks
    President, Mindshare Technologies

  12. Over the past few months, I’ve found myself in active dialogue with multiple NPS-using research executives who had believed, and so assumed, that they could learn all they needed to know about customer behavior by applying NPS and a few follow-up open-ended questions. Meaning: No insights about the rational or emotional drivers of perceived customer value, nothing about complaints, nothing about performance and benefit change over time, no statements of excellent and poor transactional experiences, etc. They have become disillusioned because, as noted by Graham Hill and Richard Hanks, and legions of other customer researchers, there is very little actionability in NPS which can lead to improving the customer experience or driving customer marketplace behavior. Now, these research execs are having to apply totally new data generation architecture and engineering to obtain value-driving information which will yield demonstrated, monetizing results.

    Michael Lowenstein, PhD CMC
    Vice President and Senior Consultant
    Harris Interactive Loyalty

  13. In the time that I’ve been working with NPS, I’ve not encountered any business leader that assumed it to be a silver bullet but rather a good indicator tool in the research mix targeted to understanding the longevity and value of customers. While some of the NPS creators from Bain and Satmetrix may tout it as the only metric necessary (for obvious reasons), the writers on this blog seem to be collectively ganging up on a potential threat to budgeting for marketing consultants.

    I see little mention of net promoter scores typically being supported by verbatims that explain the respondents’ rating. That’s like having an in-depth interview with hundreds of your customers. That’s can be a lot more valuable than a customer satisfaction survey no matter what jargon you use to make it sound like an innovative methodology.

    In terms of customer research, this blog lacks any serious insight. Your “Hot Discussions” include “How Well Do You Listen?” which espouses listening to what is said and not said by your customers. Conventional intuition at best. Your own Starbuck’s survey rates the importance of more than 25 key attributes related to the customer experience but there is no mention of the caloric or fat content of some Starbucks coffee/coffee type drinks. That has been the single biggest hot-button issue related to Starbucks in recent news. The omission would make me question the depth of understanding research design and/or topical consumer issues. You may want to ask a marketing research professional, who should be unaffiliated with a potentially self-serving research supplier, to participate in some of these discussions for the sake of credibility.

  14. D. Oakes

    The discussion on the blog is the view of many readers, not a self-serving rant against NPS as you suggest. Some people, like you, have had good experiences with NPS, but more people have not had such good experiences.

    As for the rest of CustomerThink’s many hundreds of posts, they are also the view of many readers. All of us are smarter than any of us, as the saying goes.

    Graham Hill
    Independent CRM Consultant
    Interim CRM Manager

    Gunning Fog Index: 9

LEAVE A REPLY

Please enter your comment!
Please enter your name here