Don’t Waste Your Time On External Net Promoter Benchmarks

4
401

Share on LinkedIn

It comes in many flavours but easily the most common question on Net Promoter that I get asked is: how does my score compare to my peers?

Unfortunately it is also the question with the answer that people want to hear the least: there are very few ways to really know.

While there are some reasonably comprehensive multi-industry NPS research reports unless your company is actually included in the study they are generally worthless to give you a benchmark against which to measure your internally derived score.

The reason is quite simple. NPS can vary dramatically depending on a variety of factors and unless your survey process is the same as the benchmark process you will have issues.

Don’t worry I’ll give you two good resolutions to this problem but first why are those published data of so little assistance?

Factors That Change NPS

Industry Sector

As can be clearly seen by the chart below the NPS can vary dramatically between industries. With average scores for whole industries ranging between 10 and 60, if your industry is not specifically on the chart your chance of getting a viable benchmark are low.

nps-by-sector

Country

If you are in an industry that is on the chart you might be thinking great there’s a benchmark that I can use. You’d be wrong.

NPS also varies by country and culture so even if your industry is in the benchmark set, if you are not in the same country as in the report then the benchmark is not of any use to us.

Don’t believe it, check out the image below for benchmarks for the same industry in different countries that we put together a couple of years ago.

nps-australia-vs-europe


Survey Administration

Terrific you are on the industry list and it shows numbers for your country – finally you’ve got your benchmark!

Probably not. Now the issue is that the survey administration process (telephone, internet, face to face) has a big impact on the score.

Just recently new research has been released on the issue of comparing scores that have different methodologies.

In their study Sam Klaidman and Frederick C. Van Bennekom identified that simply running the survey by phone gave one company a 40 point (58 to 18) change in the outcome for the same customer set.

The implications of this are enormous. If you don’t run your survey the same way as the benchmark then the comparison is almost meaningless.

NPS Varies Depending on How You Construct Your Questionnaire

Even if you use the same format (email, telephone, etc) the order of the questions can impact on your score. If you ask the “would recommend” question early in your survey the score will be higher than if you ask it later in the survey.

There Are More

Actually there are even more elements that can impact on the score but I think you get the picture here.

Solutions to the Net Promoter Benchmark Problem

So where to from here?

There are some real reasons why it would be good to know how your score stacks up relative to your competitors. The most important is that it has been found that your score relative to your peers is indicative of your relative revenue growth performance.

The further ahead of them you are the better your growth. So getting some type of benchmark is useful. The problem is knowing that, except in very special circumstances, published benchmarks are not very useful how can you proceed?

You have only two real choices.

1. Ignore External Benchmarks and Benchmark Internally to Your Own NPS

I tell customers that the most important Net Promoter benchmark that they have is their own score last month, last quarter and last year.

The whole purpose of this approach is culture change and continuous improvement.

So with that in mind, do you really care what other companies are doing?

If you’re ahead of them are you going to stop driving change? No.

If you’re behind them are you going to stop driving change? No.

So what does it really matter where your competitors are sitting?

Relax about external benchmarks and focus on improving your own change management processes. That is where most of the value lies anyway.

2. Construct Your Own Independent Survey

If you don’t buy the point above then you can always conduct your own independent Net Promoter benchmark survey. The steps are quite straight forward:

  1. Identify a set of your customers and your competitor’s customers. You will need 100 or more responses to aim for 500+ invites per company in the benchmark. If you are a large B2C brand this will be the easier as you can use survey panel companies to contact a large number of suitable respondents. On the other hand if you are a B2B player or a more niche provider that can be more difficult. How are you going to get hold of the 500 names and email addresses of your competitors customers? There are ways but you’ll need to bring in an external company to enable that to happen.
  2. Design a simple survey that includes the “would recommend” question
  3. Send that survey to the recipients and wait for their responses.
  4. Total up the answers.

[Blatant plug] Of course Genroe can do an independent Net Promoter benchmark study on your behalf that will accurately tell you where you rank against your competitors.

Knowing that, this may seem like odd advice but before you run that type of study I recommend you focus on your relative scores.

Get them working and you’ll already be way ahead of most companies.

Any questions or comments? Feel free to post them below!

Republished with author's permission from original post.

4 COMMENTS

  1. I think one of reasons that NPS has become popular, despite some technical “warts,” is that it’s a common metric calculated exactly the same way from company to company. CEOs like that, because they can compare to other companies in their industry, just like financial metrics (revenue, profit, EPS, etc.).

    Of course creative accounting can make financial comparisons tricky. And so it seems is the case with NPS.

    I didn’t realize scores could change so much based on how the survey was administered.

    And, of course, we’ve all experienced pressures given by companies to give a high score. Just recently a cable service installer did a great job for me, but I was “encouraged” to give an 8 or above, otherwise the installer would have “failed” in his performance. It left a sour taste at the end of a good experience.

    I wonder if NPS will go the way of CSAT, where some companies figured out how get the scores high rather than focus on making real changes.

    My take: unless an external firm is conducting the survey exactly the same way across all the different companies, comparisons are meaningless. Too easy to “cook the books.”

    Thanks for sharing these insights.

  2. Bob,

    Thanks for your comments.

    Most of the skewing factors mentioned above are applicable to other feedback scores (CSAT, CES, etc) but the [Net] part of NPS does make the effect larger. Plus the publishing of benchmark data has made it possible to check and creates a problem that didn’t really exist before.

    I think that NPS is a good and valid metric for customer loyalty. When used as an input the continuous improvement of customer experience it works and it works very well.

    The problem normally comes when organisations (and management) try to use it as a stand alone metric. Then the competitive “my score is better than your score” problem starts. Companies lose focus on the underlying process and just talk about the metric.

    When this issue gets pushed to the performance metrics of front line staff in the wrong way you get the shopping for scores you experienced.

    A good approach implemented the wrong way.

    Adam

  3. I face this issue quite often as our clients and prospects wonder why their internal NPS measurements do not match the ones we produce by opinion mining of customer feedback in social media. The arguments you provide are very similar I have used in explaining the discrepancies, but additionally in our case we arrive to the measurements algorithmically as oppose to tabulating survey results.

    We found the best way to validate whether external data source for NPS is trustworthy for benchmarking, is to compare NPS trends for at least 3-5 periods against internal measurements. If the internal and external NPS trends correlate, the external source of data is acceptable for benchmarking.

  4. Glad you found the research study that Sam & I did of value. Survey data is not like accounting data with common definitions, yet people think they can make cross-company comparisons. This issue isn’t just for NPS obviously, but for any survey data. The reason we did the study was to show in a study with some academic rigor that survey mode changes scores. Not conjecture, but proof – and the statistical significance of the difference was extremely high.

ADD YOUR COMMENT

Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here