Top

Airbnb, Travelocity, and Hilton Teach the Bad, Better, and Best of Net Promoter Surveys 

Seth Grimes | Jul 20, 2017 277 views 1 Comment

Share on LinkedIn

The Net Promoter Score (NPS) is a go-to customer satisfaction metric. It relies on a simple question, How likely is it you would you recommend <a product or service> to a friend or colleague? You respond on a 0 to 10 scale, and the net score is computed by subtracting the percentage of  0-6 responses, a.k.a. “detractors,” from the percentage of “promoters,” those who give a 9 or 10 rating.



NPS’s appeal is the simplicity and also the score’s widespread use, which allows for cross-industry and within-industry comparisons. The drawback, as many have noted, is opacity: NPS alone explains nothing. That’s why I’m big on survey questions that allow free-text response – there’s no substitute for asking Why did you give that score? – and on review analysis and social listening and the use of sentiment analysis techniques that link feelings to product aspects and issues. But let’s consider the opacity issue solvable and turn to proper and improper NPS use. We’ll do that via three examples…

Bad: Airbnb

Check out this Airbnb survey I received, in response to a customer-service interaction I had in the wake of a host’s reservation cancelation two days before my arrival date:

Airbnb-NPS

My first reaction?

Airbnb doesn’t know whether agent Jarrod W was able to help? Really? Why not?

Actually, Jarrod W wasn’t able to”sort everything out” for me. As a result, I think that Airbnb’s resolution of issues caused by late host cancellations sucks. But I still like Airbnb overall, so here’s the dilemma: Jarrod W’s ineffectiveness won’t actually affect my likelihood to recommend, but I don’t want to imply that he was effective.

Airbnb chose the wrong measure.

Better to measure customer effort when you seek to understand service/transaction satisfaction. How hard was it to accomplish get the support you needed?

This said, I’ll relay another opinion, from Jean-François Damais, Deputy Managing Director at Ipsos Loyalty.

We found that measuring customer effort is not enough. It is the customer::company effort ratio that really matters, company effort being defined as customers’ perception of how much effort a company puts in to deal with their issues. When customers feel they work harder than companies to deal with a CX issue, churn rates and bad word of mouth are extremely high.

Bottom line: Simplistic metrics, misapplied, won’t get you far.

Better: Travelocity

By contrast, Travelocity does it better although there’s still room for improvement. Here’s the classic NPS question, this time focused on the platform itself:

Travelocity-NPS

Note an extra cue given to the customer: color-coded choices add graphic appeal not present in Airbnb’s survey, without adding complexity. The flaw in this survey is, however, a missing bit of complexity. Fact is, I am unlikely to recommend Travelocity as a place to book travel because the question rarely comes up. The Airbnb survey had the same flaw. If I answered the question as asked, my response would be a 1 or 2 even though I was satisfied and would have answered 8 or 9 if asked “Are you satisfied with Travelocity travel booking?” Instead, do what Hilton does in my next example. Add a few words in order to focus the question on real-world situations.

Bottom line: Wording counts.

Best: Hilton

Hilton sacrifices simplicity for precision, in a survey I received recently. Check it out in the image below: Satisfaction questions and the Net Promoter “How likely would you be to recommend…” question, each type properly used. The survey finishes by asking “value that you received for the price paid.” I like that Hilton asks about perceived value per my article, Perceived Value Is Key To Customer Experience. And the use of a value ratio is in keeping with the effort-ratio guidance offered by Jean-François Damais, quoted earlier.

Here’s a shot of the Hilton survey:

Hilton-survey

One small curiosity: Hilton’s scale descends from 10 to 1 rather than, as in the class Net Promoter survey, ascending from 0 to 10. I’m guessing that the survey designers prioritized offering a single type of scale for the three types of question over using the type of scale – the number of choices and the order – convention for each survey type. Good choice. My only recommendation is to add some free-text response questions, to enable you to get at reasons and root causes that explain responses.

So in conclusion: Airbnb, don’t take my low score to mean I’m an Airbnb detractor. I’m not. Rework you’re survey approach and you’ll find that out. Travelocity, wording counts. If you don’t get it right, you risk getting distorted answers. And Hilton: Nice going. Give your customer experience/market research staff a bonus. I’d recommend them, if a business asked me who to hire to design a customer survey.

Print Friendly, PDF & Email


Recent Editor's Picks:


Categories: BlogEditor's PickVoice of Customer

277 views

One Response to Airbnb, Travelocity, and Hilton Teach the Bad, Better, and Best of Net Promoter Surveys

  1. Bob Holley July 24, 2017 at 7:23 am (28 comments) #

    I don’t know how to answer the question about making recommendations because I’m philosophically opposed to making them. I’m not willing to lose a friendship or annoy a relative by making a recommendation that turns out to be wrong. I might like the service or product that I bought, but the person to whom I made the recommendation may buy another one that is not so good. I’m especially bothered about making recommendations about financial services where the other person’s money is at risk. As anyone who knows anything about statistics, one transaction does not provide a valid measure of a broad range of services or products from a company.

    Since I take a lot of surveys, what suggestions do you have for me when I encounter this question?

Add Your Comment (All comments are reviewed by moderator, no spam permitted!)