In the wake of the shooting spree of Uber driver in Michigan, there have been a whole lot of discussions about how Uber customers go about rating drivers and whether a 4.73 on a 5-point scale is a good score or not. One analyst I listened to said that 4.73 is actually a bad score and Uber should have known there was something wrong. This analyst went on to say that when Uber customers rate drivers they feel compelled to give 5 stars unless the driver is just bad. So the argument goes, this driver was clearly not up to snuff.
Other analysts I listened to argued that 4.73 is actually a good score. If you’re someone who is used to CX research and has seen a lot of programs, it would not be unreasonable to come to that conclusion. Obviously, if I had all of Uber’s data, it wouldn’t be hard to say whether 4.73 is a good score or not. However, that’s not the point of my letter to Uber and any other organization that collects peer to peer ratings. Stop copying Amazon and get a better measure.
When you determine what metric you should use for your CX survey, there are two considerations you should keep top of mind. 1) does the metric allow for sufficient discrimination in ratings so that you can understand the breadth of performance; and 2) does it provide you with meaningful insight into your performance that enables you to take action?
Let’s assume for a moment that Uber’s ratings don’t discriminate very well and there are a lot of driver’s out there with a rating of 4.9 or above who all seem awesome, but actually are not all the same.
Here’s the problem: Uber needs to be able to differentiate between experiences that are truly awesome and those that met the expectations of the customer, as well as those that were problematic. Today, we might be able to identify those that are truly problematic, but we almost certainly cannot differentiate between good, better and best.
One solution could be to lengthen the scale, however, I don’t believe that would solve the problem and it would make the survey more difficult to do for customers. One could also change the question to some other metric, but that seems difficult given that it is just simply asking satisfaction with the service provided. A third option would be to keep the same question, but change the type of response we are asking for. In this option lies the solution to Uber’s problem.
Good enough gets you by with customers, but better and best experiences create loyal fans. So one suggestion might be the scale below:
3 thumbs down image & 1 thumb down image “OK” 1 Thumb up & 3 thumbs up image
Maybe it is not quite as simple as the 5 stars, but it is not far off. However, it lessens the bias toward giving 5 stars and allows a driver to get a “thumbs up,” without it feeling like the customer penalized them. It is still a 5-point scale but it better represents what is happening in the transaction and what you want to learn from customers.
EFM technologies have made it very easy to collect data from customers with an inviting, easy to use interface. Whether we are talking about a 5 star rating that is extremely prevalent online or a metric like Customer Effort Score, no metric is universally effective.
Many industry analysts espouse using one metric or another consistently wanting everyone to drink their Koolaid, but you shouldn’t fall to temptation.
Uber and other companies, please rethink your rating scale. Its not working.
