Get ready, because I’m going to ramble at bit.
I recently attended a networking meeting with about a dozen customer experience (CX) professionals and our topic of the day was metrics and measurement. We each shared some of the things we measure as part of our CX initiatives and I was impressed with the long list. Here are the items that were mentioned in no particular order (this is where I ramble):
Implementation performance, meeting customer KPIs, usage, adoption, willingness to refer, NPS, customer retention, renewals, customer effort, transaction satisfaction, quality of the experience, customer loyalty, ease of doing business, customer satisfaction, on-boarding, early warnings, partner preference, on-time to commitment, repair turnaround, open tech tickets, order activity, invoice accuracy, first-time, resolution, forecast accuracy, call answer rates, key drivers, customer engagement, personal value, and depth of relationship.
As you can see, the list provided a range of measurements, from broad metrics that measure the overall customer relationship to very specific measurements of a single customer interaction. It was also clear from the discussion that there are plenty of challenges in the way we measure CX – response rates, accuracy, shifting circumstances, benchmarks, resistance from colleagues, isolating metrics, tying performance metrics to compensation, and linking metrics, are a few that were mentioned.
What can we make of this? For me there were three primary takeaways:
- First, every business is different. We had a broad range of companies assembled – insurance, software, professional services, engineering, and more. It’s clear that there is no single metric or set of metrics that will work for everyone.
- Second, measure what matters – not just to you. It seems rather obvious that we should measure the most important metrics to help run our business. However, much of the discussion was about measuring the things that matter most to our customers. That made sense to me. Besides, what matters to the customer and to my business should be pretty much the same.
- Third, measure what is actionable. Sometimes it appears that we measure things because it is interesting or because we always have. If it isn’t going to put to use, it’s probably a waste of time to measure it.
The great Michael Quinn Patton wrote a series of theorems (with tongue in cheek, he labeled Halcolm’s Evaluation Laws) in his classic 1980 work: Qualitative Evaluation Methods. Here are a few of his theorems to guide CX Measurement:
* Conceptualizing an evaluation depends on understanding self-interest: yours and theirs. Useful evaluations put theirs first. Then there are the others…
* The scientific status of a methodological approach has nothing to do with it’s appropriateness. And, vice versa.
* Evaluation is too serious a matter to be done by someone who has never been a client in a program.
* An evaluation not worth doing is not worth doing well.
* Evaluation results always make clear to people what they had really wanted to know but forgot to ask.
* Because you can name something does not mean you understand it; because you understand it does not mean it can be named.
* Every evaluation serves a purpose, even if it is only to be a horrible example to others.
* The perfect evaluation design isn’t.
* Always be suspicious of data collection that goes according to plan.
* Research subjects have also been known to be people.
* Make sure when you yield to temptation in the field that it appears to have something to do with what your are studying.
* Evaluators are presumed guilty until proven innocent.
* And, my favorite which he called “The Law of Divine Intervention”: All skill is in vain when an angel pees in the touchhole of your musket!
Patrick, to me it seems that there are basically two types of measurements. The first is the kind that are designed to yield improvements that will benefits the customers first and then the organization. The second type is what I call the Gratification Survey when the objective is to get the “highest” score possible. We typically see this in car dealerships and repair centers and even in hospitals. These surveys waste the customer’s time and send a crazy bad message.
Also, one real value of surveying is finding out what is important to individaul customers so you can fine tune your delivery to them. Then you can combine individuals into segments with like needs and leverage what you learn from individuals.
And each company, segm,ent and customer is unique so you have to work at getting real value from a survey program.
Agree with your three takeaways. When you conclude that “t’s clear that there is no single metric or set of metrics that will work for everyone”, I’d concur that the search for a universal metric has, over the years, had something of a Holy Grail quest story attached to it. That said, there is a measurement framework that ticks the key boxes. It’s customer advocacy behavior (incidentally, not included in your laundry list of measures), which many of the major consulting organizations have posited and endorsed for over a decade, and which we have been effectively measuring and applying for almost as long.
Advocacy combines rational/emotional brand attitudes and brand favorability with informal communication about the brand to others. It identifies all brands within the consideration set and determines strength of brand franchise based on customer experience. In studies within many b2b and b2c verticals around the world (and often multi-country and multi-continent), advocacy has proven to be the best, and most actionable on a granular level, predictor of business performance.
We didn’t ‘invent’ advocacy. We quite successfully operationationalized it into a measurement framework. It can delineate customers into key behavioral segments, monetize at a high and consistent level across industries, and offer detailed, prioritize guidance for making improvements…
As my colleague Professor Philip Kotler has stated: “Leading companies use the word-of-mouth effect of unpaid advocates – truly loyal customers – to boost their reputation. Advocates will do your marketing for you….” A 2011 peppers & Rogers Group white paper, Cultivating Customer Advocates: More Than Satisfaction and Loyalty, concluded: “The benefits of building advocacy can’t be ignored. Satisfaction and loyalty are important, but they’re old news. It’s a new dawn in customer experience strategy, where the customer controls over 50 percent of the brand message. Forward thinking companies will be the ones that identify and work with their customer advocates to genuinely build trust in the brand, the customer base, and the bottom line.”
W. Edwards Deming, in one of the least-repeated, most misunderstood quotes by a statistical luminary, said “The most important figures that one needs for management are unknown or unknowable . . . but successful management must nevertheless take account of them.”
Somehow, people reduced that wordy axiom to a catchier, but more error-prone phrase, “you manage what you measure.” Yep, true! But so many businesses wind up managing things that don’t provide value to customers, or don’t matter at all. Or, they focus on “what the numbers tell us,” to the exclusion of crucial qualitative facts that smack them upside the head – repeatedly.
So the challenge isn’t in picking out things to measure (I like your long list), it’s in selecting the right things, measuring them without skewing the results, developing useful insights, and making intelligent decisions.
For me, the framework that comes closest to ensuring that irrelevant measurements can be avoided is the Balanced Scorecard (see balancedscorecard.org), which has been used since the 1990’s. Balanced Scorecard ties actions and measurements to a company’s strategic goals. Sadly for some, it’s not plug-and-play software. It’s a management perspective, which takes rigor and ongoing effort to adopt and use.
A related article I wrote might be of interest, Do You Mangle What You Measure? Eight Pitfalls to Avoid.
Hi Patrick – I very much agree with you an the other thought leaders who have commented on your post. I always teach CX professionals that there are four ‘principles’ of measurement:
1) Strategic
2) Actionable
3) Predictive
4) Sustainable
I will not provide a detailed description of each of them here, as I have that planned for a future column. However, based on my interaction with organisations all over the world, I can say that CX measurement is a core competency that is NOT being applied very well. Too many businesses are measuring the customer experience but not clear on what to do with it – that is not linked to the business strategy, that does not enable the organisation to proactively improve the customer experience etc….
It is therefore no surprise that one of the competencies that aspiring CCXP graduates are falling down on is their knowledge of ‘metrics, measurement and ROI’.
Once again, thanks for the thought provoking piece – very well written!