The CX Metrics Agnostic: Does it Matter Which Metric You Use?

13
652 views

Share on LinkedIn

I constantly hear from CX consultants that they are “metrics agnostic” or that the choice of CX metrics doesn’t really matter.

If you hear that from one of your consultants, tell them that you are “consultant agnostic” and ask if the choice of consultants really matters.

Me, I am a “metrics empiricist.” That is, I think the choice of metrics does matter, and the best metric is that which is empirically validated to best explain or predict the outcomes or customer behaviors that align with the organization’s objectives.

The End Game
The end game or overarching objective of a CX initiative should be about change. More specifically, changing the experience a firm delivers (or at least the perception of the experience) to motivate desired customer behaviors – continue to be a customer, shop more frequently and so on. The goal is not simply to improve “the score.”

In that limited sense, the metrics agnostics are right in focusing on the doing as more critical than the measuring. Change comes from taking action, both tactically, as when a local manager “closes the loop” after a customer expresses disappointment, and strategically, as when leadership has the service process redesigned to better meet customer needs. In the ideal world, a CX initiative will infuse CX thinking into the bloodstream of a company, permeating the firm, making CX an organic component of how the organization goes about its business day-to-day and plans for tomorrow. The score doesn’t do that.

Taking Aim
Let’s say you want to travel north. Would you follow the advice of a self-described guru who is “directionally agnostic”? After all, if you head south, eventually you might get north as you circumnavigate the globe. A compass might help point you in the (generally) right direction – but the compass won’t get you there by itself.

A CX metric is akin to an organization’s North Star (strained pun intended), providing direction by which to navigate. The question then becomes quite simple: do you want a North Star or CX metric that is more accurate in predicting your destination or business outcomes/customer behaviors or one that is less accurate? Do you prefer to run key drivers on and analyze data that better aligns with the firm’s objectives or that misses the mark? Unless you are “accuracy agnostic,” the answer is obvious.

Design for Success
Every journey starts with a single step. That all-critical first step in developing a CX program is determining the business objectives, be it for the firm overall or a specific activity or transaction type. What do you want to accomplish? Given that your objective is to boost renewals/return visits to the website/increase share of customer spend/make some process easier for customers or whatever target for which you are aiming, the question becomes what is the best metric that will explain or predict your success? You aren’t “objective agnostic;” so why would you be metric agnostic in picking the items to help aim for and monitor progress against objectives?

While this perhaps may be seemingly obvious, all-to-often organizations fail to articulate their objectives or, worse yet, they say their objective is to conduct a survey to collect data. Under those circumstances I return to the wisdom of my friend The Cheshire Cat: if you don’t know where you are going, any path will take you there. In other instances, the company has an objective, but uses a metric that has little to do with what they are trying to accomplish.

There are always intervening factors that affect performance and circumstances that might make it difficult to gauge the success of any initiative. While you are trying to assess the impact of a specific CX initiative or program, the world isn’t standing still: technology is evolving, competitors are on the move, customer expectations are rising, economic and labor factors are in flux. The complicating factors notwithstanding, if you have picked your CX metrics as gauges of performance against the underlying objectives, changes in scores should be an indicator of movement against those objectives.

Linking Metrics to Outcomes
Quantifying the economic value of CX and its impact on the business and customer behaviors is the holy grail. This requires linking CX metrics to business and operational measures. Unless one is “outcomes agnostic,” what could be the possible rationale for not paying attention to the CX KPIs that, in turn, are used to make the business case for CX?

In fact, it is the linkage to outcomes that is central to validating CX metrics. That is, this linkage is the key to determining if the chosen CX metrics are explaining or predicting the stated objectives and outcomes. This is the empirical test for determining the best CX metric: which has the most explanatory or predictive power relative to the targeted business outcomes and customer behaviors?

So unless you are “results agnostic” and don’t care about the success of your CX efforts, the next time someone tells you they are CX metrics agnostic, shrug your shoulders and move on.

13 COMMENTS

  1. Great post. “It’s all physics,” a friend said to me many years ago. We live in a universe governed by the law of cause and effect, and everything we observe is both deterministic and probabilistic. So starting with the outcome and then mathematically evaluating the possible factors contributing to it is the scientific approach to solving the problem. Should someone’s experience and intuition guide the investigation? Sure, provided their hypothesis is tested and their assertion is confirmed or refuted by analyzing the data.

  2. This is the key paragraph:
    While this perhaps may be seemingly obvious, all-to-often organizations fail to articulate their objectives or, worse yet, they say their objective is to conduct a survey to collect data. Under those circumstances I return to the wisdom of my friend The Cheshire Cat: if you don’t know where you are going, any path will take you there. In other instances, the company has an objective, but uses a metric that has little to do with what they are trying to accomplish.

    Too often, organizations not only fail to articulate their objectives or they mix objectives (things you want to achieve) with activities or projects (things you want to do). In many cases the objectives are so general, so vague, so open to interpretation that it is imposible to formulate a relevante metric. The result is that the metrics they use have little to do with what they are trying to acomplish and they are almost useless.

  3. The operative statement in this post, which I absolutely endorse, is “….it is the linkage to outcomes that is central to validating CX metrics. That is, this linkage is the key to determining if the chosen CX metrics are explaining or predicting the stated objectives and outcomes. ” For close to 20 years, and irrespective of whether the vertical is B2B or B2C, the only linked, consistent, actionable and real-world experience metric is advocacy behavior: https://customerthink.com/marketing_case_customer_advocacy_measurement/

  4. Great points in this post. The challenge to me is not the link between the CX metrics and the organizational outcomes (or objectives), but rather the accurate link between the subject and the tools for the metrics. Customer behavior and affinity are as complex as measuring “how much your spouse loves you.” The science (or physics) side of CX metrics world can come up incomplete when applied to emotional dimensions like “I love Starbucks” or “United Airlines sucks.” How do we explain with metrics truly oppressive customer experiences in enterprises to which customers gleefully flock? The infamous net promoter (NPS) metric often overlooks the fact that 25% of customers would not rate their experiences no matter how bad or great they were. To Michael’s point, we long for advocacy behavior, but what does advocacy look like when applied to a funeral home or intensive care. Measuring customer encounters laced with emotion must be approached with wonder and humility, not absolute certainty. And in the era of advancing technology and “artificial” intelligence (quotes intentional), we need to never stop learning about the magic of customer and their ever-changing needs and nature.

  5. I was not familiar with the term “metrics agnostic”. Model agnostic, tool agnostic, but not metrics agnostic.

    As you and other commenters point out, linkage is critical. From data metrics (quality, completeness, alignment with process), to process KPI’s (reduced time per incident, increased average order value) and then to business outcomes (Net promoter score, increased channel revenue).

    Business outcomes depend on process effectiveness which depend on quality data. The linkage should be used to drive course corrections.

    Bottom line is that metrics without context or purpose do not provide business value. The question should always be “so what?” What do you do with the metric? What action needs to be performed as a result?

  6. Great points, Chip, and I’m glad you brought up measuring emotion, which at the end of the day drives all human motivation and decision-making. Keep an eye on developing ‘neurotech’ here! There’s a new company backed by famed neuroscientist Paul Zak called Immersion (https://www.getimmersion.com/homepage) that’s doing some very cool stuff. They’re using Apple Watches to measure changes in heart rate that correspond to emotional states, and they’ve demonstrated 80% predictive accuracy in subsequent decisions. Another company to watch is Receptiviti (https://www.receptiviti.com/platform) backed by scientist Jamie Pennebaker, which mines subtle patterns in language to ascertain emotional state, health, and the nature of relationships. All of these approaches (like a lie-detector test) reveal the truth about what we are feeling by measuring signals outside of our conscious awareness. I think these approaches will become commonplace and they will be much more predictive of actual CX outcomes.

  7. Thank you all for your thoughtful comments and the engaging dialogue.

    Ed, you sound like me: show me the data! Hypotheses are great; now let’s get some proof (or at least some confirmatory data).

    Teodoro, even today (literally) I still see RFPs that have functional objectives (collect data on X from Y), but not even a hint of what the business objectives are.

    Michael, I’ve heard about and lived with your passionate support for advocacy for years, although I’m not certain that it is “the only linked, consistent, actionable and real-world experience metric.”

    Great points, Chip. But the probem isn’t that emotions are or aren’t critical; it’s that company’s sell “stuff” and they are inherently more comfortable “fixing” their stuff than addressing how people feel about their stuff. People — including CXers — are more comfortable calling in the folks in the white coats to improve their products than trying to grasp how to motivate people to feel differently. It’s easier to make the car accelerate faster and quieter or to add another cup holder than it is to address how owners feel about their vehicles — even though the data typically suggest that changing how customers feel has far more impact on their experience and relationship than nominal improvement in products.

    You’re right, Seth: the metrics don’t provide the value. But chosen correctly, they shine a light on what drives value and help illuminate the pathway for action to creat value.

  8. Howard –

    In response to your skepticism re. customer advocacy measurement, all I’ll say is that, irrespective of vertical, the advocacy behavior metric is significantly more accurate at forecasting business outcomes than NPS, CES, Csat, or any similar measure. It tracks with retention, share of wallet, and other financial results. How? Because advocacy measurement yields four levels of brand-related behavior, based on a question battery which includes several pivotal elements not considered in current CX metrics::

    – Customer favorability of brand
    – Future consideration
    – Intensity of brand support
    – Use of online and offline social networks
    – Positive and negative brand buzz
    – Distribution of most recent purchases

    With customer advocacy segmentation and analysis, companies have a) the best understanding of customer loyalty behavior for their brand or business, b) strategic pyramiding of their customer base (to focus on best customers and reduce brand damage from alienated customers), and c) have the most direct linkage to business performance and improvement opportunities.

    For those interested in further detail on the business value of advocacy measurement, see https://asq.org/quality-press/display-item?item=E1410

  9. Without a doubt, Michael, any reasonable multi-variate metric will outperform any univariate measure. Of course, this means additional real estate and complexity, but i’m in the camp that “advocates” for higher precision over those concerns. I’m not exactly sure why you would label a metric based on the variables you identify as “advocacy,” but what is in a name?

  10. Howard, the advocacy behavior metric is, and always has been, an aggregated score derived from scale results to the component questions. And, in the univariate world, for years and years didn’t NPS claim to be the the only CX metric any company would need?

    Why ‘advocacy’? Well, going back about 15 years, it was determined after extensive multi-sector research that collective results of these key questions reflected deep positive brand connection at the high end and deep brand alienation at the low end: https://www.slideshare.net/lowen42/wragg-lowenstein-customer-advocacy

    My statements also include comparisons to aggregated metrics, such as the Secure Customer Index (SCI), which, because one of its components is Csat, rarely lines up well with actual downstream business results. (especially relative to customer advocacy).

  11. Alignment is challenging within organizations. Political or historical trends connected to bonus structures, executive compensation or positive PR has a strong influence on which metric is considered an organization’s north star. In conversations and engagements with organizations across various industries, metric agnostic can be a position taken to defuse a particularly sensitive subject. If two sides cannot come together to discuss how to improve CX over a metric dispute, it is highly unlikely that any progress will be made from the customers’ viewpoint.

    Most conversations must start several steps before a metric is selected to have long term benefits on CX and business outcomes. Understanding the key points along the customer journey and how customers feel when experiencing these touchpoints will tell us where to focus improvement efforts. Locating the root cause of customer dissatisfaction, understanding the emotional response to that disappointment, and measuring the impact of downfalls along the journey, gives us CXers the insight needed to identify and correct underlying issues.

    As issues are identified, addressed, and remedied, the metrics that communicate a customers’ holistic experience should improve. If they do not, and business outcomes do, the organization needs to revisit the potential metric or combination of metrics that best connects business performance with CX. In organizations with particular sensitivity around this topic, having strong data with performance outcomes can be exactly what is needed to align disparate opinions.

  12. To tell you the truth, Nicole, I’m not sure if you agree or disagree. I’m all for flexibility to meet client constraints. There are legacy, political and even legal issues sometimes at stake. (I had a client that had written a particular metric and approach into their 401(k) match!) It is a mistake, however, to confuse flexibility with the ambivalence of agnosticism. Even when we compromise on whatever metric, there still is an empirical answer to “what is the best metric for explaining the desired business outcome?”

    Then you go on to discuss “where to focus improvement efforts” — but improvement on what? Ostensibly you mean a metric of some sort. But don’t you want to focus improvement on the key metric that is most important and not some randomly chosen metric or a suboptimum metric?

    Your last paragraph seems to suggest almost a trial-and-error approach. If the metrics move in the opposite direction of the business outcomes, change metrics. That seems to be my point precisely, with one caveat: don’t wait to see what happens; test the metrics upfront and select the ones that have the most explanatory power. In other words, don’t preach agnosticism and take a wait-and-see approach. Preach empiricism and scientifically test your metric.

  13. Howard thank you for your response. It seems that my position to keep an open mind at the outset of a conversation related to selection of the best metric may have been misconstrued as advocating for agnosticism or a form of ambivalence. This is not the case. I do agree that the data must tell the story. What I’ve encountered most often is that metrics are not tied to business outcomes. They are selected based on the preference of an executive or historically embedded within the organization. In these instances, we should advocate for the openness to consider and analyze the appropriateness of the metric that is being followed. This makes the possibility of unseating a poor performing metric in favor of one that does have an empirical connection between CX and business performance more palatable.

    My discussion on improvement efforts is not tied only to those items that move the metric. I agree that when an organization is following a key metric that is empirically tied to successful business outcomes, it should be used as a guide to focus CX improvement efforts. Practically, however, most organizations have not conducted a return on investment analysis related to their CX performance. That caveat aside, those aspects of the customer journey that have the greatest impact on the key metric of choice but fail to perform at the desired levels (aka key drivers) are often also the most difficult to improve. Not impossible, but difficult in that they require more patience, diligence, time and highly coordinated improvement initiatives to remedy.

    Knowing that it will take time and increased effort to move the needle on these areas of the customer journey, we can build momentum through tackling items that are more easily corrected. These items are identified in a full journey analysis and while they may have less impact on the organizational key metric, there is still value in improving their performance. Achieving a positive change in an established point of irritation can provide the proof of possibility needed to get employees behind a larger effort. Finally, when communicated, these quick win initiatives will also help to show customers that the organization is listening and taking action reinforcing their commitment to improving CX.

LEAVE A REPLY

Please enter your comment!
Please enter your name here