“Are My Scores Good?” Well, Compared to What?


Share on LinkedIn

Once Upon a Time…

It’s the 2nd business day of the month. The Digital Channel Manager just received the monthly scorecard, and she can’t wait to open it and see what her OSAT score is this month!

“Oh, yes!” she exclaimed. It was a 72%!!! This was great news, because it was only 69% last month, and it had not been above 70% in a year.

Well done!” she thought to herself.

A week later, she set off to the quarterly BU Action Planning Meeting with great confidence, having increased her key metric over the past month. She smiled as she entered the room.

“I won’t be picked on once. In fact, I may not even have much action to come out of this meeting at all!

Well, she was wrong. The Director of Customer Experience opened the meeting by reviewing the overall key metrics obtained for all departments, followed by some industry benchmark comparisons. The latter one was not shared in the past three meetings she had attended, so this was new for the Digital Channel Manager.

“We are not doing very well in the Digital Channel performance,” revealed the Director.

What he was saying didn’t make sense to her, since they had seen their score rise by three points.

The Director continued, saying, “Across ten companies measured in the competitive benchmark study, we are among the bottom three, 11 points below the market leader.”

“Wow, that’s a big gap!” The Channel Manager reflected on what was just shared. Her smile disappeared. “Am I not doing very well, then?”

All Things Relative

The story may have turned out to be gloomy, but unfortunately it is not uncommon for such an instance to unfold. The important reminder in this story, and in real-life situations, is that all scores are only meaningful in relative sense.

It may be great to see a change in scores, but what does it real reflect? And in that regard, compared to what standard? Where is the line drawn between great, or just OK? When setting out to measure progress within our CX programs, it is important to pay attention to relative comparisons while evaluating business success and determining where to refocus investments for improvement.

In this modern day and age of CX, most organizations measure their customers’ experiences at a high level across multiple touch points. These organizations monitor key metrics from these measurements over time to assess if they’re improving, or if there are new areas to apply focus on for better scores.

Measuring only “internal” CX lacks the frame of reference the decision makers need—especially for key business metrics such as growth and retention. To achieve a holistic view, businesses must be able to assess how good or bad they are doing in the marketplace, and how they need to prioritize CX improvement initiatives in order to maintain or gain a better position in the marketplace. This frame of reference comes from competitive measurement of experience, among all consumers of the product and service categories related to the company’s business. It also comes from measuring and understanding what consumers think our business is doing compared to other businesses, and where are they’re doing better or worse than competitors.

This is the frame of reference needed, because at the end, besides doing the right thing for the consumer, companies have certain business and financial goals to meet.

Competitive measurement has several benefits for the organization, including:

Unbiased Measure of Experience (blind survey)

  • Unbiased feedback from customers and prospects
  • Objective and realistic reflection of actual experience

Relative Competitive Performance Measure

  • Better measure of true success in the marketplace
  • Business outcome-focused performance goals and focus areas determined based on competitive position

Perceptions Based on Comparative Perspective

  • Consumer feedback and perceptions based on their expectations built by exposure to competitors or other comparable experiences
  • New strategies for solution innovation, operational improvements, and marketing messaging

Insights on Customer Consumption

  • Better understanding of penetration in the marketplace, use of single vs. multiple-brands, and usage behavior
  • New strategies for solution innovation to consolidate customers in one place to fulfill their needs, to reduce risk of exposure to competitors

Unbiased Measure of Experience

Competitive surveys are usually conducted using a panel sample of all consumers in the product/service category related to the company’s business. Since it is essential to include non-customers of the company, in addition to customers, panel sample is leveraged to include all relevant consumers. This allows companies to capture comparative perspectives. Additionally, when speaking to both customers and non-customers, even if the company’s list is utilized, it is advised to conduct these studies blindly, meaning without revealing the sponsor of the survey. It is shown through research that customers tend to provide higher ratings for a company if they know the survey is done on behalf of the company, than if it was just a sponsored survey.

Therefore, blind competitive measurement tends to provide a more unbiased reading of customer perceptions and opinions. Some organizations find this beneficial not only to obtain objective opinions of their consumers, but also to check and calibrate their sponsored customer-list satisfaction surveys; considering the potential respondent bias for knowing that they sponsored the survey.

Relative Competitive Performance

As indicated by its name, Competitive surveys are mainly used to identify the company’s position in the competitive marketplace. The primary goal is to see where they are compared to competitors offering same or similar products and services, and to determine strengths and weaknesses. These studies showcase the key areas that the consumers care about, also helping to identify the main areas needing focus, specifically the areas that consumers weigh heavily on when making their purchase decisions. Strategies and action plans can be built around these specific areas, competitively improving the customer experience.

Comparative Perspective

Frequently it is questioned why different customers perceive the same experience differently. There are many factors that influence perceptions, including personality, demographics, and individual needs.

Another key factor that influences a person’s experience is the frame of reference they align their expectations to. The expectations going into an experience are built by the same personal characteristics mentioned above, and by relevant past experiences.

For example, when making retail purchases online, I personally find it hard not to compare my purchases or overall experience to all the tens of purchases I’ve made before on Amazon.com. It’s not only past experiences that are directly relevant, such as online shopping, but also experiences that are similar in terms of the method or channel. Therefore, as a customer, the standard for experience is not just measured against your own company but measured against the entire industry.

An example of this type of experience comparison is the app I use for my gym to pay my dues and schedule classes. One day I looked on the app for some way to be notified when my monthly payment is made, so that I can use that as a reminder to go in and schedule next month’s sessions. On the app I use for banking, I can set notifications for payment reminder alerts. This is an extremely helpful tool that has made my mobile banking experience easier. Although the two companies, my gym and the bank, may serve me in different ways, I can’t help but compare the usage of their apps as a consumer; favoring the more useful of the two.

Consumers build certain expectations, based on similar instances, which greatly influence how they evaluate each of their new experiences. These “other experiences” differentiate consumers’ perceptions of the same experience with a company. In my case, I am disappointed with my gym app due to my expectations built based on the experiences with my banking app. Yet, someone else who may not have been exposed to notification settings on other apps wouldn’t have any preface to compare to. They wouldn’t have the same reaction as me, having no disappointment over a lack thereof. Since every customer experience is different, it is important to understand consumers interactions with competitors and build a frame of reference based on their expectations. A more holistic understanding, in turn, helps to identify what really matters to customers, what the key differentiators are, and what innovate product/service features are necessary in improving the experience being delivered.

Purchase and Consumption Behavior

Finally, Competitive surveys can be leveraged to better understand customers’ and furthermore general consumers’ product/service usage:

  • Whether they use one brand or multiple brands;
  • If multiple brands are used, do they use the same product type from multiple brands or do they choose different brands for different products?
  • If they use multiple brands for the same product, how do they use those products differently or why they use multiple?
  • What needs drive this choice; how did they choose the brand?

These explorations about how consumers choose brand and products versus competitors will determine strategies with the end goal of consolidating more business, and reduce the risk of customers’ exposure to competitors for variety of products or needs.

Competitive measurement of customer experience is a beneficial tool and an essential component of understanding the overall customer journey, the customers’ needs and expectations, and how customers find what they’re looking for whether it’s with us or our competitors.

In the end, the insights received from measuring customer experiences from a competitive standpoint provides a relative idea of positioning in the marketplace; the place where customers assess their options to meet their needs and expectations. Being exposed to a variety of experiences in a vast retail marketplace constructs a consumer’s expectations for future experiences, and only assessing their competitive position will companies be able to identify their strengths and leverage them for a plan of action. Determining new strategies and plans based on the identified gaps, across the entire board, will gain competitive advantages and increase in market shares. At the end of the day, consumers will always make comparisons of what they like, to no particular standard.

Knowing what’s going on outside of the business market is just as important as what’s on the inside. Like the Digital Channel Manager from the story, getting excited about results is a good thing. But by taking it further and considering the whole picture, a holistic view of the competition will be achieved and pave the groundwork for great results.

Republished with author's permission from original post.

Funda Whitaker
I lead the Strategic Research & Insights teams at InMoment, managing a group of highly-experienced researchers and CX consultants who design, execute, and deliver strategic market research projects for our clients. I have been with InMoment (and its preceding organizations) for over 20 years holding a variety of positions from research and project management to department leadership, with experience in various client sectors conducting a variety of research techniques. I also consult for CX program design and insights. I obtained my master in Marketing Research at UT Arlington.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here