Did you know that CSI can help you understand, in just one word, the health of your customer relationships? No, I don’t mean Crime Scene Investigator, although that might help in some cases. I am writing about the Customer Sentiment Index (CSI), a measure based on a single word that customers use to describe a company.
The CSI is calculated using one-word responses from the following customer survey question:
What one word best describes this company?
Results show that the customer-generated words can be reliably scaled along a sentiment continuum and that these scores (CSI) are logically related to important customer experience metrics, like customer loyalty and satisfaction with the customer experience. The CSI is useful in helping businesses understand the health of their customer relationship.
Measuring Customers’ Attitudes using Structured and Unstructured Data
You can measure customers’ attitudes in two ways. One way to measure customer satisfaction, using standard questions that require a rating (structured data) from the customers. While there are different rating scales (1 to 5, 0 to 10, 1 to 7), the underlying premise of all of them is that the value of the rating scale is used as the measure of customer satisfaction. Lower ratings indicate lower satisfaction than higher ratings (e.g., 0 = Extremely Dissatisfied to 10 = Extremely Satisfied). Customers convey their satisfaction through the ratings they choose to provide; customers who give higher ratings (say, a rating of 10) are more satisfied (have a more positive attitude) than customers who give lower ratings (say, a rating of 3).
Another way to measure attitudes is to apply sentiment analysis to unstructured text regarding what customers say about you (unstructured data). Formally, Wikipedia says “sentiment analysis (also known as opinion mining) refers to the use of natural language processing (NLP), text analysis and computational linguistics to identify and extract subjective information in source materials.” Unlike the use of structured questions and ratings, in sentiment analysis, algorithms are used to assign a numerical value to what customers say. This numerical value represents the customer’s attitude (from positive to negative) about you.
In One Word
I crafted a survey question that you can use in customer relationship surveys. This question asks the customer (respondent) to provide one word that best describes the company. As such, this measurement approach is designed to intentionally elicit descriptive words about the company. Customers are free to write anything for their response. I have used a couple of slightly different questions, but both appear to show similar results. The question is:
What one word best describes this company?
Scoring the Customer Sentiment Index
To be useful in analysis, I needed to map each word along a sentiment continuum, essentially assigning each word a particular sentiment value. I developed a sentiment lexicon, a list of words that reflect an opinion on a negative-positive continuum, and this lexicon to this list of one-word responses to determine each word’s sentiment value. The sentiment lexicon is scaled from 0 (negative sentiment) to 10 (positive sentiment).
Correlates of the Customer Sentiment Index
I used the CSI in two independent studies, one using a B2B technology company and the other using a B2C wireless service provider company to determine if the CSI is related to other common CX metrics. Results from both studies showed that CSI scores were positively related to customer loyalty measures. Customers who have higher CSI scores also report higher levels of customer loyalty compared to customers with lower CSI scores. Additionally, CSI scores were more highly correlated (average r = .37, r = .46) with advocacy loyalty (i.e., overall satisfaction, recommend, buy same) than with purchasing (i.e., buy additional, expand use) and retention (i.e., renew) loyalty (average r = .18, r = .27).
Additionally, in both samples, CSI scores were positively related to each of the CX measures. Customers who have higher CSI scores also report higher levels of satisfaction with different CX touch points compared to customers with lower CSI scores.
Identifying At-Risk Customers Using the Customer Sentiment Index
Figure 1. Relationship between the CSI and Likelihood to Recommend for both (B2B and B2C) study samples.
Customer surveys can be used to identify at-risk customers. “At-risk” is defined as those customers who will exhibit disloyal behaviors (e.g., switch) or those who will not exhibit loyalty behaviors (e.g., not recommend). I examined the relationship between CSI and the likelihood to recommend question (see Figure 1). The recommend question was grouped into segments made popular by the Net Promoter Score: Detractors, Passives and Promoters.
The relationship between the CSI and recommendation intentions was more linear for the B2B sample (top plot) compared to the B2C sample (bottom plot). For the B2B sample, the CSI cutoff point at which most respondents become Detractors is around 3.0. For the B2C sample, the comparable cutoff point for the CSI is around 6.0.
Summary and Implications
Asking customers to describe a company/brand using one word appears to hold value in helping businesses understand and manage the customer relationship. The Customer Sentiment Index (CSI) provides reliable, valid information about customers’ sentiment. The CSI measures customers’ general attitude about your company/brand.
The CSI was positively correlated with all customer loyalty and CX metrics. Customers with higher customer sentiment reported higher levels of customer loyalty and were more satisfied with their customer experience. The CSI was more closely associated with advocacy loyalty (e.g., overall satisfaction, likelihood to recommend) than with other types of loyalty and CX metrics, suggesting that the CSI measures customers’ general attitude toward the company.
As part of a customer experience management programs, businesses need to identify at-risk customers so they can address their concerns immediately. I illustrated how the CSI can be used to reliably identify at-risk customers (e.g., not likely to recommend). Setting the right cutoff point that optimizes the identification of loyal and disloyal customers varied across the B2B and B2C samples.
The process of using a single word as a measure of customer sentiment/satisfaction is useful for a variety of reasons:
- The CSI provides an good overall metric regarding the quality of the customer relationship. The CSI can augment current annual customer metrics. Rather than waiting an entire year to survey customers using traditional customer relationship surveys, businesses can easily employ the CSI to gauge customers’ attitude throughout the year.
- Customer experience professionals needs to optimize how they capture customer feedback in this mobile age. Soliciting customer feedback over mobile devices needs to be quick and easy for the customers. The CSI method is simple and easy for customers.
- Compared to traditional survey questions that require satisfaction ratings from the customers, the CSI method provides richer information about your customers. In addition to extracting sentiment from the words, the specific words themselves may provide some additional insight. For example, businesses can study specific words to understand the sentiment behind their usage and improve their customer communications by using words that resonate with certain customer segments. Interestingly, the two subject matter experts thought that word, “Expensive,” held a negative sentiment. In reality, the customers who used the word, “Expensive,” to describe the company were satisfied with the company.
- You can build your own sentiment lexicon for your company by including the CSI question and a rating question of overall satisfaction. Building specific sentiment lexicons for your business could help improve the predictive power of the CSI. Additionally, a company/industry-specific sentiment lexicon can be used for other analytics requiring natural language processing (e.g., analyzing sentiment of emails or of contact center conversations). Having a better lexicon helps you make better predictions and decisions.
- The methodology proposed here can be used to measure the sentiment of other stakeholders. For example, I used this one-word method in an employee survey to assess employee sentiment and found that an employee sentiment index could be useful to managing employee relationships.
The CSI measures the extent to which customers hold positive sentiment toward you. I will continue exploring different uses of the CSI to show how companies can use this metric in their customer experience management programs.
For the interested reader, details of the research on the CSI can be found in a series of blog posts:
Bob, fantastic food for thought.
Hello Bob,
Honestly, I find myself saying “So what!”. OK: there is yet another way doing measurement. What difference does it make? As I work with clients what is it that I find missing? The willingness, the courage, the passion to take action.
All the best
maz
Maz,
Thanks for your question, “So what?”. I think it’s a good one and needs to be asked more often. Looking back on some of my early notes on this research, I found that I even wrote to myself, “Who cares?” I tried to answer that question via the six bullet-points in the post. But there is a bigger reason.
I wholeheartedly agree with you that a big part of improving the customer experience is about “the willingness, the courage, the passion to take action.” In my research on CEM best practices, I found that factors related to strategy of the program (e.g., use customer feedback in executive compensation, CF as important as financial measures, CF incorporated into company’s strategy/goals, executive is champion of feedback program) are a big driver of CEM program success. Operational aspects of your CEM program (including measurement) don’t matter as much as strategic aspects of the program.
My take is that the lack of executives’ willingness to take action could be due to information overload. Their executive dashboards are cluttered with too many metrics. I’ve seen dashboards that are so overwhelming, they need their own summary dashboard. Furthermore, with our ability to track so many metrics in the era of Big Data, there are so many different pairs of variables to examine, executives get stuck in viscous circle of continuous hypothesis testing, not taking action any given result (aka analysis paralysis).
One solution to reduce the number of customer metrics reported in dashboards is to include fewer questions in your customer survey. I found that short customer surveys provide just as much information about drivers of customer loyalty as longer surveys. We simply ask our customers too many questions that don’t provide any insight. While I’ve stated elsewhere that about a 20-question survey is ideal, my latest thinking is that maybe we need even shorter surveys.
If the primary goal of relationship surveys is to help executives set company strategy and effectively allocate resources that optimize ROI (and I think that it is), we need to think about how to optimize that process with the fewest questions possible. The work on the CSI is one step toward that goal.
Also, I think that the one word method I propose provides more information than a single rated question. Conventional questions with ratings measure one thing. In addition to sentiment, perhaps there are additional constructs we might be able to infer from customers’ words by applying other lexicons (e.g., stress, anxiety level) to the words, perhaps giving addition insight on how to best manage that relationship.
Also, using this one word method as a measure of advocacy loyalty reduces the problem of common method variance that can artificially inflate the correlation between satisfaction and loyalty. The correlation between CX satisfaction and loyalty is used to denote the importance of satisfaction on improving customer loyalty. The size of the correlation is due, in part, to the fact that the same method is being used to measure both satisfaction and loyalty (i.e., self-reports using a similar rating scale). Because of the problem of common method variance, I think that we have been over-inflating the importance of the customer experience, perhaps contributing to an unrealistic expectation on the part of executives of what CX improvements will do. Maybe the executives’ lack of action stems from their lack of belief of what ROI analyses of CX improvements typically show.
Self-reported loyalty metrics are unnecessary in the world of big data. Companies combine different data silos, including metrics about real customer loyalty behaviors. These are the loyalty metrics that need to be used as dependent variables in statistical modeling. We no longer have to ask our customers what they think they will do. We will know exactly what they will do.
Customer relationship surveys have remained pretty much the same over my career. My ultimate goal is to simplify the customer relationship survey. Maybe the solution to improving executives’ willingness to take action is about cutting out the unnecessary questions.
That research is still to come.
Hello Bob,
I thank you for making the time to share such a thorough explanation of what you are up to and why. Totally get it. If it matters then please know that I find myself supporting you on your mission – to make life simpler; to take unnecessary (almost always man made) complexity out of the picture. I appreciate the “zen-ness” of what you are doing.
Much more important than your explanation is the tone of your explanation. Thank you. I find myself liking you and hoping that one day our paths cross and I get to buy-share a meal with you. And in the process get to know you better.
Until our paths cross in person, I wish you the very best.
At your service | with my love
maz