I recently developed a method to measure customer sentiment, the Customer Sentiment Index. This method combines a structured and unstructured approach to measuring attitudes. I’ve gathered evidence of the reliability and validity of this measure (also, see here,here and here), showing the usefulness of this measure for CX programs. In this post, I study the stability of customer sentiment (as measured by the CSI) over time.
A B2B technology company included the open-ended survey question in their annual customer survey, “Using one word, please describe COMPANY’S products/services.” The Customer Sentiment Index (CSI) was calculated by applying our sentiment lexicon to each of the words provided for this question. CSI scores could range from 0 (negative sentiment) to 10 (positive sentiment). This survey also included a question that required customers to provide a rating on a measure of overall satisfaction (0 – Extremely dissatisfied to 10 – Extremely satisfied) and a rating about their likelihood to recommend (0 – Not at all likely to 10 – Extremely likely). The survey also included 12 customer experience (CX) questions that required customers to provide satisfaction ratings (0 – Extremely dissatisfied to 10 – Extremely satisfied).
The time period between these two survey administrations was about six quarters; the first survey was administered in the Spring of 2014, and the second survey was administered in the Fall of 2015.
Descriptive statistics of and correlations among the study variables are presented in Figure 1. Average ratings did not change meaningfully over the two time periods for any of the three measures. The average sentiment rating was 7.10 for both time periods. A total of 68 respondents completed surveys from both survey periods. Consequently, the correlations in the bottom part of Figure 1 for CSI, Satisfaction and Recommend are based on samples sizes of 36, 68 and 67, respectively.
Stability of Customer Sentiment
The correlation between CSI from time 1 and time 2 was .60 while the corresponding correlations between the satisfaction and recommendation questions were .42 and .27, respectively. This pattern of correlations shows that customer sentiment is more stable over time compared to recommending intentions. People who reported positive sentiment at time 1 tended to report positive sentiment at time 2, while people who reported negative sentiment at time 1 tended to report negative sentiment at time 2. This pattern was less apparent for the recommend question.
Predictability of Customer Sentiment
Using the survey data collected at time 2, I found that satisfaction ratings with the 12 customer experience (CX) areas were more highly related to recommendation intentions (average r = .53) than they were with the CSI (average r = .29). This finding suggests that improvements in the customer experience will have more of an impact on improving recommendation intentions than it will on improving customer sentiment.
In a previous post, however, I found that CX questions were correlated with CSI across in different B2B companies at a much higher level (r ~.47). Also, it should be noted that the correlation between CX questions and recommendation intentions are likely driven by common method bias; both metrics use 0 to 10 rating scale, artificially inflating the correlations between them. I suspect that the true correlation between CX satisfaction and real recommending behavior is a lot lower than .53.
Summary and Conclusions
Customer sentiment, as measured by the CSI, appears to be somewhat stable over a long time period. In the current study, customer sentiment toward a vendor accounted for 36% of the variance in customer sentiment toward that vendor a year and a half later. While customers tend to report similar levels of sentiment over time, however, changes in customer sentiment are amendable to differences in satisfaction with the customer experience.
My research on the measurement of attitudes generally points towards the simplification of annual customer surveys. Over this past year, I tried to show the value of shortening annual customer surveys using the CSI. My goal in the next year is to advance this idea of simplifying annual customer surveys by helping leading edge companies improve how they measure attitudes. After all, in this Big Data world, businesses have many different data sources to measure how the business is running. By using operational, financial and operational data in conjunction with basic attitudinal metrics (i.e., CSI), businesses can efficiently capture, analyze and aggregate data in a customer-centric way to optimize value of all their data.