There are those of us in the customer experience consulting, training and research community who have, over the years, challenged the claims of granular actionability, if not the logic, of single number performance metrics such as NPS, CES, and CSAT (from ACSI). There have been frequent CustomerThink blogs on the subject. Sometimes this has provoked response (http://customerthink.com/emerging_chinks_and_dents_in_the_universal_application_and_institutionalization_armor_of_popula/), and sometimes there has been silence.
A 2011 article in the International Journal of Market Research, “The NPS and the ACSI: A Critique and An Alternative Metric”, by Professors Robert East and Jennifer Romaniuk of the University of South Australia (http://www.ijmr.com/AboutIJMR/Samples/Sample5.pdf) principally presents weaknesses of both the NPS and ACSI, and offers an alternative measurement approach based on 1) the volume of positive and negative word-of-mouth, 2) the behavioral impact of word-of-mouth, and 3) the effect of given and received word-of-mouth. For everyone wishing to better understand the contemporary and real-world dynamics of customer decision-making dynamics, this is a ‘don’t miss’ piece of analysis and reporting.
In consumer word-of-mouth research conducted in the U.K. from 2006 through 2009 in consumer goods, telecom, retail, and banking/financial services sectors, and following several data gathering protocols, the authors arrived at several very important conclusions. Here is how they summarized the net effect of word-of-mouth, vis-a-vis ACSI and NPS:
“Our evidence suggests that the NPS does not provide adequate measurement of negative word-of-mouth and, because of the way it is restricted to customers of recently used brands, we conclude that the ACSI is similarly poor at capturing dissatisfaction. We also note that the NPS lacks a measurement of the impact of word-of-mouth. The total effect of word-of-mouth should be established by the separate measurement of the volume and impact on purchase of positive word-of-mouth and negative word-of-mouth.”
When East and Romaniuk applied a new approach in three consumer product and retail categories which incorporated incidence and volume of positive and negative word-of-mouth, they determined that “…most negative word of mouth is given by those who are ex-customers or never-customers. By contrast, most positive word-of-mouth is given by current customers. As a consequence, metrics based only on current customers, such as the NPS, do not measure negative word-of-mouth effectively. Detractors give little of the total negative word-of-mouth on the brand and that, in two of the three categories that we studied, detractors were responsible for more positive word-of-mouth than negative word-of-mouth. Similar patterns are found in an analysis based on the ACSI measures, which suggests that the NPS and ACSI are closer than their respective proponents are willing to claim.”
Their conclusion: “We show that the NPS and the ACSI do not measure negative sentiments about brands effectively. We suggest the form of a word-of-mouth metric that could provide measurement of the effect of both positive word-of-mouth and negative word-of mouth on brand performance, and show that this metric differs from the NPS and ACSI.”
Reflective of customer advocacy and bonding research which has been conducted, and improved, for a decade (http://customerthink.com/customer_advocacy_behavior_personal_brand_connection/), the importance, and decision-making leverage, of brand favorability level and incidence/volume of positive and negative word-of-mouth has long been recognized, and used for competitive advantage, outside of academic circles. That said, it’s refreshing to have supportive, corroborative evidence from down under.
