Integrating Text Mining with NPS: The Story Behind the Score at Sage Software

Sage North America, a provider of business management software, has long embraced the notion of customer centricity. This fact is evidenced in our reliance on Net Promoter Score (NPS), a score obtained by asking customers about their likelihood to recommend a product or service. Our product general managers gauge the relationship health of our business by monitoring NPS surveys, as well as other empirical diagnostics. But it wasn’t until the adoption of a broader Customer Experience Management (CEM) program that also leverages text mining that Sage North America became capable of discerning the true voice of our customers.

Drawing from survey verbatims and other text-based customer channels, text mining enables a company to tap into the true feelings of each of its customers.

The recent rise of text mining and CEM at large Fortune 1000 companies speaks volumes about the importance these companies are placing on making their customers happy. While CEM isn’t necessarily a new philosophy, the way that companies are now able to capture and analyze customer opinions on products and services has changed drastically with the introduction of text mining. Drawing from survey verbatims and other text-based customer channels, text mining enables a company to tap into the true feelings of each of its customers. Text mining can automatically convert any text-based feedback from across and beyond the enterprise into timely, actionable business intelligence.

In the case of Sage North America, complementing NPS with text mining-fueled CEM has uncovered the root cause behind the scores our customers give us. This has also given us the ability to engage new, high-value customers early in their relationships with Sage North America, which has resulted in dramatically higher NPS scores. Plus, text mining and CEM have significantly enhanced our ability to address the specific, qualitative comments of dissatisfied customers, “detractors,” and convert them into “promoters.”

A Commitment to Customer Centricity

Sage North America provides customizable business-management software and services for accounting, customer relationship management (CRM), human resources, merchant services and time tracking, as well as specialized solutions for customers in construction, distribution, healthcare, manufacturing, nonprofit and real estate, among other industries. About 2.9 million small and medium-sized businesses rely on our products, and their needs vary depending on applications and industries. We are dedicated to providing superior customer experience to the entire range of our clients, and that requires staying attuned to their unique and changing needs and desires.

This is one of the key goals of our Customer Relationship Program. We strive to contact and survey 200,000 customers quarterly, to ensure we stay in touch with what they need and how they feel about Sage North America’s products.

When the Customer Relationship Program was launched three years ago, it was exclusively quantitative data on which we based our primary metrics for trending, ranking and assessing customer feedback. But the open-ended comments that were captured in the surveys could not be efficiently processed. The general managers for Sage CRM Solutions and our other products had to manually read and categorize this qualitative information. The process was complex and time-consuming, and it also was nearly impossible to accurately link customer loyalty metrics to customer behaviors.

This issue was certainly not unique to Sage North America at the time. In 2006, when we started the Customer Relationship Program, CDC Respond reported that, while 95 percent of surveyed companies collected feedback in some form from their customers, only 35 percent used the data in some way. Furthermore, only about 10 percent made changes in response to the feedback—and just 5 percent told their customers what they had done in response to the suggestions.

At least we were leveraging the NPS scores and other quantitative data that we were collecting. And we always understood the potential value of the qualitative, textual feedback gathered in our surveys; we just didn’t have a good, efficient way to convert that feedback into dependable, timely business intelligence.

Implementing CEM and Text Mining

For years at Sage North America, we’ve felt that data are like garbage—you don’t want to collect it unless you know what to do with it. CEM and text mining are what we now do with the open-ended responses we receive in combination with our customers’ NPS scores.

We implemented Clarabridge’s Content Mining Solution service in the fourth quarter of 2007. Clarabridge’s text mining solution augments NPS by illuminating the “why” behind a customer’s scores.

Text mining can capture and analyze valuable customer intelligence found in various text-based feedback channels and indirect sources of consumer opinions, such as call center notes, qualitative survey feedback, Web 2.0 content, online consumer forums, social networking sites and even customer warranty forms. Aberdeen Group in July 2007 estimated sources of unstructured data such as these to account for 85 percent of all of the data available to a company.

Our text-based customer feedback is automatically categorized, and associated sentiment is determined. The qualitative feedback is integrated with quantitative via a single report system, and this allows us to efficiently rank the issues that customers are talking about in relation to their NPS scores and identify the aspects of our business that cause our customers to recommend us to others. It is a fully automated, end-to-end approach to CEM that enables us to make more intelligent business decisions on behalf of our customers.

We have come to see that the most useful insights from our surveys are to be found in the verbatims from open-ended questions. Consequently, we have overhauled our surveys—cutting back from 29 to five questions, three calling for open-ended responses. Our customers have more freedom to tell us in their own words exactly what they think, and text mining and CEM are the tools that enable us to respond accordingly.

Measurable Impact

Since implementing text mining and CEM, we have incorporated 21 product lines into our survey program. Hundreds of thousands of verbatim responses have been analyzed. The positive impact is far-reaching. The insight we receive from our customers allows us to develop short-term plans to maximize our “wins” and longer-term plans to address the “pain points.”

Because text mining avails our product general managers to real-time customer intelligence, it serves as a valuable, early warning system to identify burgeoning concerns and address issues (such as software bugs and customer service issues) before they reach a broader base of customers. This ability helps us keep small problems from growing—avoiding potential damage to our reputation for quality among customers, as well as the costs of dealing with large-scale support issues.

Not only does our expanded Customer Relationship Program enable Sage North America to make short-term corrections or enhancements, it also provides guidance to our general managers so that they can develop well-founded, longer-term strategic plans. Our product general managers use our “Category Influencer Report” rankings—leveraging the business intelligence harvested via text mining and CEM—to assist them in the development of their annual action plans.

With our new integrated capabilities, the Category Influencer Report automatically ranks customer issues based on volume of comments in a category, NPS score and customer sentiment of all Sage products such as Peachtree by Sage, ACT! by Sage, and Sage SalesLogix. The report illustrates the categories that have the most positive and negative impact on our customers’ experience, and the intelligence can be filtered by key customer metrics.

The insight we receive from our customers allows us to develop short-term plans to maximize our “wins” and longer-term plans to address the “pain points.”

Our general managers can use the Category Influencer Report to see the NPS scores and drill down into the drivers behind the promoter/detractor rankings. As a result, they can call new high-value customers to engage them early in their lifecycle and influence a more positive experience—resulting in NPS scores for those customers that are 300-percent higher than for our entire customer base. Plus, the general managers can follow up with detractors to address their qualitative comments with a quick and focused response. For some of our brands, we have experienced a 30 percent success rate of using the Category Influencer Report to convert detractors to promoters.

Implementing CEM and text mining has enabled us to expand our Customer Relationship Program in a variety of valuable ways. We have been able to identify best practices that could be replicated across all of our brands, and we are now even using NPS to assess our customers’ experience with not only our products but also all of the portals they visit at our web site. Our Customer Relationship Program—now enhanced with CEM and text mining—is growing steadily more integral to how Sage North America operates.

Creating a Competitive Gap

Customer experience is the gap that industry leaders are exploiting to distance themselves from their competition.

For “The Customer Experience Index, 2008,” Forrester asked 4,564 U.S. consumers about the usefulness, ease of use and enjoyment of their experiences with 113 companies across 12 industries. Only 11 percent of the companies received an “excellent” rating among the 4,564 consumers surveyed; 38 percent were rated “poor” or “very poor.” In a March 2008 report, “The Business Impact Of Customer Experience,” Bruce D. Temkin wrote, .”.. Our analysis shows that good customer experience correlates highly to loyalty—especially when it comes to consumers’ plans for making additional purchases. When we examined how this might affect the annual revenue of individual companies, we found that customer experience quality could cause a swing of $242 million for a large bank and $184 million for a large retailer …”

Sage North America is a perfect example of the trend among leading companies to adopt CEM, powered by text mining, in order to improve their customers’ experience, differentiate from their competition and reap the financial rewards. We are at last able to tap into to the true voice of our customers and analyze and act upon all of their feedback.

CEM and text mining have enabled us to finally get to the stories behind our customers’ NPS scores and other quantitative assessments of our business. Sage North America is making better-informed, more customer-centric business decisions on both operational and strategic levels, leading to measurable, positive impact on the bottom line.