Last week I gave a talk on “The Future of Text Analytics” at the Text Analytics Summit in San Jose, California. And while futurecasting is usually a subject for December blog reflections, I figured since Starbucks has already rolled out the Red Cups, it would be appropriate to share with you some of my thoughts on the subject in November.
One of the major trends in text analytics that has become apparent is that practical applications are driving a lot of the innovation. I think digital marketing expert Mike Moran said it best in 2008: “The future of text analytics is that the kinds of applications that are newsworthy now will become mainstream in the future.”
And there are lots of applications out there. Just a look at the agenda for the Text Analytics Summit gives a pretty good snapshot. From forecasting markets to policing social networks, to routing social media communications, to voice of the customer analytics and beyond – it seems if you can dream up an application, you can dream up a way to make that application better through the use of text analytics.
Of course, all of this innovation also brings new challenges on the technical and business fronts. For example, the growth of social media as a source for analysis has resulted in a two-fold challenge: managing the costs of ingesting and processing all of that data, as well as developing new ways to make sense of it. Interpreting “social speak” not only means making sense of abbreviations, emoticons and phonetic spelling, it also means you need to be aware of recent cultural events. A sentence like “I feel Kardashianed about Klout” requires the reader to have a knowledge of both Klout and Kim Kardashian’s recent short marriage – IE, this translates to “I’m SO OVER Klout”.
The growing globalization of business brings another challenge. As business analytics consultant Meta Brown said, “The pressure is on to analyze text written in languages that the analyst cannot read.” For a company headquartered in the U.S., but with customers who speak many different languages, this means you have several options. Do you translate everything and then process it in English? Do you process text in its native language and then present the results in English? Or do you hire analysts in each country that do pure native language analysis and then roll up that analysis in a BI application that synthesizes the fields?
Plus, as text analytics applications expand to more companies and more users, there is a lot more room for misinterpretation as well as more need for customer education. For example, it used to be that customers looking for “sentiment analysis” were looking for simple things like “I hate your product” (negative) or “I love your product” (positive). But now, you’ve got customers who want “sentiment scores” (“I love” = +3, “I like” = +1, “I kinda like” = ???). And you’ve got customers who define sentiment not in terms of positive and negative emotions, but positive and negative events – for example “I sold my LR3 and bought a Leaf” might be called “positive” event for Nissan, and a “negative” event for Land Rover.
Perhaps the most interesting challenge for me personally lies in applying text analytics to routing and other “process-based” applications. Whereas getting it 90% right might be ok for some reporting applications, when you’re looking at passing individual messages to a contact center agent or other process, you need far greater accuracy. This includes balancing recall vs. precision (would you rather deal with more “noise” or “never miss a tweet”?) and understanding language nuances (there is a big difference between “I think you should close your account”, “I’m closing my account if someone doesn’t call me back within an hour”, and “I closed my account yesterday”).
What do you think are the challenges in the year ahead? Will applications continue to drive innovation, or will pure play vendors lead the way? What corporate goals do you have for 2012 in which text analytics can play a part?
