At a local meetup organized by Tatyana Kanzaveli, Esteban Kolsky did his usual masterful job moderating a panel of “sentiment analysis” experts from Attensity, Bing, and Scout Labs.
A few highlights from a excellent discussion…
For those not familiar with the term, sentiment analysis means the use of text mining/analytics to help determine whether written text has a positive or negative tone (hence, sentiment) and why the writer was happy or mad.
Text mining has a long history and lot of complicated algorithms at the core, but in recent years the explosion of social media has ramped up interest. The massive amount of content generated by users on Twitter, blogs, forums, etc. is an excellent opportunity to gain insight, learn about developing problems before they show up on CNN and improve the user experience.
So for example:
- Attensity can help companies enhance Voice of Customer programs by analyzing write-in comments and other text feedback.
- Bing can help users find the best products by analyzing reviews posted on the Web.
- Scott Labs can help monitor the health of a brand by analyzing Tweets and other social sources
But text analysis is not an exact science. Well-trained people might achieve a 90% success rate in categorizing text documents. Good automated methods might hit 70-80%, but it all depends on the task, how the application is trained and tuned, and the methodology being used.
And some text analysis problems are nearly impossible to solve, like understanding sarcasm and metaphors. I guess us humans will continue to have some work to do for a time…
In short, you won’t get the precision that you’re used to with real data, but text analysis is the only effective way to deal with huge volumes. Given the choice between fuzzy insight and no insight, most business leaders will choose the former.
I’ve been following text analytics the past couple of years because it’s a cost-effective way to gain insight from unstructured data. Now social media has made this challenge and opportunity even more compelling.
On a industry note, I think text mining in various forms will become deeply integrated into mainstream CRM applications like customer service, while also continuing to be packaged into focused SaaS-based applications for VoC, social media monitoring and more. The next couple of years should be very interesting to watch as vendors scramble to incorporate and package the technology.
Thanks for covering this topic! Sentiment measurement is definitely in its infancy stage. We help our clients reach additional accuracy by measuring sentiment at the topic level as well as the article level.
Keep in mind that when 2 humans are asked to rate a sentiment of an article, they agree only 85% of the time, so any machine sentiment is never going to be above that.
Cheers!
– Maria
@themaria @biz360
This was a great read and it’s nice to see that the industry is becoming more aware of the importance of sentiment analysis.
At ListenLogic, our social media monitoring tool uses what we call “adaptive sentiment” to look at sentiment. We train our system what is positive or negative about the specific brand in question, and it learns to distribute negative and positive accordingly going forward. This combination of human action and machine learning is what gives us a very high accuracy rate for not only sentiment, but relevancy as well.
Once again, great article, looking forward to more on the topic.
–
Chris
@listenlogic
Ooops, bad statistic. Two humans will agree 79% of the time about sentiment of an article.