Big Data is not a product or tool. It’s a condition. The human race has wired together so many computers, phones, cars, smart devices, and sensors that we are collectively creating massive piles of data that, if printed out, would cover Long Island in paper.
Most of this data is actually a new form of industrial waste. It’s the secondary byproduct of other primary events that took place. It’s the digital residue of a social interaction, the virtual sawdust made by an online purchase, the electronic tailings from a financial transaction. We’ve digitized so much of our lives that the collective waste is starting to pile up. But instead of causing global warming or algae blooms in the Gulf of Mexico, it’s causing a condition known as “Big Data.”
This is both good news and bad news. The bad news is that data is growing and the curve is starting to get steep. That means you’re going to need big help and big investments to go along with your Big Data. The good news is that, since this isn’t really a new phenomenon, we’ve got decades of experience figuring out what data has to say and we won’t have to invent a whole new branch of mathematics to make use of the data. This is mostly because the real value isn’t in “Big Data” but in gaining “Big Insights,” sometimes from small data.
The Power of Small Data: Big Insights vs. Big Data
If Social Media has taught marketing departments everywhere one thing, it’s this: Power is now in the hands of individuals. The democratizing power of communications technology makes every customer count. Aggregate information is still useful, but we can no longer ignore “small data” shared in a big way.
Similarly, many big insights are gleaned from small amounts of data hidden away inside vast data warehouses. It’s not the size of the data that counts; it’s the value of the insight. You actually can’t put a number to your volume, velocity or variety and decide if it’s “big” or not, because it’s different for every company. For example, Mindshare Technologies helped a multinational auto parts retailer discover that a key factor holding their “passive customers” back from becoming “promoting customers” was the uncomfortable plastic chairs in their waiting area. This wasn’t discovered by analyzing petabytes of data in aggregate but by filtering data effectively and then applying text analytics to the properly filtered data. Improving business results comes from big insights, not from Big Data.
Recommendations for Getting Big Insights
Getting to the big ah-ha Big Data (or small data) can provide is easier than it sounds. There are a few things you can do right now that will yield great results.
1. Begin with the end in mind. Many business leaders make the mistake of thinking they will start with a metric ton of data, apply analytics, and amazing previously unknown insights will randomly start popping out. While it’s possible to use pattern-matching techniques to find random things, it’s much simpler and more reliable to take a more targeted journey through your data. If you know which kinds of things you are trying to learn, it’s easier to spot holes in your data, pick proper analytics techniques, and filter data down to a reasonable size.
2. Add context. Most big discoveries don’t happen in a vacuum. Adding related data makes it much easier to spot highly correlated data, explain anomalies, and recognize patterns. For example, while analyzing text analytics for hidden gems, the Mindshare analytics team frequently uses structured survey data and transactional data to act as a road map into the data.
3. Leverage text analytics. Many Big Data scenarios, such as social analytics, involve unstructured textual data. For other scenarios, text data can provide the “why behind the what.” It’s often not enough to see the pattern of data. Explaining the pattern can be difficult without the backdrop of context provided by text.
4. Rely on proven analytical techniques. Just because you’re using Big Data, don’t think you have to learn new analytical techniques. Use simple linear regression to spot correlations and uncover key drivers. Use logistic regression to predict categories and clustering and correlation to understand groups.
5. Experiment with machine learning to make predictions. Machine learning techniques—like decision trees, neural networks, and Bayesian classifiers—are easier than ever to use thanks to the plethora of information on the Web and free and open-source tools. Machine learning can make predicting data sets much easier for analysis, such as fraud detection and pricing analysis. Just be careful that you depend on results only when you understand what you are doing.
6. Visualize everything. Visualizations make your results much easier to share. They also lend themselves to telling a story. Narratives always play better than a boring wall of data. Perhaps more importantly, certain insights are much easier to spot visually than they are to spot numerically. For example, Mindshare recently visualized customer satisfaction data for a major quick serve restaurant chain by creating a color-coded, stylized calendar. We quickly saw that customers were having a poorer experience every Wednesday afternoon. This conclusion was easy to see visually but would have required heavy mathematical lifting to discover using numerical analysis.
Big Finish
Big Data isn’t necessarily what you think it is. The big insights you hear about all the time don’t have to come from Web-scale mega feeds. You’re sitting on a gold mine of data right now that can help make critical business decisions. Using the right tools, you can explain what has happened, discover why it happened, predict when it might happen again, and decide what to do about it.
