If the prognosticators are right, we’ve moved from a manufacturing economy to an experience economy. Today, businesses have to create amazing, memorable experiences, rather than simply delivering reliable products at the right price. But this is more critical than ever in a world awash in instantaneous, high-volume information delivered through every conceivable channel.
While oil fueled the manufacturing economy, data is the fuel that drives the experience economy. But unlike oil, data is increasingly abundant. In some cases, it’s overwhelmingly abundant–leading to the phrase “Big Data”.
Everyone is talking about big data, but too often what they’re really talking about is small data. As Inigo Montoya said in The Princess Bride, “You keep using this word. I do not think it means what you think it means.”
So what does Big Data really mean?
In a nutshell, big data is a catch-all term for data sets that are so large and complex that they necessitate new forms of processing beyond the SQL databases prevalent since the early 1980s. The typical example is a Hadoop stack housing petabytes of “unstructured” data: things like Twitter comments, video content, recordings of call center conversations and other information that isn’t organized in a pre-defined fashion.
To illustrate the scale of this data firehose, Twitter processes 400 million tweets every day. Cisco predicted that by this year, annual traffic flowing over the Internet would reach 667 exabytes annually. That is equivalent to 2.6 million times the amount of information stored in the U.S. Library of Congress. In just five years, the amount of digital information shared globally increased ninefold, to nearly 2 trillion zettabytes in 2011.
If it’s truly big data, on top of volume and variety, it also needs velocity and veracity to round out what IBM calls the Four Vs. That means to be big data, it has to be able to be processed within seconds and it needs to have the trust of the C-Level–what’s the benefit of learning from data within seconds if you’re always second-guessing it?
Marketing analytics does not equal Big Data.
Think about what truly makes data “Big,” and it becomes pretty obvious that most marketers talking about big data are really talking about small data. Marketers live and die by their analytics dashboards. We ask ourselves, “Which keywords are driving traffic?” “How is my latest referral initiative performing right now?” “How many leads did we generate from social this month?” On landing pages alone, think of everything marketers are testing. Should a telephone number be a required field to access certain research? How much text is too distracting? Does an image help increase the number of submissions? Should that banner image have been red instead of green?
These are all great things to do. They’re all questions that every marketer should be able to answer. For years I’ve told marketers to test more. When marketers have a handle on their analytics, it helps them build respect and credibility. That said, they’re all examples of great small data points that feed our analytics activities.
They’re not big data. They are all answered using traditional SQL databases. They don’t require high-variety data sources or unstructured information, and they can’t answer complicated questions quickly.
So what does Big Data look like in marketing?
Some marketing questions are so complex they do require real big data (not marketing analytics small data). Want to break your SQL database? Ask it to pull a list of every consumer who visited your website twice in the past month, opened more than one email, and retweeted your social content. That’s a useful list of people. If they haven’t purchased from you yet, they probably don’t get riper. But this is the kind of list that only big data can pull for you.
Another example is algorithm-based prediction– something many marketers don’t even know is available to them. Want to determine the best next offer to send a prospect in real time, forecast marketing’s effect on revenue in the next quarter, or present different web experiences to different prospects based on their recent social activity? You’re going to need big data.
Use big data correctly– both the term and the thing itself — and you’ll be the marketer that saves the proverbial princess at the end. Don’t, and you’ll be shouting, “inconceivable!” And we all know what the answer to that is.
