This morning, I’m sitting in a series of presentations extolling the value of big data. I get it–kind of. I get that more data has been created in the past 2 years than in the history of mankind. I get that data is everywhere, we can know so much about so many different things. I get there are very powerful tools, enabling us to gather disparate types of data from thousands of sources, slicing and dicing it in ways previously unimaginable.
I think if I hear one more statistic, hear any more testimonials about the power of big data, I’ll throw up.
I wonder though, why don’t we hear presentations or talks about “big questions.” Without big questions, big data is nothing more than billions of 1?s and 0?s. Big data actually isn’t powerful, it’s the big questions that make the big data powerful. But we don’t talk about the big questions. We don’t have workshops discussing things like, “What insight are we trying to get? Why is it important? What are we trying to model? How are will we validate the models and it’s meaning? What do we intend to do with the answers once we get them? How do we trust the answers we get? How do we discern the garbage?” The list of questions can go on.
Big data can’t give us big answers or great insight unless we are modeling creating powerful questions. Big data can give us great insight and fantastic answers. Likewise, big data can point us in the wrong direction causing us to do terribly stupid things. Ask bas questions, do analysis on flawed assumptions and premises, big data will always give you an answer–but it could be a stupid answer.
Many years ago, I was part of the founding team of a “big data company.” We had a fantastic analytic tool, it could provide all sorts of fantastic insights and answers. It was really a breakthrough technology. Our sales people and modelers could demo the system to customers, giving them insights they had never seen before. Customers couldn’t provide purchase orders fast enough. We shipped the product to the initial customers. Anxiously. we monitored the results. Pretty soon the complaints and questions came flowing in. See the problem was, customers couldn’t come up with the important questions or problems they wanted to solve. Actually, they could, they had the high level questions, but they couldn’t express them and model them in ways that would produce meaningful results. They didn’t have the skills, analytic capabilities, or tools to leverage the power of our analytic engine.
We ended up having to “ship” analysts and modelers with each installation–helping customer construct the questions, build the models, develop the big questions.
So I get the power of big data, I get the potential of the tools. I don’t need to hear a about this any more. What I really want to learn about is the big questions. How do we develop them? How do we model them? How do we interpret the results? What do we do with the results?
I want to hear someone say, “We wanted to learn these things about our customers, prospects, and markets—this is why we wanted to learn these things—this is how we tested our models–these are the alternative models/questions we considered—this is what we are doing with the insights we got.
Technology and tools make big data relatively easy. But they don’t help a whole lot with the big questions. That’s where the work is, that’s what drives the insights, that’s what makes big data valuable.
