Making predictions beyond real-time analytics


Share on LinkedIn

Recently it was announced that a collaboration between Microsoft and the Technion-Israel Institute of Technology created software that predicts future events using a combination of archived news material from the New York Times, real-time data and information from a variety of web sources like Wikipedia.

In their research paper they focused on predicting disease outbreaks, riots and other public events and claimed their accuracy was between 70%-90%. They claim to be able to see links between droughts and storms in parts of Africa and cholera outbreaks from these disparate sources of new and old data. While previous research has been done in this area (see excellent Wired article on the subject from 2011) none have focused on using the data to try to look ahead and forecast future events before they happen, they said.

“I truly view this as a foreshadowing of what’s to come,” said Eric Horvitz, Codirector at Microsoft Research. “Eventually this kind of work will start to have an influence on how things go for people.”

But there’s more at stake here than just predicting the next public disturbance or disease outbreak, this has rather interesting commercial implications on a number of levels.

The rise of the digital fortune teller

As I wrote last year, you’ll soon be able to ask for an entire dump of what every social site holds on you into a cloud-based repository. Obviously what you’ve put into these social sites, loyalty cards, location-based apps, etc., is what you expect to receive back but the sheer amount of data processed, correlated and extrapolated will vastly outweigh what you put in in the first place. The sites hosting this information have all made use of it first.

A quick Google search will reveal a small advert for digital fortune telling, an offering to reprocess all that data so you, yes YOU, can make smart use of it. You will want the power that all those sites and companies held over you for so long. You want to be able to know your future before you make it. And they can offer you the chance to predict your future with it the same way the Microsoft and Technion team have just demonstrated. Will you cross their palm with silver to find out?

Real-time was yesterday’s dream

Predictive analytics and statistical modeling is nothing new in the enterprise, they are used every day in businesses like insurance and financial services. But they are used to drive commercial decisions based on a product level, for example targeted consumer marketing using data mostly held within the company. Once you begin to mine and correlate information from external sources at the same level shown in the research paper you have the ability to factor future events that may completely alter the behavior of the consumer, allowing the business to prepare to scale on a future demand that snaps into play with little to no warning. Reactive decision making will be a thing of the past. Real-time and Big Data seem like yesterday’s dream already.

The real power lies in making business decisions that are outside of the consumer insight sphere. Would you trust making one based on a future event prediction and bet your company budget on it ?

Can you imagine if Oreo had already predicted the blackout and instead of having a command center within their digital agency partner ready to react to any event as it unfolded they had already prepared for it because they predicted it?

I predict many will baulk at this idea. But the reality for us all may soon be very different.

Republished with author's permission from original post.

Theo Priestley
Theo Priestley is Vice President and Chief Evangelist at Software AG, responsible for enabling the marketing and voice of the industry's leading Business Process, Big Data/ In-Memory/ Complex Event Processing, Integration and Transaction suite of platforms. Theo writes for several technology and business related sites including his own successful blog IT Redux. When he isn't evangelizing he's playing videogames, collecting comics and takes the odd photo now and then. Theo was previously an independent industry analyst and successful enterprise transformation consultant.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here