Many companies are drowning in data from many different sources, and BI tools such as Power BI, Tableau or Qlik are making it easier than ever to draw actionable insights from all this information. But how can you be sure that the data you’re inputting into your BI tool is consistent? Data management tools are stepping into the void, collecting and integrating data so that data teams can make more sense of the explosion of data.
Like oil before it, data has become the world’s most valuable resource. The titans of this industry, Alphabet, Amazon, Apple, Facebook and Microsoft, are amongst the five most valuable companies in the world. Google and Facebook alone accounted for almost all the revenue growth in digital advertising in America in 2016. And data – as current events have highlighted – is at the heart of it all.
How did we get here?
Whether you realise it or not, almost every activity you undertake, whether you’re going for a run, listening to music or reading the news, creates a digital footprint which can be used to learn more about you, your lifestyle and buying habits. With the Internet of Things growing almost by the day, the amount of data we produce will only increase. With the right insight, this information can be used to great effect, not just by tech giants, but by all manner of companies. People freely give up this data because of the perceived benefit – algorithms can can be used to show you all manner of relevant stuff: recipes that contain the ideal post-run ingredients, new bands you might like and news sources you’d be interested in.
The issue for many companies now is not how to collect data, but how to ensure its consistency in order to use it effectively. Within one mid-sized company, there could be half a dozen different departments collecting and sorting data on their customers. Some departments will be storing it within a CRM, others on a spreadsheet. Some will be collecting reams of data on each customer, other will only have a name and email address. More confusingly, different departments will have different definitions as to what a customer is. To the marketing department, a customer is someone who responds to a campaign; to the support team, it’s someone to whom they provide support; in finance, a customer is someone who has paid their bill.
None of them are wrong, but equally none of them are right.
With all these inconsistencies in collection, it’s almost impossible to provide the kind of accurate insights that can drive your business, even with a BI system like Qlik or Tableau. These systems wouldn’t be able to tell if Jon Smith at 11 Chancey Lane, Warwickshire was the same or different to Jon Smith, 11 Chancy Ln, Warwickshire or Jonathan Smith, 11 Chancey Lane, Warks.
From Fragmented to Managed
In order to turn this fragmented information into a data set that can be read by your BI tool of choice, businesses are developing data management strategies – often involving data management tools that sit between their data and their BI tools. That is, a software system able to automatically connect to, collect and integrate all data from all sources, before securing it and preparing it for analysis. As Bill Tennant, VP of Sales at Zap says:
“This may not be seen as the most exciting area of business intelligence by some, but it’s clearly a must.”
As an example, I once worked in the marketing department of a mental health care group. Our key customer database was made up of NHS mental healthcare professionals. Collection and maintenance of this data was left to a combination of our regional sales teams and the staff at each of our 50+ facilities.
A common issue we encountered, aside from our mental healthcare professionals having better things to do, was that many of our facilities shared the same local NHS Trust, meaning that when the information was pulled together we would have a lot of duplicated data, often with minor differences.
There was also the way the data was collected. The sales teams prefered to use Microsoft Dynamics while the facilities staff would use Excel spreadsheets. Over time, this information became less and less valuable, culminating in the marketing and sales people not knowing where they should be targeting their efforts. This was a company with a valuation of close to a billion pounds who didn’t know where their customers were.
Ultimately, it required me to spend approximately 30 hours on the phone, updating details as I went. Time that could have been spent actually marketing our service, or delivering some kind of insight derived from the data – rather than managing it myself.
With a data management tool, this information could have been collated, sorted and ready for analysis within minutes. We would have had actionable data and the sales and marketing teams could have got on with making decisions based on accurate data, rather than trying to call people who had left the service 6 months prior.
Collecting data is easy: so long as you are collecting it in a compliant manner and you have robust data governance in place..Tools like Tableau and Power BI provide real-time analysis of the data, giving you the most important metrics all in one place. The difficulty is ensuring the data you’re putting into your BI software is accurate and consistent. Without clean, consistent and harmonized data, errors creep in, time is wasted, and ultimately, you may struggle to compete in today’s economy.
