One thing that is very difficult to quantify in BI initiatives is the effectiveness of the information produced. Frankly, your key concern should be ascertaining a qualitative understanding of decisions made with BI. To be sure, there’s a dramatic difference between justifying meetings with BI insights and actually taking action as a result of this information. If you crack the nut on measuring that, let us know. In the meantime it’s food for thought.
Financials (revenue, operating margins, shareholder value, etc.).
All other things being equal, an organization with superior analytics ought to exhibit superior financial performance at the level of revenue and profits. Over the long term, the effect of running a more efficient, intelligent, and data-driven business should show up in the bottom line. Has overall corporate performance improved in step with business intelligence initiatives? How well can we judge the contribution of BI to better decisions and strategies? Look for evidence that the BI strategy is making the company more valuable. If the evidence is not there, reconsider both the data and the models used by the BI platform. Remember, Top Performers track whether decision makers are, in fact, using the BI. A system that is not being used will never achieve its intended results.
Employee performance/productivity.
Employees ought to be more productive when they do not have to waste time compiling reports or searching for information that is now readily available through the BI platform. The effect should be measurable, at least for knowledge workers and targeted categories of customer service and operations employees. Certainly, you can measure productivity on specific tasks. Pick a few benchmarks, such as compiling a report that is due at the end of every month, and put a stopwatch on them. Be sure to measure the results before and after BI initiatives. Time spent can be quantified by salary and multiplied by resources involved. This figure is often quite compelling when aggregated globally over a number of years.
Operating margins.
Top Performers are 8 times more likely than Everyone Else to rank operating margins as a top three metric for measuring the success of BI. That’s a staggering difference, and one that required further exploration than the survey revealed on the surface. Discussions with Top Performers revealed BI tools were frequently the only means of aggregating the disparate sources of information necessary for operating margin analysis. Operating margin is a measure of revenue minus variable costs. It demonstrates what is left over for the company to pay fixed costs. The mere fact that BI tools were being used to calculate this metric suggests that Top Performers are capable of correlating operational excellence to the insights derived from BI.
Number of active users.
How many users have accounts on the BI platform, compared with the penetration you would ideally like to see across the organization? How many of them are actually regular users, indicating that they find value in the information provided? How frequently do they access the system?
Agree with all of these, especially building in employee performance/productivity (although would extend this to level of advocacy behavior, because it is proven more monetizing over time), with one caveat. Would incorporate a more action-centric approach to resultant customer behavior: http://customerthink.com/is-there-a-single-most-actionable-contemporary-and-real-world-metric-for-managing-optimizing-and-leveraging-customer-experience-and-behavior/
“. Pick a few benchmarks, such as compiling a report that is due at the end of every month, and put a stopwatch on them.”
I actually think that is a fantastic idea! Don’t ask them for the hardest report ever, but something they should need to run on a semi-regular basis anyway and see how long it takes them to do what you want. If they can’t do it in a reasonable amount of time figure out what went wrong. Was is the employee or the system?