The vast majority of enterprise reporting and dashboarding is confined to “showing the current state“. We spend a huge amount of time showing people what happened – virtually none showing what’s likely to happen or what’s driving those numbers. We’re like weathermen who only report the temperature – never the forecast or the factors explaining that forecast. It just isn’t that useful. By embedding models into reporting, we have the opportunity to fundamentally transform them: from static snapshots that serve as little more than “Warning” signs to real business tools that help users understand what levers they have to drive performance and the implications of adjusting those levers. Embedding models inside our dashboards also creates a powerful discipline – connecting analysis and reporting – wherein every time our forecast is wrong we’re driven to go back and figure out why, continually refining our understanding of what drives the business.
It sounds great (I hope), but leaves open the question – how do you do it? How do you build models of complex digital marketing or operational systems?
There are actually quite a few different approaches – and it’s not as hard as it might seem – particularly to create an initial forecast that can be your entry-way into that virtuous cycle of continuous forecasting, analyzing, and tuning.
One technique that we’ve been experimenting with is simulation.
Why Simulation?
We’ve built models for many aspects of digital marketing: attribution, mix, internal optimization, use-cases, conversion funnels, onboarding and more.
Historically, none of these models have been simulation models. We’ve used a variety of techniques for these models – from multivariate regression to factor analysis to cluster analysis (at least to drive underlying segmentations).
So why add simulation to the mix?
Well, here are some examples from a simulation design program that we’ve created in a couple different problem spaces.
Here’s a high-level piece of one of our early, exploratory simulation models for a Product Launch:
And an even smaller piece of a broader simulation of a digital marketing system:
These aren’t just diagrams. The simulation software they are built in allows you to define the stocks, flows, and parameters that make up the system and then use the simulation engine to see how changes will impact the whole system.
What do you gain by using simulation around these problems as opposed to our traditional models? The beauty of simulation is that it allows you to combine two or more very different models into a single system. For example, both our Product Launch model and our Digital Marketing System model require elements of campaign optimization – particularly mix modeling. Those elements are built using traditional techniques. But there’s more to a Product Launch than optimizing your media spend and there is more to a digital marketing system than campaigns. In both cases, representing the Site System is also critical.
Now here’s the really interesting point – we use completely different techniques (segmentation and use-cases) to analyze the Site System. Those techniques are NOTHING like the techniques we use for campaign optimization. What’s more, there’s no way (at least as far as we’ve ever been able to figure out) to combine the two analytic techniques into a single method.
If you want to create a comprehensive dashboard of the digital system, however, you can’t just create models of campaigns and site and the leave them as independent entities. If you do, then you’ll miss the deep connections between them. As you adjust your marketing mix between online and offline or between display and PPC, you don’t just change the amount of traffic to the site; you change the mix of traffic to the site and the distribution of visit types. This means that campaigns and site systems need to be related to create an accurate view of the larger system.
Simulation is the only way I know of to accomplish that.
Challenges
It isn’t all hunky-dory though. First, simulation is a completely new discipline for us. We’re working to build out this capability, but I won’t pretend it’s easy. Building simulations isn’t much like building regression models (though there are some similarities), and the results of some of my initial simulations were downright puzzling.
Maybe that’s a good thing, though. Simulations create a real discipline in that, if you don’t have a reasonable working model of the world, the chances are high that your simulation won’t produce results anything like the real-world. Building a simulation forces you to tackle reality head-on and make sure you’re actually capturing most of the important elements of the system.
There’s a second problem with simulation that I’m less sanguine about. The images from above are directly from simulation software. They have a gee-whiz quality to them that I must admit I like. Particularly becuase they help capture something I think is vitally important in reporting – showing the interconnections between factors. But they most certainly are NOT the stuff of an executive dashboard. When you’ve created and tested your simulation, you need to instantiate it in a real tool. There’s no obvious tool for doing that – so grafting your simulation into a dashboard is very much a technical exercise.
Summing Up
Simulation combines traditional analysis (to drive the parameters of key systems) and the inter-relationships of complex independent systems, to create models that transform dashboards into working tools. They provide a way to forecast the future, identify the drivers of performance, track how actual performance deviated from expected, and give decision-makers the ability to try what-if scenarios with much higher real-world fidelity. It’s cool stuff. They change your whole approach to reporting and dashboarding and replace static views with powerful tools for running the business.
Afterword
I’m speaking at the AIM Conference on Monday in Los Angeles – on how to measure beauty (or the Science of Aesthetics as my PPT is titled). I won’t pretend to have an answer, but it’s certainly an interesting topic. I’m also very excited to be presenting out at VoC Fusion in a few weeks. It looks like a terrific conference. I’ve been building that presentation and it’s a really nice overview of my recent posts and work on creating a comprehensive Customer Intelligence System as well an early distillation of our integration with E&Y’s existing Customer Experience practice. It represents the first fruits of what I hope will soon be a very rich garden. Not only am I much pleased with the presentation, it’s great to be presenting outside the traditional analytics community. Really looking forward to it!
