I started off this year with Luther-like epistle on “Measurement that Matters.” It was a small (much smaller than Luther’s) set of theses about how enterprise measurement can be made to matter. The projects I described were chosen because they force or are strongly tied to driving actual use of data in the business. Why? Because far too often measurement is something that the enterprise DOES but doesn’t USE.
In upcoming posts, I’m going to tackle those projects in more detail and show why I think they are powerful, how they can be done, and why, when put together, they create a transformative measurement program. But before all that, I wanted to take a quick look at the opposite end of the equation – why enterprise measurement is so often divorced from change or action.
Setting Yourself up for Failure
In a large enterprise, no single factor may be more important in your long-term success or failure than how you’re organized. Madison’s famous dictum: “If men were angels, there would be no need for government” is apropos in the private sector as well. Good people will often transcend organization. If you happen to have a great measurement lead, you may have success no matter how poorly organized you are. But individual successes are rarely sustained in the large enterprise, if only because success breeds promotion (or, more likely these days, poaching). So over time, it’s hard to sustain a level of human excellence in a large organization that is much above average. Running a large enterprise is fundamentally an exercise in generating good outcomes with average people – a task utterly different from what’s demanded in small, entrepreneurial settings.
Unfortunately, digital measurement has not been well structured in most large enterprises today. Here are the most common and most serious mistakes in organizing enterprise measurement teams:
Failure to Integrate Measurement into Teams: Good measurement can’t be distant from teams if it’s actually going to drive change. Transformation has to come from within since only hands-on workers have the expertise to take advantage of data. This is particularly true in the digital world where information changes rapidly.
Failure to Isolate Measurement from Teams: This isn’t a misprint and it isn’t a contradiction. Measurement has both an internal and an external function. The internal function has to support day-to-day operational change and customer conversations. The external function has to provide senior managers the ability to channel resources appropriately and make broader organizational investment decisions. These are two completely different functions and they MUST be treated separately if your organization is to work. Centralize both functions and no one will use information. Distribute both functions, and you might get measurement use, but you’ll NEVER know what’s effective or where to invest.
Think about it this way, decentralize and embed analytics to drive customer conversations or process; centralize and make analytics independent to drive external reporting, audit and investment functions. Nearly every enterprise I know blends these two functions.
It’s a big mistake.
Here’s a related organizational issue that can just about cripple the entire digital measurement effort: separating testing from analytics. About half the enterprises I see have completely isolated site testing from site measurement. Most (but not all) of the time, these enterprises have a vendor managing their testing. This makes my head spin. What exactly is your analytics team for if not to create test plans for the Website? Talk about measurement without use! If your site tests aren’t being driven by your measurement team, something has to change. If your internal measurement team can’t think up good, data-driven tests, find a new measurement team. If they can, and you won’t let them, re-organize your testing.
I can’t pass onto the next topic without mentioning the single worst mistake you can make as an enterprise – letting your creative agencies self-measure.
Not only are most digital agencies pitifully bad at measurement and analytics, they have a strong self-interested bias in producing “win” oriented measurement. If you think you can optimize your business spend by asking people to self-assess and then paying them based on their own assessment, by all means have your agency measure the channel they own. Paying your digital agency for measurement is like funding their marketing department. Stop the madness.
Hiring all the Wrong People
If your digital measurement department is filled with…well…digital measurement people, then you’ve probably missed the boat. There’s tremendous demand for analysts skilled in tools like SiteCatalyst. There’s nothing wrong with that. It’s a good and useful skill. On the other hand, these folks are too expensive right now AND too limited to be a complete solution. Useful digital measurement requires more. In fact, it requires a blend of at least four different functions: analysts skilled at data manipulation, traditional customer analytics and targeting (usually in SAS), analysts focused on information formatting and delivery (Excel, Tableau, etc.), analysts focused on Voice of Customer and traditional customer research, and analysts focused on digital/site analysis. If you’re missing any of these skills or you’ve segregated these teams, then you’re team or team structure is probably a limiting factor on success. The very best departments I see are thriving because of the creative blend of traditional customer analytics folks with digital marketers. If enterprises were just a little more aggressive in blending in opinion research folks, they might really have something special.
Skipping the Foundation
Measurement is all about use. So the drive to create product is natural and I’m sympathetic to it. But there’s democracy and then there’s anarchy. The difference between the American Revolution and the Arab Spring is all about having the right foundation for change. Before you start building reports and democratizing data, there’s a foundational exercise you simply MUST do.
The single biggest challenge in digital reporting is that your marketing stakeholders almost certainly don’t have a strong understanding of how to measure success in the digital channel. In some cases (like purchasing products) it’s fairly straightforward. But comprehensive digital measurement for the large enterprise will demand success measurement across a wide-variety of digital touch-points and functions: from ecommerce to branding to customer support. You can’t leave decisions about how do this measurement up to individual managers or business units. If you do, you’ll get metrics that simply don’t capture the necessary business success or capture only some piece of it.
From page views, to time-on-site, to site-wide satisfaction or NPS, the history of digital measurement is littered with failed paradigms. Don’t be a measurement Marxist! Failure here has deep ramifications for the whole of your measurement and marketing effort. Bad success metrics mean bad optimization. It’s that simple. Getting this right is a demanding exercise that requires business acumen, deep understanding of the digital channel, good methodologies, and quite a bit of tool knowledge. It also takes a significant up-front effort before you start churning out reports. Do it.
Accepting the Status Quo
If you’re a senior manager in digital or a C-level wondering why your company doesn’t seem to be winning the data revolution, here’s one very good place to start: a mirror. I’m not sympathetic to the tendency among analysts to disparage real decision-makers. I loathe the term HIPPO (derogatory slang in our industry for Highest Paid Person’s Opinion). Making decisions with data is far harder than people think.
But here’s my advice to senior folks frustrated with the (f)utility of their enterprise measurement. Don’t accept the status quo. Don’t accept measurement that siloes everything. Don’t accept reports that tell you what’s happened but provide no insight into why or what to do. Don’t accept analytics that are divorced from every possible channel of action. Don’t be intimidated by the buzzwords, the data-blizzard, the fancy reports, or the digital slang. If it’s not helping to make decisions, it isn’t any good.
Far too often, decision makers fail analytics by failing to demand enough.
So there it is – a whirlwind tour of the biggest failure points in digital measurement with a few side-notes from Luther, Madison and Marx to keep you on your toes. I’m sure that’s enough philosophy to last a good long while – next up a far more practical and prosaic deep-dive into the six projects that make up my measurement manifesto.
