Andy Beaumont recently wrote an article, The Value of Content, which is (at its core) a screed against the annoying web design trend of making readers click through overlays in order to get to the actual page they want to read. Beaumont struck a nerve with his “Tab Closed; Didn’t Read” collection of annoying examples.
Beaumont’s article is an excellent argument for all the reasons against this technique, and one point in particular struck me. In responding to the argument that overlays get used because they work, he writes:
This is what happens when analytics make decisions for you…Analytics will tell you that you got more “conversions”. Analytics will show you rising graphs and bigger numbers. You will show these to your boss or your client. They will falsely conclude that people love these modal overlays.
But they don’t. Nobody likes them. Conversions are not people. If you want the whole story here you should also be sat in a room testing this modal overlay with real people. Ask them questions:
- “Do you like that overlay asking you to sign up for the newsletter?”
- “Do you understand what will happen if you do sign up for it?”
- “Do you know that there is content behind it?”
- “Do you know how to close it to get to the content?”.
This gets to the heart of the difference between customer feedback and other metrics and analytics: there are two sides to every story, and if you’re not collecting customer feedback then you’re only getting one side of the story. In this case, the overlay may be effective at getting newsletter signups, Facebook likes, etc., but not at all effective in generating actual customer engagement or useful sales leads.
The key is to recognize that the signups and likes are not, in themselves, the true business goal. The true business goal is the customer engagement or the sales lead. And it’s important to recognize that the reason behind the signups and the likes is more important than the actual signup or like.
Put another way, if someone signs up for your newsletter because they want to receive your newsletter, there’s a good chance that’s a useful sales lead. But if someone signs up for your newsletter because they thought it was the only way to close an annoying window and get to the article they wanted to read, that’s just an annoyed web surfer.
We see this same thing play out all the time in the customer service world when companies get overly focused on a particular set of metrics and forget to ask what’s really going on for the customer. The classic is using containment to measure how well an automated customer service system is working, and assuming that anyone who hangs up successfully self-served.
We even see this problem when it comes to customer feedback surveys. Many companies track their survey scores and assume that if the number is going up things are getting better, and vice-versa. But it’s also possible that employees are gaming the survey, some customers are being blocked from taking the survey, or customers’ expectations are changing.
The lesson here is that any given metric is, at best, an approximation for the real business goal. It’s important to always keep that in mind, and constantly ask not just “what” is happening but also “why.” And the most powerful tool for doing this is using a truly closed-loop process, where you close the loop with the customer, the business, and also your metrics and feedback.
