Aloha!
Yes, I’m in Hawaii and I’m not supposed to be working. But after a full day of giant water-sliding and resorting, it’s a relief to sit down and exercise my brain. That being said, I plan to keep this fairly short – I thought a quick overview of some fascinating work Jesse Gross and I just delivered to one our clients last week, the core of which is also part of one of my upcoming Think Tank classes at the is year’s X Change.
The class is all of a piece with my recent blogs and covers Customer Experience Analytics. It includes an overview of Customer Experience engineering and an introduction into how our analytics methods (everything from Functionalism to VoC Customer Intelligence Systems) fit inside the broader concept.
For many sites (perhaps most), measuring the actual value of Website visits is challenging. You don’t have a shopping cart and checkout to rely on. You can measure various milestones; you can measure engagement. But how do you measure impact?
This is probably the single most commonly asked question in digital marketing. It’s profound, important, and it’s hard. There is no way, using straightforward behavioral analytics to address the question of impact. Worse, if you can’t measure impact accurately, the optimization efforts you do make are more like guesses. I’ve seen countless measures of engagement. What unites nearly all of them is that they are simply subjective guesses about what actions might be proxies for success.
That’s not good.
Our goal in the research we did was to answer ALL of the following questions with data not with guesses:
- What is the value of each visit type to the Website
- What is the value of improving satisfaction within each visit type
- What are the key site behaviors that are associated with increased value
- What is the incremental lift of those behaviors
Here’s how we did it.
Step #1 was to establish our visit intent based segments (Use-Cases or Tier 2 of our Segmentation). We did this with a four-step methodology using an integrated onsite survey.
We start with the basic “reason for visit” question in an online survey. We assume that this is nearly always accurate, but we know that we can only survey a tiny percentage of total site visitors. However, the survey responders give us a data set with a target data variable we can use in Step #2. Here, we create behavioral signatures that accurately predict the “reason for visit” variable taken from the survey. In Step #3, we apply these signatures to the entire population. In Step #4, we validate them against new survey takers to measure the accuracy of the behavioral prediction. We know we’re not going to be able to accurately classify every visit type – but the more visits we can classify and the more accurate our classification, the less noise there will be in every subsequent measurement.
With this approach, we now have a data-driven and validated segmentation of every visit type to the Website. So far, this is our standard Use-Case / Two-Tiered Segmentation methodology.
Now here’s the cool new stuff.
We take our initial online survey population and re-survey after 3-6 months using email. This second survey gives us a look at subsequent behavior. In the re-survey, we focus on two types of questions: outcome questions (did the customer buy, from whom, how much did they spend) and driver questions (why product / brand choice).
This second survey technique certainly opens up a certain amount of sample bias. I have no doubt that we’re over-surveying for engaged and brand committed users. However, that sample bias is less critical for two aspects of the work that are absolutely unique.
First, with this second survey in hand, we have data with which to measure the differential value of each visit type. We’ve classified every visitor into a visit type. We’ve surveyed their subsequent buying behavior. This gives a direct answer on the relative value of visitors within each visit type. It’s important to read that correctly and note how I phrased it. We haven’t established that the Website is responsible for that value. We’ve simply identified how valuable the average visitor in each use-case actually is. That’s not everything, but it is important. If you’re thinking about where to invest your testing or site improvement resources, wouldn’t it be nice to know what the most valuable customers to your site are looking at? Sure it would.
This gives us the answer to the first question:
- What is the value of each visit type to the Website
Now for question #2 – the value of satisfaction. From our initial survey, we have visit satisfaction. By correlating satisfaction to actual (self-reported) spend, we now have a measure of the value of satisfaction. This isn’t a perfect incremental measure. But it has some virtues – not the least of which is that it can be used to compare the impact of satisfaction improvements by use-case / visit type. It also establishes a true baseline from which we can eventually develop a complete and true answer of lift. Because as we improve (or lose) satisfaction in a use-case, we can now measure the actual increase or decline in subsequent spending.
If you’ve ever been asked (and I’ll bet you have) what is the value of increasing satisfaction, this is a powerful data-driven process for generating a good answer.
I want to emphasize how powerful this is. It’s not just about answering questions it’s about driving fundamental decisions about resource and effort allocation. Knowing that a 1% point improvement in satisfaction for Visit Type X will generate $2.60 per visitor in incremental revenue and that a 1% improvement in satisfaction for Visit Type Y will generate $5.25 per visitor in incremental revenue is every bit as important as knowing how many visitors traverse each use-case. When you know both those things (as our client now does), you can make data-driven decisions about where change will have the biggest impact.
We’ve now answered these two questions:
- What is the value of each visit type to the Website
- What is the value of improving satisfaction within each visit type
The next step is the one that I think is both the coolest and most important of the whole bunch. In this step, we looked at which behaviors in the site visit were associated with higher than average revenue in the use-case. In a Product Browsing visit-type, for example, we were able to measure exactly which site behaviors were correlated with successful outcomes. Did looking at multiple product detail pages correlate to higher value? What about exiting on a product detail page? Spending lots of time on a Product Detail Page? We were able, in short, to look at dozens of different potential measures of engagement for each specific type of visit and USE THE DATA to decide which ones worked.
From these variables, we built a predictive model to assess the actual dollar value of any given visitor to the Website with any given set of behaviors.
As always, this doesn’t prove the behaviors we identified drove success. However, it does prove that they can be used as proxies for success. In other words, if we want to optimize our digital marketing spend (which was a key goal of this process) we now have a fully data driven way to assess the ACTUAL DOLLAR VALUE of any given visitor to the site. Think about that. It’s huge. This isn’t some subjective proxy. It’s not a guess. It’s a hard, data-driven dollar value. It can be compared. It can be added. And we have every reason to believe that it’s right.
That makes it incredibly useful as an optimization measure – far more valuable than any type of ad hoc engagement score created by some witches brew of subjective addition.
We’ve now answered the first three questions:
- What is the value of each visit type to the Website
- What is the value of improving satisfaction within each visit type
- What are the key site behaviors that are associated with increased value
The last question (the incremental lift of the predictive behaviors in our model) isn’t readily answerable from data alone. However, with the work we’ve done and the ongoing re-survey methodology, we’re now in a position to definitively answer the lift question too. All it takes is some testing.
Once all the potentially interesting behavioral factors have been identified, we can target testing programs specifically to create the “valuable” site activities. By measuring the increment in the demonstrated behavior and the impact on subsequent spend, we have the ability to prove actual lift.
Here we’re at the level of causation not correlation. And with this in place, we’ve answered all four of our original questions:
- What is the value of each visit type to the Website
- What is the value of improving satisfaction within each visit type
- What are the key site behaviors that are associated with increased value
- What is the incremental lift of those behaviors
This process ties together much of the best practice we’ve developed at Semphonic and EY. It combines Two-Tiered Segmentation (behavioral) with a Customer Intelligence System. It reaps the rich fruits of combining powerful behavioral and VoC techniques into a single analytic system. It uses data-driven methods to answer critical Website questions with a high-degree of confidence and accuracy. And it produces a framework for driving every aspect of Website performance – from digital marketing optimization to targeting site improvements.
Truly, we are talking analytics paradise here. It’s as cool (and as hot) as Hawaii!
To get the full deep-dive, come out to X Change and join me at Think Tank as we take a hard look at the methods I’ve overviewed here and the arduous but oh-so-worthwhile path from data to experience to value.
