I had the opportunity to catch up with Robert Raines, VP of Product Management & Creative Services at E-LOAN last week. I first met Robert several years ago when he was getting the company’s email programs off the ground. Robert shared with me some of the things that he’s accomplished since we last spoke and, as always, he had some great insights.
E-LOAN wanted to evaluate the impact of its email program
Robert is a firm believer in testing and, more specifically, using control groups to measure the impact of email marketing activities. To determine the long-term benefit of its email program, the E-LOAN team created a randomly selected universal control group. While the control group still received transactional email communications (e.g., “We have received your application”), it received no email marketing treatment whatsoever for a period of 18 months.
To ensure that individuals selected into the control group were excluded from all email marketing efforts, the team created an exclusion table in its marketing database and automatically excluded the control group from any database extracts that were sent to its email service provider (Responsys).
What E-LOAN learned
What was E-LOAN looking for? The company wanted to evaluate the application rate of the mailed population vs. that of the unmailed population (the control group) over an 18-month period. As you might expect, at the beginning of the test there was very little difference in the application rate of the two groups. But, over time, the emailed population had a significantly higher application rate. So much higher that, according to Robert, the difference alone is enough to justify the total annual cost of the company’s email program.
Robert is also quick to point out that it’s not just about being in the inbox that matters. The E-LOAN team works hard to ensure that its email program is relevant and it uses a mixed strategy that includes broadcast messages (e.g., “The Fed has lowered interest rates”) and highly targeted, event-triggered communications.
Recommendations
Email marketers often complain that they don’t have enough staff and struggle to manage what’s already on their plate given the staff that they have. I believe that this complaint becomes a self-fulfilling prophecy… Email is cheap, just blast it out, and we’ll keep the bare bones operation going… To break through this cycle, email marketing managers should:
-
Devote 6-8 hours of the team’s time to implementing a test and measurement strategy. As a manager myself, I know that it’s possible to squeeze some amount of extra time out of the week. If you are really committed to making email marketing more strategic in your company, find 15-20% of someone’s time and focus it on test and measurement as Robert has done at E-LOAN. Sure, the E-LOAN test that I’ve shared here took 18 months, so start with something smaller. For instance, test the difference between a broadcast newsletter and a newsletter with dynamically targeted content.
-
Trend results over time. A quick test to show that targeted content generates higher open and click rates is indeed interesting. But, it’s more interesting to trend this information over time to evaluate the sustained value of a targeted vs. untargeted program.
-
Document a business case. If your goal is to improve the internal stature of your email efforts, get more budget, and grow your team, then it’s imperative to document your case. Avoid doing this at an individual campaign level and comparing metrics — like opens and clicks — against industry averages. Focus on the bigger picture and build a case that exposes the real business value of your efforts. What is the ROI of targeted vs. broadcast communications? Or, as in the E-LOAN example, do your customers buy more if they receive email marketing communications from you? This is the kind of case that your bosses need to free up more resources.
