When Big Data Goes Bad…

Big Data is the word.

You may have heard something about it.

Big Data in loyalty programs is being used to learn detailed profile metrics on our customers and use predictive semantics to identify triggers of loyalty/disloyalty, responses to promotions and other incentives.

We know their favorite seat, their favorite day to fly, their preferred car hire, hotel, grocery store, their favorite wine, who they travel with, and we can draw amazingly detailed profiles on their household structure, income and spending simply by analyzing credit card types and spend.

Loyalty programs are being swamped by incredulous volumes of customer data. I previously wrote that many airlines are now investing what seems to be the GDP of several nations in mining Big Data.

But can Big Data provide all the answers?

Are there limits to relying on Big Data alone?

In this article — we’ll explore some of the limits of relying solely on Big Data, taking a look at three loyalty programs who got it wrong by over-relying on data alone.

We’ll then explore exactly what a “data mirage” is, and how you can avoid common data hazards in your own program initiatives.

As HBR notes: “Data… reveals what people do, but not why they do it.”

“…data mining does not equate to developing “customer intelligence.” Human behavior is nuanced and complex, and no matter how robust it is, data can provide only part of the story.
Desire and motivation are influenced by psychological, social, and cultural factors that require context and conversation in order to decode.
Data can reveal new patterns that point a firm in the right direction, but it can’t indicate what to do once there.”

Despite the limitations of data, many organizations and program managers have developed a concerning level of hubris — basing significant business strategies solely on big data analytics alone, whilst blinded by what Gerardo Dada refers to as “The Mirage of Data”.

“It’s misguided to think that data alone enables us to make the right decisions and bulletproofs our path” — Gerardo Dada

WOOLWORTHS

Woolworths (an Australian grocery store chain), had an enormously successful rewards program based on a strategic partnership with Qantas.

In late 2015, Woolworths decided to ditch Qantas in an overhaul of its rewards program.

Customers were happily swiping their rewards cards every time they shopped, and Woolworths had an envious volume of customer purchasing data. They even invested in a data analytics company.

The data, combined with qualitative feedback and program modeling — told them that most customers weren’t redeeming their Qantas points and would prefer cash-back instead. They even commissioned academic research which showed that customers would receive a higher “payback” under their new program design.

It was an unmitigated disaster…

Initially — Woolworths stood defiant against the social media backlash.

Customers stopped swiping their rewards cards — the data collection stopped dead. It became impossible to ignore the customers that were deserting both the rewards program and the store itself.

A mere 10 months later — Woolworths was forced to reverse course with its tail between its legs.

You can relive the sorry episode here, here, here, here, here, as well as here, here, here, and here.

Or you can simply read any 2017 edition of Australian undergraduate marketing textbooks, for a case-study into what has become known as the worst disaster in Australian loyalty marketing history.

Tim Burrowes, the founder and content director of Mumbrella (and one of the smartest marketing minds in Australia), has the best synopsis of the saga that I’ve seen.

He summed it up the best:

“It was a bad idea, badly executed. Now it’s time to go back to what worked.”

Some more key insights where Tim is ‘on the money’:

“A really well-designed loyalty program promotes irrational behaviour…
One of the places where Woolworths went wrong was in trying to appeal only to the rational consumer.
It figured it could win the argument that it was spending the money on lower prices instead. And it gave far too much weight to the fact that many customers were doing nothing with their points. That wasn’t the problem they thought it was — even those who didn’t do much with their points liked earning them.
Another principle of behavioural economics kicked in — loss aversion. It infuriated existing customers who felt they were losing something, and months on, it still annoys them…”

The data had betrayed Woolworths.

STARBUCKS

There has been plenty of commentary surrounding the changes that Starbucks made to its reward program earlier this year — mostly poor in quality.

Starbucks overhauled its loyalty program in what should have been an overwhelmingly positive improvement. But Starbucks made a fundamental error in unnecessarily devaluing its reward proposition.

For a review of the program changes and what Starbucks got wrong — see “How Starbucks Botched Its Reward Program”, and “Did Starbucks Need To Upset Customers?”.

Starbucks is held up as the “Gold Standard” when it comes to retail loyalty programs, in particular when it comes to data collection and mobile payments.

Like Woolworths — Starbucks had no shortage of data on its members and their purchasing habits.

And like Woolworths — Starbucks initially stood defiant against the social media backlash.

But it wasn’t just a noisy social media response… Starbucks had screwed up. And they knew it.

After 25 consecutive quarters of above 5% sales growth — Starbucks dramatically fell short of estimates in the 3rd Quarter.

Starbucks openly admitted in its earnings call that it botched the execution of its reward changes, the result was that it undermined a key summer promotion.

Starbucks had no shortage of data, and knows full well that they should have handled the program changes better.

Luckily for Starbucks — they had the resources to weather the storm, and have spent the last 6 months aggressively subsidizing reward promotions to buy the necessary uplift in reward program metrics to ensure future success.

Starbucks recently revealed an additional data blind-spot, “What we are seeing is basically how we record the transactions of customers in our stores and what we had in the past, but we didn’t necessarily have visibility to, are people or parties of people coming in and buying multiple items… And, of course, there would have been no way for us to see that before. But what we saw were multiple beverages being put on Starbucks Rewards accounts where they hadn’t been put on there before” — Matthew Ryan, Starbucks Global Chief Strategy Officer.

The data had betrayed Starbucks.

AMERICAN AIRLINES

In April 2014, American Airlines made changes to its AAdvantage frequent flyer program with zero notice.

It removed an aspirational award option which allowed members to redeem Oneworld-based awards, popular for round-the-world itineraries.

Even worse — American announced that the change was immediately effective from the previous day!

Like Woolworths and Starbucks — American had no shortage of data.

But their data showed that the rewards were rarely redeemed. They were confident that hardly anyone would even notice the change, let alone be upset by it.

American was wrong — and they were surprised.

American however — responded positively to the backlash (which wasn’t restricted to only social media) — with then AAdvantage President Suzanne Rubin candidly admitting that the backlash caught them off guard, and that they would be sure to take member concerns about the lack of notice on-board in the future.

Say what you will about American’s recent 2016 changes (which you can read more about here and here), but American has been at pains to provide advance notice of changes since their April 2014 snafu.

Earlier this year I had the chance to discuss the April 2014 issue with senior AAdvantage managers — and they remain frustrated (and oblivious) as to why and how people were upset by the program changes “but no-one was using them!” was a clear comment by one senior manager.

To this day — many senior AAdvantage staff still don’t “get it”.

They had over-relied on data without understanding motivation and behavior.

The data didn’t show the aspirational importance of the awards.

The data had betrayed American Airlines.

THE DANGER OF THE DATA MIRAGE

Gerardo Dada wrote a brilliant article called “The Mirage of Data”.

Gerardo explains it the best:

“The mirage is that the answer is in the data, that more data is better, that if we collect enough data and analyze it we will arrive at the right answer.

A couple years back, Mohan Sawhney, one of the most brilliant marketing academics from the Kellogg School of Management, gave us a problem to solve. The case study included a couple data tables and multiple data points: prices, costs, volume, market research, etc. Everyone in the class, myself included, fired up our spreadsheets and came up with a couple conclusions — all wrong. Our mistake was assuming that because we had a lot of data, the answer must be in data analysis. The answer was in front of us the entire time. There was no need to analyze the data to find the answer, because the answer was found in basic marketing strategy. The data was a trap.”

Gerardo breaks down the data hazards to watch out for:

1. Data only looks at the past.
Despite conventional wisdom, data is limited as a predictor of the future. Otherwise, it would be easy to predict stock prices, sports scores or even lottery numbers. Albert Einstein knew the act of observation itself alters reality. By the time you collect and analyze data, the environment will have changed, limiting its usefulness.

2. Data does not tell you why or reflect emotion.
It is so easy to confuse information with evidence. Modern management practices focus on quantification of all the elements of a business. We are all human. We make emotional decisions and then justify them rationally. From your morning Starbucks, to the car you drive, to billion-dollar-decisions in corporations, it is all driven by emotions. You cannot fully understand customers through a numerical lens. Data can show us what customers purchased, how they paid and how often they visited. But it cannot tell us why.

3. Data is always biased. Every step in data collection and analysis has the potential to introduce bias. Surveys are biased by the subset of customers you survey and the subset of those who respond. Your transaction data is biased based on your existing customers. It may not accurately represent the entirety of the market because you are blind as to why non-customers fail to buy. Collection can result in a self-fulfilling prophecy. For example, when you display the most popular products on your website, they become even more popular — not because they are the best products, but because you chose to display them. Psychologists tell us we are all victims of confirmation bias, giving more value to the data that confirms our hypothesis and ignoring the data that contradicts it.

4. We often look at the data we can collect, not at the data we need.
Not all data is useful. Not all data is interesting, and most is not insightful. We tend to assume the data we have (or what we can collect or observe) must have the answer to our questions. The weight of a television set has nothing at all to do with the clarity of its picture. Even if you measure to a tenth of a gram, this precise data is useless. The number of Twitter followers a person has is probably not a good indicator of actual influence, even if it easy to measure. It takes guts to stop measuring things that are measurable and even more guts to create things that don’t measure well by conventional means. An online retailer can focus on conversion rates (percentage of visitors who buy), while being blind to the fact those larger number of buyers have a lower lifetime customer value (LTCV). Conversion is far more easy to measure than measuring and correlating with LTCV.

5. Data makes it easy to confuse correlation with causation.
In search of an ROI story, social marketers are quick to point out the higher customer value of those who follow a company on social networks, implying they buy more because they follow the company. However, is it more plausible that they follow the retailer on social networks because they are loyal customers? Without understanding the why and the emotions behind people’s decisions, it is hard to properly interpret data. While your data will show that a customer purchased a large-ticket item and a small-ticket item, it won’t reveal which item the customer came to the store to buy and which item was the impulse buy.

6. There may be more than one explanation.
It is common to look for the (singular) root cause of a problem, to find the one thing that drives customer buying decisions. The reality is that the way people make decisions is far more complex. Dozens of factors influence a single decision. It is like proposing marriage. Do you know why she said “Yes”? Was it your looks, your money, your smile or your sense of humor? As much as we want a simple answer that clearly points to a single explanation, chances are that events and decisions are the results of more complex, interdependent factors.

7. More data (or big data) is not better data.
You can survey thousands of people and ask them who invented the light bulb, which will result in a false sense of security in perfectly inaccurate data. We tend to believe that more data gives us more precision, when it usually has the opposite effect. It is easy to draw a chart with two axes and it is possible to compare data using three dimensions. However, adding a fourth, fifth or sixth dimension makes data visualization an impossible problem in itself. With more information and more data points to correlate, we are forced to ignore or minimize large sets of data, increasing our chances of arriving at the wrong conclusion.

HOW TO OVERCOME THE DATA DISTORTION?

“The best insights come from customers, not spreadsheets or analytics.

Sometimes we look for technology, sophisticated data models, complex analytics and expert advice to tell us more about customer behavior, when it would be easier to simply get out of the office and listen to them.
The best insights come from observing and listening to customers.
If you make it a habit to speak with a customer or three, every day, you will gain insights that no computer technology can give you.
Bernadette Jiwa captured it beautifully: “We assume that the most valuable data is static and lives on graphs and spreadsheets….The truth of what we need to know and some of the most valuable data live in plain sight. The wrinkled nose of the diner. The sigh of the shopper waiting in line. The posture of the customer as she walks out the door. What she packed in her bag before she left home this morning. Noticing what people do is often more valuable to us than listening to what they say they think”” – Gerardo Dada

SUMMARY

The lesson is that Big Data continues to bring immense opportunities to organizations to improve their analytics, reporting, targeted and personalized communications and offers, and ultimately — drive new revenue.

But there are limitations and traps to watch out for. Most of these limitations can be overcome.

But it’s important to understand that data is only one part of the equation and when it comes to loyalty programs — a deep understanding of customer emotions, behavior and psychology is a mandatory ingredient — to being able to accurately leverage the data you have (see The Psychology Of Loyalty Programs).