If you apply predictive analytics to help make important decisions in your organization, I can guarantee that many people will think the predictions are wrong. In their mind, you (or your model) will be an idiot. So, who would want to use predictive analytics?!? The people who are focused on the big, long-term wins and not the specific, short-term battles.
A great example comes from a recent CRM magazine article by my colleague, Patrick Gibbons, titled “The Risky Business of Predictive Analytics.” As background, there is a robust analysis showing that NFL coaches would win more games by going for it on 4th down more often. In fact, the analysis suggests that teams would increase their win probability by attempting 60% more 4th down conversions.
Gibbons’ article references a 2009 game between the Indianapolis Colts and the New England Patriots. The Patriots coach, Bill Belichick, made a controversial call and went for it on 4th down. His team did not convert, the Colts got the ball with great field position and scored a game-winning touchdown as time expired. Belichick was skewered in the press for his “ridiculous” decision that “cost his team the game.” However, the analysis was pretty clear: Attempting a 4th down conversion in that exact situation gave his team a 13% increase in their win probability. Belichick knows this analysis, he has read the research paper on it, he goes for it on 4th down more than most coaches, and he really doesn’t seem to care what outsiders think. However, it is easy to see why other coaches might shy away from these decisions.
Let’s look at another game where Belichick made a controversial 4th down call. In 2013, the Patriots were losing to the New Orleans Saints by 1 point with 2:50 left in the game. With the ball on their own 24-yard line, Belichick went for it on 4th-and-6. The receiver dropped the ball, and the Saints took over. Another failed conversion near the end of the game deep in their own territory. The Saints scored a field goal, but the Patriots got the ball back with over 2 minutes remaining and ended up scoring a touchdown to win the game.
Some would still say that Belichick made a bad call but was bailed out by his players. The analytics say Belichick made the right call because he knew his offense would get the ball back even if they failed to convert. It’s not the result of a single play that matters, it is making decisions that consistently put your players and your team in the best situation to win. When that happens you get more success over the long-term, even if the results of specific plays make you look like an idiot to the uninformed. As a Denver Broncos fan, I hate to say this, but Bill Belichick and the Patriots are great examples of this.
As a CX professional looking to implement predictive analytics, be sure to keep yourself, your organization, and anyone impacted by the predictions focused on the 
big picture. Beware of what I call the “anecdotal idiot test,” which is when someone says, “The prediction for this case was wrong, and I was right, so your model is wrong!” (aka, “Your model is an idiot”). When this happens, do 3 things:
- First of all, remember the big picture. The goal of predictive analytics is to put your organization in a place to win more over time, not win every time. Research has shown over-and-over that good predictive models outperform experts over the long-term.
- Second, ask why they think the prediction was wrong? What did they see that the model missed? You will usually get a bunch of reasons that the model could never pick up, but sometimes you will get info that can help improve the model.
- Lastly, rely on the program sponsors to maintain compliance and reinforce the big picture. If the big wigs aren’t on board and supportive, then a few of these anecdotal idiot tests can ruin your program. But when they are bought into the long-term view, then these instances will be of little concern.
