How Biased Are Your Customers?


Share on LinkedIn

We all have biases. These biases are formed of our experiences in life and create prejudices in favor of, or against, different things. One of the many ways these biases manifest is in our responses to psychometric tools like ability tests, personality questionnaires, and even responses on Net Promoter Score (NPS) inquiries. These differences might be little, but when you group them together over a large set of data, you discover that they add up to some big biases.

Net Promoter Score (NPS) is used to determine how likely you are to recommend a product or service to one of your friends or family. Expert NPS analyst Brendan Rocks, Head of Data Science at Satmetrix says demographic details of your respondents can bias the NPS. And you can detect these biases accurately—provided you have enough data.

Most people want to believe they aren’t biased, as bias is generally considered to be a negative thing. However, because we are human, by our very nature, we have prejudices or biases. When you consider how biases shape our behavior, it’s easy to see they can change the way we respond to certain stimuli, like, for example, a Net Promoter Score inquiry.

Rocks says that most of these biases are “small and noisy.” However, because Satmetrix has a large database to sort through, they can identify demographic biases in NPS at a higher level of understanding. For example, Rocks says that all other things being equal, on average, men will rate a brand around four Net Promoter points lower than women (e.g. on a scale from -100 to 100). Also, he says the data shows that older consumers tend to give higher scores than younger ones.

“These little differences are cumulative across demographic dimensions, and can add up when you’re comparing segments in aggregate,” Rocks explains. “For example, say you have one brand marketed towards older women and another towards younger men. Are the differences you see in NPS attributable to performance or demographic bias? Often, it’s just not possible to answer questions like this with data from a single company. But by looking at data across several hundred companies, we can get a pretty good estimate of demographic differences, and get to the bottom of the performance question.”

The demographic bias is not just tied to gender or age, however. It can also be caused by where you live in the world. Rocks explains that similar demographic biases are often associated with B2B companies that look at NPS for different countries. When employee compensation structures are linked to NPS, it can cause a lot of problems for the various international regional managers—particularly if they are in charge of a region where the demographic bias for the nationality skews the scores lower!

Understanding these demographic biases and the reasons behind them is a complicated business, to be sure. Like many of the things associated with people’s behavior, however, it is essential for today’s businesses to use this understanding to help move their Customer Experience to the next level.

“Making data-driven decisions has become central to acquiring customers, and the same is increasingly true for retaining them. Customer experience practitioners are often put in a very tricky position. They are being asked to do things like establish causality with imperfect data and produce accurate predictions, all under considerable uncertainty, and for a business that’s constantly changing. This is where the skills of statisticians and data scientists come into play,” he says.

Rocks admits it is a complicated process to tease out the demographic bias in Net Promoter scores. However, the resulting understanding can help businesses make better decisions to optimize their business and improve NPS.

I am pleased to be doing a keynote speech at the next Satmetrix Conference in New Orleans in May. Find out more and register here for Unite, the Net Promoter Conference.

Republished with author's permission from original post.

Colin Shaw
Colin is an original pioneer of Customer Experience. LinkedIn has recognized Colin as one of the ‘World's Top 150 Business Influencers’ Colin is an official LinkedIn "Top Voice", with over 280,000 followers & 80,000 subscribed to his newsletter 'Why Customers Buy'. Colin's consulting company Beyond Philosophy, was recognized by the Financial Times as ‘one of the leading consultancies’. Colin is the co-host of the highly successful Intuitive Customer podcast, which is rated in the top 2% of podcasts.


  1. I couldn’t agree more with the real underlying issue: companies are increasingly committed to making data-driven decisions, but they pay insufficient attention to the integrity and reliability of the data on which those decisions are based. This is a classic garbage-in, garbage out problem.

    Pure statistical reliability lives only in textbooks, the laboratory and accounting. While data and applied statistics for business purposes have a more “relaxed” (i.e., practical) standard, there still are principles and best practices to assure a level of validity in the data. Time and again, however, I see shoddy data collection, management and analysis practices that threaten to undermine the value of the data that is used for decisions.

    While Colin and Rock rightfuly point to the importance of weighting and scale usage heterogeneity (the propensity of different groups of people to use scales differently), these issues can be meaningfully addressed only if the underlying data is well managed in the first place.

  2. Colin, this is very valuable insight and leads to two separate conclusions.

    1) NPS results, and also CSAT in general, can be used with minimum risk if you are looking at a trend for your customer base. If the survey responses are large enough, and your customer base remains the same, then all the biases (age, gender, location, survey mode, survey design, etc) do not come into play. They are essentially the same over time as long as there are no changes to the bias inducing attributes or the survey instrument and/or survey mode.

    2) You can run into serious issues if you compare one customer set to another. For example, comparing American results to results from France, even for the same company and exact same survey instrument, can lead to bad decisions because of cultural bias. And when you try and compare your business to a competitor you are opening yourself up to major issues. As the survey audience gets narrower and narrower the likelihood of bias is reduced but even when interviewing one person from each business we can all imagine significant differences just because of age, gender, and all the other factors.

    I think the moral of the post is to compare yourself to yourself and look for a steady improvement trend.

  3. Hi Colin: among sales practitioners in B-2-anything, there are rampant misconceptions about bias. Many pontificate about C-Level prospects as the ultimate rational decision makers, and B2B trainers admonish salespeople to always “make a business case.” The assumption is that when all the dust settles, C-Level buyers are “numbers driven,” and the best case will win.

    In my experience, that’s a myth. As you point out, biases color not only decisions, but approaches to making decisions. But based on your title, I was hoping to find some insight about the influence of bias in decision making, because this is difficult to measure.

    The best explanations I have found come from Dan Ariely’s book, Predictably Irrational. After reading his book, I believe that a business developer who thinks that the best financial case wins is either delusional, or hasn’t sold long enough.

    Our understandings of bias in buying decisions are nascent. Once demographic and other patterns have been allegedly identified, it’s hard to distinguish what’s signal and what’s noise. And even when there are signals, conditions have changed by the time companies act on the insights. As the pundits say, “things are changing faster than ever.”


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here