The Twelve Pitfalls of Mystery Shopping


Share on LinkedIn

Why it simply doesn’t work and what to do instead

What is Mystery Shopping?
First established in the early 1940s, mystery shopping has been around for a while. But just because it’s been in action for almost 70 years, doesn’t mean that it’s an effective method of data gathering or performance improvement.

The entire goal of mystery shopping is to measure employee integrity and operating standards. Together these form: ‘Customer Experience’.

So what is the best way to do this?

Maybe, engage a third-party provider. They send in undercover customers so you can find out everything you need to know about the customer experience and keep an eye on your staff at the same time.

Sounds great, right?

Unfortunately, it’s not quite as clear-cut as that. Not only is this old-school approach ineffective for judging customer experience, but it has at least a dozen reasons to cause any business to think twice about its use.

Who Uses Mystery Shopping?
Mystery shopping can be used within any industry, but the most common tend to be in retail, healthcare, and hospitality. These include:

· High-street stores
· Hotels and accommodation providers
· Movie theaters
· Restaurants, cafes and fast-food chains
· Health care facilities
· Independent shops

What is the Purpose of Mystery Shopping?
Mystery shopping was designed to gauge the effectiveness of an employee within a specific organization, or the collective performance of employees within a business. The mystery shopping method is predominantly used by market research companies, watchdog organizations, and by companies themselves. They measure the quality of service, and to confirm whether employees are complying with regulations and policies.

It can also often used to gather specific information about products and services. In either case, the information is then fed back to the organization and used to improve internal processes and audit employees.

OK, BUT Does Mystery Shopping Work?
Much debated by many, but, in a nutshell:

Mystery shopping isn’t an effective method of measurement in any form.

Asking fake customers, paid by a third party to assess unfamiliar processes and procedures, using a synthetic transaction, once a month, isn’t going to provide valuable and long-lasting data.

This conclusion is based on hard facts and scientific evidence. Over the years, there have been several studies on the effectiveness of mystery shopping. The judgment that mystery shopping doesn’t actually work comes down to the data.

Don’t believe me? A detailed article from The Journal of Retailing from earlier in 2019 can be found here.

12 Reasons Why Mystery Shopping is Flawed
Here are just some of the reasons that mystery shopping isn’t worth it. The dirty dozen.

  1. Expensive – Mystery shopping is a considerable expense to the organization carrying it out. Each visit from a mystery shopper costs between $30 – $100 per visit. Yet alternatives to this cost the same for unlimited feedback.
  2. Lack of ROI – Not only is it expensive, but there is little evidence available that mystery shopping brings in much return on investment. Moreover, there is no definitive way of calculating it.
  3. Not Enough Data – Unlike in-store surveys and exit polls, mystery shopping is a one-man, one data point job. Therefore, the level of data that comes from just one mystery shopping experience is always going to be very minimal, making the data unreliable for operating improvement.
  4. False Premise – While mystery shoppers should be unbiased, they are not real customers. Consequently, the overall experience cannot be genuine. They bring their own preferences, habits, and shopping senses to the table, and are looking for issues. This means mystery shoppers have a built-in negative bias.
  5. Not Specific – Mystery shoppers don’t always cover enough of the shopping experience for the results to be effective. This means they may never uncover the main issue that drives customer behavior.
  6. Staff Awareness – Often, employees know who the mystery shoppers are or when they are expected to visit. They can prepare and react differently to a regular customer, likely over-performing in presentation and service.
  7. Not Time-Sensitive – Shopper findings take a while to reach the relevant people so becoming irrelevant quite quickly. Even worse, the actions taken by staff are often not tracked so the business cannot close the loop on the original issue.
  8. Unnatural – Most mystery shoppers are instructed to give feedback on specific areas. These may not be familiar to them. They may also be incompetent in the subject matter. They may also be instructed to shop in a certain order or process. This means it is difficult for a mystery shopper to always behave like a typical customer, and will produce unnatural results.
  9. Inconsistent – Mystery Shopping is only as good as the Mystery Shopper. Mystery shoppers themselves vary enormously in capability. This is partly driven by the high turnover of the industry. They may also not represent the demographics of the real business. All this means that any result that one shopper produces may be entirely inconsistent with another.
  10. No Authenticity – Questioning a mystery shoppers’ skills, reporting and credibility is common practice by employees and managers, in order to improve scores. This takes time, deflects from efforts to improve operations and chips away at the credibility of the Mystery Shopping organization.
  11. No Prioritization – As there is no little data, the business has no method to rank any findings. Instead, all issues are treated as equal. Yet we know some customer issues are fatal flaws, while others may be just annoyances.
  12. Heavy Workload – Using a mystery shopper can take up more internal administration than planned. This can tie up employees and resources.

The Alternative to Mystery Shopping – Use Your Own Customers
Your business already has hundreds or thousands of ‘mystery shoppers’ – the customers that purchase from you every day – so use them! Customer feedback technology exists to take the mystery out of these shoppers by providing you with validated, instant feedback. Armed with this, you can now continuously improve the aspects of your business that you know your customers care about.

Moreover, real customers are authentic – they are the real thing, not a proxy.

There are solutions out there that provide a customer feedback and recovery platform which captures comments and feedback from real customers using their mobile phone. This is done “on the spot” ie. at the point of experience. The feedback data arrives in real-time and at any time throughout the customers’ experience. This means it can be used to assess any part of the offer eg. Staff courtesy, queueing, product availability, cleanliness, product quality – even restroom cleanliness.

This allows you, as a business owner, to track all feedback and immediately respond to any issue – within seconds. Consequently, you can fix the problem and connect with the customer – while they are still in your location.

The big benefits here are:

  1. Improved customer retention
  2. Increased operational standards
  3. Fewer complaints on social media
  4. Lower maintenance and HSE exposure
  5. Lower overall feedback costs

In this modern-day and age of technology-driven opportunities, there are far more responsive and results-focused ways in which to find out how your organization is operating. Many of these leverage the ever-present, cell phone. This means that even without a mystery shopper, it’s entirely feasible to measure the performance of your business from customer entrance to exit and everything in between.

More to the point, it is the more effective strategy to take.

It’s time to give mystery shopping up and use a digital alternative instead ie. a real-time data capturing and actionable customer feedback system using their own cell phone.

It’s a mystery no longer.

Matt Selbie
Founder and President of Oberon3 - virtual comment card via the mobile phone. A Brit with 20 years of international experience in the energy, retail, and software businesses. Lead 2 startups and founded my own. Now based in Portland, OR


  1. It is true that lots of mystery shop agencies do not train their staff properly, but certainly some do! One of the significant advantages of mystery shopping is that shoppers can shop the competition at the same time, giving the client an illuminating ‘apples to apples ‘ comparison.

  2. It has its place like any other measurement. Granted you don’t have the volume of data.

    What it does give you is:
    1) Depth of the transaction – hence corrective action
    2) Not having to rely on customers (who may not tell the truth)
    3) Unbiased opinions
    4) Prompted recall
    5) ROI – easily in the thousands of percent (yes thousands).

    It is part of the measurement matrix. I would say that Msytery Shopping is more about ‘compliance’ than measuring service though. Horses for courses.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here