Research methods designed to quantify solution preferences are standing in the way of achieving predictable innovation.
Innovation is the process of devising solutions that address unmet customer needs. For decades companies pursued an “ideas-first” approach to innovation—they encouraged brainstorming to yield many alternative solutions and used market research methods that were designed to evaluate customer preferences as a means to determine which solution was best. Many gated development processes, including StageGate®, recommended this approach.
As time passed, and innovation success rates remained low, companies began to realize that uncovering the customer’s needs before devising a solution would potentially eliminate guesswork and likely lead to an improved innovation success rate. Adopting a “needs-first” approach to innovation made intuitive sense and is widely adopted today in many companies.
Unfortunately, as companies made the “ideas-first” to “needs-first” transition, they made one very big mistake—they tried to use the same market research methods they were using to quantify solution preferences to now quantify the customer’s needs. It turns out they were the wrong tools for the job and that new tools were needed to advance the innovation process.
Here’s what we discovered: while market research that employs max diff, forced choice, paired comparison, conjoint, stated/revealed preference and other techniques may be appropriate in quantifying solution preferences, none of them work when applied to quantifying customer needs.
Interestingly, this problem is not widely acknowledged as most companies continue to use the wrong tools to quantify customer needs. To make matter worse, many companies struggle to agree on what a need even is (see Inventing the Perfect Customer Need Statement).
These issues are stifling innovation. Fortunately, the problem has been solved. The research methods employed as part of the Outcome-Driven Innovation process were created specifically to quantify customer needs—or as we call them, the customer’s desired outcomes.
Before we describe the solution, let’s look at the problem more closely.
Methods for Quantifying Solution Preferences
All of these popular research methods have one thing in common—they are asking customers to make tradeoffs or to choose between alternative solutions, e.g., features, offerings, brands, etc.
The MaxDiff is a long-established academic mathematical theory with very specific assumptions about how people make choices: it assumes that respondents evaluate all possible pairs of items within the displayed set and choose the pair that reflects the maximum difference in preference or importance.
Two-alternative forced choice (2AFC) is a method for measuring the subjective experience of a person through their pattern of choices and response times. The subject is presented with two alternative options (often product features), and is forced to choose which one is the preferred option.
Paired comparison is any process of comparing entities in pairs to judge which of each entity is preferred, or has a greater amount of some quantitative property, or whether or not the two entities are identical. The method of pairwise comparison is used in the scientific study of preferences, attitudes and choice.
Conjoint analysis is a survey based statistical technique used in market research that helps determine how people value different attributes (feature, function, benefits) that make up an individual product or service. The objective of conjoint analysis is to determine what combination of a limited number of attributes is most influential on respondent choice or decision-making.
Revealed preference theory is a method of analyzing choices made by individuals, mostly used for comparing the influence of policies on consumer behavior. These models assume that the preferences of consumers can be revealed by their purchasing habits.
Here is the problem: when quantifying customer needs you should not be asking customers to make a choice between needs or asking them what needs they prefer. When applying Jobs-to-be-Done Theory and Outcome-Driven Innovation, customer needs research is not concerned with preference—it is concerned with discovering opportunities for growth: desired outcomes that are underserved or overserved.
Quantifying Customer Needs
When we compile a list of metrics that customers use to measure success when get a job done (usually 100 to 150 desired outcomes), the outcome statements are not a set of alternatives. For example, when interventional cardiologists are trying to restore blood flow in an artery, they want to “minimize the time it takes to reach the lesion in the vessel” AND (not or) “minimize the likelihood that a vessel is inadvertently punctured,” AND (not or) “minimize the time it takes to decalcify the lesion.” They may have over 100 desired outcomes — all of which must eventually be addressed to varying degrees to get the job done perfectly.
A customer may have 100 desired outcomes—10 of which are unmet. The goal is to discover those 10 unmet outcomes and to determine the degree to which each is unmet. Then a product team can decide how to best satisfy all 10 underserved outcomes.
When quantifying customer needs, the customer should never be asked to make a trade-off. The customer should never be given the choice of two distinct outcomes to determine which one they prefer. Tradeoffs are not made on customer needs. They are made when the product team is trying to come up with the optimal solution. The product team may not be able to address all the needs and may have to decide which they can best address. The product team makes the trade-off decision, not the customer. Another way to say this is that trade-offs are made in solution space not needs space.
If a company is using the traditional methods defined above to quantify customer needs then it is either: (1) using the wrong method to quantify needs, or (2) not quantifying customer needs—and probably still quantifying solutions.
Overcoming these issues is the biggest problem we see plaguing the market research functions in companies seeking to excel at innovation.
For additional information on this topic see Quantifying Customer Needs.
