This is one of a series of blogs from Paul Weston, the architect of The Customer Framework’s SCHEMA® toolset. In each blog he reveals the thinking behind the individual capabilities and practices that make up one part of the SCHEMA® Capability Assessment, which is becoming the World’s leading customer management benchmarking tool. There are 110 capabilities in the assessment, containing almost 400 individual practices that together provide a comprehensive definition of what ‘Good’ looks like in today’s “Customer World”. It is this definition against which organisations can be assessed and benchmarked.
In the “EXPERIENCE MANAGEMENT” area of the assessment we have a specific capability called “Researching the Customer Experience” and this is what we look for:
Event and sample-based customer satisfaction research
Research is carried out with customers who have been through a broad range of interaction types (e.g. complaint, purchase, enquiry, web session) to understand their experience of having done so. This is carried out regularly enough to identify changes in the quality of experience being delivered. A complimentary but not identical set of research is carried out periodically with a sample of customers who have not been through any specific interactions to understand the attitude and perceptions of customers at a ‘background’ level. The nature of both types of research is carefully controlled to prevent individual customers being researched too often and the sets of questions becoming too long. A small set of questions form the consistent core of all the research in order to enable comparison across customer types and over time. These core questions are reviewed at least annually in the light of findings from qualitative research and behaviour analysis. A robust process approves the non-core questions for each type of research that considers the customer experience of going through the research itself.
Qualitative customer research used to ensure focus of other research
Professionally executed qualitative research methods such as depth interviews, focus groups or projective techniques are used at regular intervals to enable both prompted and unprompted input to be captured from a range of customers. The research allows unstructured views, thoughts and feelings to be captured that would often be missed in quantitative research. It always probes and challenges received wisdom about what makes customers and potential customers ‘satisfied’ or otherwise with their experience. It seeks to identify new factors that impact customer perceptions, potentially due to new market entrants or product evolution, and factors that were previously trivial in terms of impact but are starting to become more important. A clear process ensures that the interpretation and analysis of the findings can either result in changes to wider research to assess the scale and impact of new factors or are explored further using techniques such as on-line listening or observational techniques.
Design of research to enable comparison to customer behaviour
Customer research is designed and carried out in a way that enables the organisation to compare the way that customers actually behave to the way that they say they intend to behave in research activity. Where the research is not carried out with a promise of anonymity then this simply involves ensuring that the relevant reference numbers are included in results reporting to ensure that responses can be matched back to systems storing behaviour data such as transaction systems, data warehouses, self-service systems etc. Where respondents are promised or would expect anonymity then a specialist, third-party supplier is used to merge behavioural data provided by the organisation with research data provided by the research company. They are briefed on the analysis required, carry this out and provide not-individualised results regarding the correlation between attitudes / claimed intentions and observed / recorded behaviours.
Inclusion of predictive (of loyalty or engagement) questions in research
Customer ‘satisfaction’ research carried out by the organisation is designed to assess and predict levels of customer commitment / loyalty (repeat purchase) rather than just understand whether or not they are satisfied. This has involved the creation of a scale of loyalty which runs from those customers who are known or suspected to be in an almost constant process of looking at other providers through to those who are extremely forgiving of any mistakes that the organisation makes and are very unlikely to take their business elsewhere. Customers are placed on the scale by the use of both direct questions about their intentions and by the analysis of answers to other questions that have been shown analytically to correlate with certain levels of loyalty and commitment. Loyalty and ‘Satisfaction’ levels for customers of different types of customer are compared to understand the relationship between them with particular attention being paid to understanding of customers where satisfaction and loyalty cannot be seen to correlate.
Disaggregation of customer expectation impact on satisfaction
Research questionnaires and analysis of satisfaction research results are designed to identify where specific customer expectations and experiences have an impact on what they say is important to them. This includes trying to understand factors that appear to have a big impact on overall satisfaction but which competitors do not even include in their research. This can often be because they perform so much better and their customers simply don’t consider it as a potential failure point. Industry or syndicated research (which includes competitors) is analysed to identify areas that are important enough to include in the organisation’s research. The results of this understanding are fed into the interpretation of satisfaction research to prevent highs and lows being considered as competitive strengths and weaknesses where they are overly driven by expectations and experiences that are peculiar to this organisation.
Use of mystery shopper activity to give a feel to satisfaction research
Mystery shopping is carried out to provide a degree of feedback on what the experience of dealing with the organisation ‘feels’ like to another person. This often adds a different perspective to the relatively ‘dry’ information provided by quantitative customer research and can also provide powerful illustrations of what quantitative research is saying. Verbatim feedback from the shoppers can often be as powerful as the analysis of their structured input and is particularly powerful where it is combined with video or audio recording. Some of the research is carried out by professional providers but this is supplemented by formal, internally resourced mystery shopping using staff from one part of the organisation to test the experience of dealing with another part of the organisation and to report back on the experience. Care is taken in all the research to prevent using examples of particularly good or bad experiences that are out of alignment with the general findings of other research unless they are being used for a specific purpose or have been further tested to ensure that the organisation is not over-reacting to a single isolated experience.
