What is an analysis plan? And why do you need one?
In response to my CEM Toolbox: Making Sense of Your Data post, I was asked to provide some details about an analysis plan.
What is it? It’s a roadmap for how to analyze your data – and probably more importantly – why you’re analyzing it. Typically, it’s done for survey data, but you can compile one for any type of data that you are analyzing. For this post, let’s assume it’s survey data.
Why do you need one? To make sure that you ask the right questions – actionable questions – and are actually able to solve the issues that you are trying to solve. It forces you to focus, to think about the problems and the outcomes before you waste a lot of time with analysis ad nauseum, or worse yet, asking questions that don’t help solve the problem at all.
What does an analysis plan include? A lot of details. A lot of thinking about what you’re trying to do. Unfortunately, very few people take the time to think about this step in the process of designing their VOC efforts, but this is why we say: “Garbage in, garbage out.” Design your program, design your surveys, with the end in mind. Have a plan.
Specifically, the analysis plan is going to outline the following:
Background
Include the following background information; it will help to keep the analysis focused on what matters.
- Objectives and goals of the program
- Purpose of the analysis
- What questions you are trying to answer/issues trying to solve
- Who owns each question being analyzed
- Data sources (survey data, customer data)
- Population or subsets of the population
- Segments, how the data will be segmented
Details
Think about the types of analysis you’ll need to conduct to tease out the story from the data, how you want to prepare it, and how you want to present it.
- How to handle missing values (and other data rules)
- Key outcomes (dependent variables)
- Inputs (independent variables)
- Statistical analysis to be performed, e.g., regression, correlation, chi-squared, factor analysis, cluster analysis, etc.
- Descriptive statistics
- Outputs and formats, e.g., means, percents, two decimals, etc.
- QA requirements (yes, someone needs to QA your analysis!)
Questions
For each question in the survey, you’ll spell out:
- Question text
- Purpose of question (issue it solves)
- Owner of the question (who’s going to do something with it)
- How you want to analyze it, e.g., crosstab by segments, correlate against dependent variable X, etc.)
- Against what?
- And why? What are you trying to uncover with that particular analysis?
How long does it take? Depending on the level of detail, complexity of the problem(s) you are trying to solve, and the amount of data you anticipate, an analysis plan could take several hours to a couple of days to complete.
Who should have input? Ideally, your stakeholders will have given their input during stakeholder interviews as to what problems they are trying to solve. But it doesn’t hurt to run it by some or all of them again, once you’ve compiled the plan, to ensure you are still on the same page. It doesn’t hurt to also think about what actions will be taken as a result of what you uncover. If you don’t think the organization is going to act on it, don’t ask the question to begin with. Confirm with stakeholders.
When should I write it? It’s a good idea to think about this sooner rather than later, as it will help with survey design, too. If you think about the output and the outcomes, you will be forced to think about how you will use the question and if you need certain questions at all. Write the plan before or in conjunction with survey design.
Your biggest enemy is the unknown and assumptions. -Lt. General Claude Christianson
