Your Marketing Surveys Suck! How to Get Accurate Usable Results that Will Help, Not Hurt, Your B2B Marketing

Much of my early B2B marketing career and graduate program was focused on survey design and methods to gain marketing insights and develop marketing strategies. Because of this, I have a clear understanding of when surveys are appropriate, how to get unbiased information for them, and how to measure their results.

That said, I have seen many, many surveys (occasionally even from Marketo) that didn’t follow best practices. Often this was either because each department in the organization was siloed, or because managers were using surveys as a way to justify their ideas – even if the results ended up being inconclusive or was not measured appropriately.

There are a few different ways to deploy a survey. These include mail, phone (through automated prompts or a live person), on your website, in person, or a mix of any of these. To drive people to surveys, any of a number of tactics may be also used, including incentives (gift cards, cash, rebates, sweepstakes entry) and promotions (email, banners, log in pages).

While the method you choose to deploy your survey will have significant impact on its results, the promotion itself can skew or sway answers, especially if the survey taker believes that specific answers may lead to different incentives or improved chance of reward.

Of course, the reason you choose to do the survey definitely will affect the way you should implement it.

Why do we survey at all? Typically it is because we are seeking:

Below are some best practices for each.

• Industry information or market analysis.

• Information about your industry or a general market analysis is usually best compiled by a third party. Ideally this is an analyst firm run by people who understand your industry and can survey based on their deep, specialized knowledge.

  By surveying through a third party, you eliminate the risk of bias from your own firm affecting the outcome. Also, participants are more likely to share accurate information if they are talking to an organization they know and respect.

  That said, be sure your marketers clearly explain the information they are looking to gain to the surveying firm, so they don’t end up with a bunch of data that doesn’t answer their questions.

• Benchmark information.

• Benchmark information is one of the most valuable types of survey data – it can be extremely interesting to the press, your customers, and prospects.

  When putting together benchmarks based on your clientele, it is generally best to survey them from your own organization, via online, telephone or in-person surveys. If your organization lacks the survey know-how to do this, bringing on a consultant to help can be a great way to keep this in-house without giving up quality.

• Lead generation.

• Often online or phone surveys are used by marketers for lead generation. An organization typically does this in one of two ways: either by surveying people in their database to understand Budget, Authority, Need and Timeframe (BANT) to understand when prospects may move forward in their purchases, or by surveying those who are not yet engaged in the sales process to see if they are a good candidate for a product or whether they may be making a purchase.

  The first type – BANT surveying – is best done in-house or by working very closely with an appointment setting or survey company. The trick here is to make sure survey questions don’t ask BANT questions directly (like- will you have budget for this), but instead asking in a less direct way (like- what percent of you 2011 budget is allocated to the purchase of x). If you do want to ask directly you should use a third party, but I still wouldn’t rely on the results to be true indicators of budget, authority, need or their timeline. Regardless, I suggest these surveys be done through phone surveys, and not online, to avoid be sent to the email trash bin.

  The second type – surveying to find leads – is best done by a third party who has a database that includes those who are purchasers of your product. With this type of survey, you often pay a price per lead for those who answer questions with certain answers.

• Content polls.

• Many companies use online surveys as a method of creating interesting content. You may see this done via Twitter polls, with widgets on the side of blogs, in webinars and more.

  Typically these types of content marketing polls consist of only one (or a few) general questions that are unrelated to a particular product or service. These polls are then used for content to share back, either in the place where they were deployed, or by including results in a blog post or article.

  It is important to remember that this type of surveying is often very far from having a random sample, so the results should not be presented back as scientific fact, and definitely should not be used to make strategic business decisions. (Example: “In a Twitter poll, we found that 70 out of 100 people said X.”)

• Customer surveys.

• At Marketo, we have at least a half dozen departments that want to survey customers online at any given time. These proposed online surveys often ask the same questions or intend to use the data to support a single goal or objective. The problem is, when multiple surveys go out per month, customers get frustrated or confused, and the organization itself looks disconnected.

  A much better bet for your B2B marketing is to conduct an annual survey of your customers. This will prevent list fatigue and force those who want the survey data to think critically about the questions they really want to ask customers. Note: everyone at your company is not going to be a fan of this because they won’t be able to reach out to customers whenever they want, so it’s best to get executive support of this approach.

  Because your company will rely heavily on this data, the survey should include incentives for those that participate. Also, you may want to consider using an organization that can help deploy and interpret results. You can deploy this online for practitioners and via a phone interview for more executive participants.

  If you need to survey a CEO, this annual survey may need to be done with an in-person interview. A survey company can help you understand how many results will need to be captured to get statistically accurate and meaningful results. You don’t have to get feedback from everyone – but enough people so you can use the survey to make strategic decisions.

  Then when surveying needs to happen outside of this annual survey, this should be done through phone or in person customer interviews. These in-depth discussions may not be as easy to deploy as a quick web survey, but they are likely to convey deeper information about a specific the problem or question.

  The exception here is with product feedback. Since product features are often released multiple times a year, your best bet is to allow customers to submit ideas and feedback through a community. This will allows the product team to get immediate feedback without polling. The product team should use the annual survey results and customer interviews to gain insights for general road map direction, not for gaining feedback feature by feature.

• Competitive surveys.

• You lost a deal. Now you want to follow up with a survey. If you deploy this survey online, and only to people who have selected an alternative to a product from your company, you will likely get back biased “information”: they will either be back-peddling because they feel guilty for not purchasing your product, or they will be frustrated with your sales process and complain about the entire process. Either way, you will not get clear insight into what the main factors were that caused them to go with a different product.

  Instead, surveys of this kind should be deployed by a third party and should keep questions focused on the buyer’s overall purchase process, rather than specifically about your organization. You also should use the survey organization to interpret results and trends as part of the process, since doing it in-house will likely mean results being conveyed in a way that matches the interpreters’ feelings about their organization.

  This same process should be used for surveying customers who have purchased your product, so you can get insight into what about your process did work.

Still don’t think you need to re-examine your surveys? I recently read an article in Financial Times that pushed the idea even further, suggesting we have entered into serious survey fatigue and that today surveys can almost be replaced by capturing data and feedback in social media and mixing it with in-depth interviews.

What do you think?