Why High-Quality User Research Participants Are Worth The Investment
What is a High-Quality User Pool?
When conducting user studies, getting the right kind of participants matters. So what defines a high-quality study participant? That is for you to define. Have a clear definition of what roles participants should hold, and confirm their ability and desire to share their opinions, as well as their ability to follow instructions in order to ensure you get the right kind of participant.
A pre-screening questionnaire can help you narrow your search so you are only inviting people that match the customer profile you have defined.
Things to consider for your prescreen:
- What role does an ideal candidate have?
- What demographic group(s) should they belong to?
- What level of communication does the participant need? For example, does the ability to articulate in English or a different language matter? Understanding the kind of feedback you need will help you narrow down the pool to match the profile you require.
- What other factors matter? For example, some products are geared toward mothers, so targeting a woman of maternal age may not be enough. You may need to confirm that she is, in fact, a parent as well.
Tips to Improve the Likelihood of High-Quality Participation and Data Outcomes
The following are tried and true ways to protect participants, respect their privacy, and get the results you want – valid data.
- Avoid bias
- Use a recruitment script to ensure all participants get the exact same information.
- Employ quotas and recruit a diverse participant pool to ensure the sample is reflective of the population you’re studying and you’re getting everyone’s perspective that was deemed necessary.
- Follows ESOMAR guidelines to provide
- Informed consent means that participants know about the study prior to their participation.
- Protect their privacy and data.
- Avoid deception unless required for the study.
- Offer Incentive
- Payment for the participant’s time signifies that you value their opinion and time. They are much more likely to provide in-depth, focused feedback when they are compensated properly. For example, for longer studies, pay more money.
Data from High-Quality Participants
There are many benefits to recruiting the right people for your study. Data from high-quality participants:
- Saves time and money because the user fits the profile you defined for your research.
- Better study validity – because you won’t be relying on self-reported data, which can be seen as less valid and credible, your study’s validity is positively impacted. In other words, screening measures matter.
- Creates confidence in the decisions being made and resources being allocated based on the data. Businesses can allocate resources toward UX development when they can rely on the data gleaned from quality user research studies.
What is a Low-Quality User Participant?
When discussing user participation, we are not speaking to the innate value that individuals hold. We are discussing their alignment with the predetermined qualities needed for a successful user pool.
A low-quality user does not represent the user avatar as defined by the brand, cannot express themselves to the extent needed, or has a conflict of interest or malintent.
A low-quality candidate is more likely to produce low-quality data because of boredom, frustration, or disengaging before completing all the tasks. This kind of poor performance often occurs because candidates were not properly screened. When users are not reflective of the desired user pool, it impacts the validity of the data.
Imagine a hypothetical company called YouGram. It’s a new social media platform that caters to young people. YouGram wants to expand its users to include Millennials and Gen X. It decides to test on individuals born between 1965 and 1990. Its goal is to have test participants compare the user experience on YouGram to other social platforms.
YouGram should not allow just anyone within the target age demographic to participate in the study. They likely want to start their search on other social media platforms in order to get users that are more likely to have reliable feedback about their new platform in comparison to other platforms. If the company decides to market its research study off of social media platforms, the user pool will need to add a screening question to make sure the potential participants do in fact already use a social media platform.
Let’s imagine that YouGram got its participants from grocery store bulletin boards and Craigslist and did not prescreen for users that already use social media. We can see how the opinions provided by the user pool not be ideal data.
What happens with the data from a low-quality user pool?
In the case of our example social media platform YouGram, the data would lead them to spend time and money on a poorly suited user study pool, resulting in bad data, improper business decisions, and possibly catastrophic outcomes.
If they had created an ideal customer profile, added prescreening questions, and marketed the study in a more natural setting, their study’s data would have been more trustworthy.
Risk and Reward
At the end of the day, all companies want to be in business to offer the products and services they specialize in. They need to understand their users to deliver the increasingly sophisticated user experiences users are coming to expect. Product and service users can and do jump ship to competitors when they have bad experiences, so the focus on good UX is mission-critical to organizations. Good user studies with high-quality user pools should be a part of every organization’s strategy. First, to ascertain trustworthy data, and second to protect the investments and produce better ROI.
If your organization needs support in conducting user research, an expert user research platform like Respondent can help. They have millions of pre-qualified users in their database and have the expertise needed to conduct research to produce valid data quickly.
