What SaaS StartUps Need to Know About User Testing


Share on LinkedIn

Software as a Service has been a fantastic innovation for Startups. Gone are the days when a new business had to design, test, and implement their own IT applications. One aspect of quality software development has, however, become even more crucial.

This is user testing. Why? Because in today’s world of IT self-service, user experience is king. A great experience can be the difference between success and failure for a SaaS Startup. This is underlined by research compiled by Toptal, which reported that 90% of users had stopped using an app due to poor performance.

Here are the key things a Startup will need to ask when planning user testing.

Why are we carrying out the testing?

Ultimately, this all comes down to your users’ experience. A great design makes for a great user experience. In turn, a great user experience rewards a company. Companies that focus on their customers are 60% more profitable than those that don’t.

What are the key activities users will want to undertake? This might vary depending on your business, but typically activities such as registration, navigation, product selection, purchasing and delivery tracking would be high on most lists.

These key user activities need to be easy and intuitive, and leave the user with a positive feeling. Get it wrong and you will lose people. Think about the things you want to learn from the testing and focus your activity on these.

You’ll need to go into this with an open mind. Don’t have preconceptions about the things you like. It’s the user’s view that matters, and it’s this that will drive the next steps for your startup. Remember, you’re seeking a great experience that makes you stand out, not an okay one.

Are you developing a mobile application?

The huge growth of mobile apps is driven by users’ increasing expectations that these will be available. If you want your app to succeed, it will need to be very thoroughly tested before users have access to it.

If you want your app to stand out from the crowd, give serious consideration to your automated mobile app testing strategy.

What about functional and regression testing?

With SaaS systems, user testing is a priority. Your application may, however, need periodic functional and regression testing.

The former focuses on system functionality, whilst the latter checks that changes haven’t disrupted existing functions.

Where possible, you can use automated testing to reduce time, effort, and specialist skill needed.

When should I test?

Start early, ideally at the prototype/design stage. You might have ideas that seem fantastic, but when a user tries to negotiate them, they run into problems. That may not be the news that you’re hoping for, but it’s better to find out early. Otherwise, you could waste time and investment on designs that won’t work.

You’ll also want to revisit user testing at key steps in development or after an important design change. 

Think of testing activity as a set of iterative steps rather than one big exercise. In a world where an agile approach to IT development is increasingly common, this is even more vital. Keeping on top of user testing will mean less wasted effort and enable your energies to be invested in improvement and refinement.

How should I develop the tests?

Consider the activities that users will undertake. Prioritize those that will be undertaken most frequently and those that are the most important to you.

Develop a narrative for user journeys through the system by creating imaginary users. Then plot their activities in your system. You can then turn these into scripts; instructions that the tester can follow.

Remember, though, that you want the process to be intuitive, so don’t be too specific in your instruction. You’ll want to be very clear about ‘what’ you want them to do but not so specific about ‘how’ they should do it. It may be useful to learn about process mapping here to be able to fully realize the strategy you want to execute.

The scripts will often be written by someone who knows the system but they are aimed at people who don’t. It’s a good idea to check them out on someone without that knowledge before you introduce testers to them. You want to be testing the process, not the standard of the script!

Exploratory software testing can also be a useful way of gauging the effectiveness of a system. Instead of following set scripts and seeking a certain outcome, users are given free rein to investigate and explore the system. This approach can have unexpected results and uncover problems that might otherwise have been missed.

It will also be important to set the scene for users. Thank them for taking part and let them know how important their contribution is. Ensure they know that it is the process, not them, that is being tested.

Make sure to also acknowledge and show appreciation for how valuable their time is to you. It may be helpful to set up something such as a countdown timer for meetings in this case.

How many users will I need?

The thought of recruiting and managing a small army of users can be daunting. Well, the good news is that you don’t need to!

It is self-evident that no testers will result in zero insights. Bringing in just one tester makes a huge difference. But a single user may not be representative of general practice, so much will be gained from a second user.

With each additional user, you will repeat the insights already gained. The incremental benefits that the tester identifies will diminish. 

This article by Neilson Norman Group demonstrates why just five testers may be the optimum number. It is claimed that just five testers will uncover around 85% of problems. Rather than add additional testers in one go, further tests at regular stages, again with a small number of users, will be much more effective.

Something to bear in mind when selecting the number of testers is whether there are distinct groups of users within your target audience. An example would be adults and children. These groups might act differently, so if your process is aimed at both, you’ll need to ensure that each is adequately represented in the testing.

(Graph source: Maze.co)

Where can I find users? 

This will depend on how specific your users are likely to be. Perhaps your users come from a general audience? In this case, you could recruit volunteers by setting up a stand in a shopping centre, outside a train station, or even in a coffee house.

If your users are more specific or specialist, social media groups can be a good source. In particular, Facebook groups can be a useful resource. Your testers should be representative of your current or future users. Another often overlooked avenue is to make your business easily reachable. There are many conference call service solutions out there that make this possible.

You could rely on service providers such as Fiverr or Craigslist to do the work for you. However, as we have seen, the numbers needed are quite small. Finding a group of five testers shouldn’t present too much difficulty.

One question that comes up is whether to offer a reward to testers. These could take a number of forms; gift cards, vouchers, even chocolates could be enough. Be careful though, you need testers who are interested in the activity, rather than motivated by the reward. Testers motivated by money could lead to skewed results.

Some would advise to avoid rewards for this reason. They can sometimes be useful as an incentive, however, the best advice would be to keep the level of incentive low if you do offer rewards.

What can I learn from the tests?

It is useful to interview the testers. Don’t take too long, 30 minutes should be sufficient. Find out what they liked and what they didn’t like. Don’t take their criticisms personally!

Typical problems encountered might be:

  • Failure to complete an action
  • Delays in progressing items because of problems or uncertainty
  • User frustration (take note of the tone of the feedback)
  • Using a workaround – the users had to take a different route to achieve their objective.

You’ll need to flush these and other problems out. Use open questions such as ‘how did you find the process?’, ‘where did you expect the button to be?’, or ‘can you describe the problem that you encountered?’.

Face to face discussions have obvious advantages, but interviews can effectively be conducted over Skype, Zoom, or other similar applications.

What about integration?

If high quality user testing addresses current usability, how can you ensure ongoing Quality Assurance of your system? 

This is especially important given the changing environment that it sits in. Look to a blend of automated testing and exploratory software testing to deliver continuing integration.

What next?

The key thing is to act on your findings! Put time into analyzing the results and prioritizing the problems.

Don’t lose sight of the fact that this is all about user experience. Don’t just tweak a problematic process if it is really in need of a redesign.

Sort out the problems that come up frequently and those that impede critical steps, and you’ll see huge benefits. I mentioned earlier that testing should be seen as an iterative activity. You may want to keep yourself open to customers and testers alike and consider a CCaaS (contact center as a service) solution. Ultimately, though, it’s not the testing but the benefits reaped that will be iterative. Great user experience means more users and more frequent users.

For your startup, this means increased customers, higher profile, and greater profit.

Jenna Bunnell
Jenna Bunnell is the Senior Manager for Content Marketing at Dialpad, an AI-incorporated cloud-hosted unified communications system that provides valuable call details for business owners and sales representatives. She is driven and passionate about communicating a brand’s design sensibility and visualizing how content can be presented in creative and comprehensive ways.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here