Agile software development calls for new, agile usability testing methods

A software development team can assess its own success only to a point. To be successful, many aspects of the service being developed should be tested with actual, living-and-breathing end users. This helps us answer questions like: Does our service make sense to them? Can they carry out the required actions by using our service? Is the service generally easy to use?

Services often undergo a structured usability test, carried out with 6-10 end users, who are asked to use the service to complete given test tasks. To ensure that the results are comparable, these test tasks are usually the same for all participants. At the end of the test, the test moderators compile a report that covers any problems observed as well as suggestions for how to fix them.

However, the traditional usability testing process can take weeks, from recruiting users to compiling the report. When the report is finally completed, agile development has already advanced to the next thing. The features tested, as well as several new ones, may already have been launched in the service. What’s more, because tests are often infrequent, they can include testing of a substantial portion of the service. Agile development works in much smaller chunks, with focus on only one or two things at a time; this means that even in the best case scenario only a very limited portion of the new report is useful to the team, and the rest goes to waste.

Get the benefits immediately

To make usability testing more useful for an agile team and to allow them to make use of the results faster, usability tests need to be made more agile. Here’s how it’s done:

1. Fixing user interfaces immediately is often a good idea.

If you discover a fatal flaw during your first usability test, you can fix the problem immediately, before you meet with your next test user. This allows you to validate your corrections immediately. This is not possible with traditional testing, where the aim is to find all possible problems first, and produce results that are comparable between test users. With agile user testing, the goal is to directly improve the end result, which means that the user interface can be modified between the tests.

2. Genuine use reveals more than anything else.

In traditional tests, users carry out made-up tasks that ask them to step into someone else’s shoes. However, it may be difficult for a single student in her early twenties to step into the role of a mother of two and imagine how she’d book a cruise to Stockholm. Can you put the kids in a cabin of their own if no shared four-bed cabins are available? Instead of asking users to role-play, agile testing focuses on users resolving situations that are realistic in their own life. This means that the development team can directly see how well users manage real-life use. Usually a single test session involves just one or two situations.

3. A few users at a time.

Almost without exception, the main issues in any service become clear after being tested by three users. Once they have been fixed, you can proceed to the next test round of three users, and then to the next round. This cycle can easily be repeated until the team is satisfied. The number of users testing a certain portion of the service will most likely stay below ten.

4. Taking action is more useful than reporting.

Designers who will actually make the improvements to the user interface should always be present when user tests are run.This allows any discovered issues to be recorded immediately, and resolving them can begin at once. Modifications and fixes can be done at the latest after the final test, and there is no need to wait for a separate report. Testing becomes more effective and meaningful if the person carrying out the tests is also part of the development team. This way, you don’t have to spend time on introducing the moderator to new concepts, and any suggestions for improvements are likely to be more easily implementable, as the moderator is already familiar with the service.

To create a good, meaningful service, high-quality data and a deep understanding of user goals and needs are necessary. These need to be collected from real end users before the design work even begins. User testing cannot save the service if understanding and design fail. But when used well, usability testing is a valuable tool for the team. It just needs to serve the team optimally – in an agile way.

How does your team utilize usability evaluation methods?