In the last UX Bites Webinar of 2020, we discuss the role usability testing plays within the user centered design process. UX Bites #9 explores the why, when and how to of usability testing – and most importantly how to synthesise, analyse and feed recommendations into the design process.
Usability Testing 101
Video: UX Bites by Fathom webinar 9 ‘Usability Testing 101‘. Find this webinar and more on Fathom’s YouTube channel.
Q&A
Q1. What methods/software did you use for usability testing during the pandemic period? Not all users/customers are very familiar with this type of software, did you encounter any issues with this?
A1. During the pandemic the primary method of usability testing that we used was ‘remote–moderated’ testing. Remote testing always comes with some risk due to individual’s familiarity with software, but during the pandemic period we found that for the majority of users they became very comfortable with the remote video format with screen share over time. We believe that is partly down to COVID forcing more people to use more remote conferencing tools, more often – resulting in feeling more proficient in using tools like Zoom, which we often use.
Q2. How does moderated user testing work during COVID?
A2. Since March 2020, we have relied on platforms like Google Meet and Zoom to facilitate moderated user tests. In the beginning we found users a little hesitant to interact with video calling software, but as lockdown progressed we saw users becoming more familiar with the platforms. It can be harder to build rapport with users during a moderate remote usability test and as such we typically have a few ice breaker questions/topics at the beginning of the session to make the user feel a little more comfortable.
Q3. Which method would you say is the best way to convince non design stakeholders of certain design choices taken based on the testing and why?
A3. From my own experience, I would lean more towards unmoderated remote usability testing. I like this method of testing as it is usually automatically recorded and most of the platforms will provide you with transcripts so it’s really easy to get direct quotes/clips of users. These artefacts are invaluable to researchers trying to get more senior stakeholders to gain empathy with the users.
Q4. Do you have to offer an incentive? If so, what is typically offered?
A4. You don’t necessarily have to offer an incentive but it depends on a few factors, the most important being, the type of user you are looking to recruit. Is it an already engaged and enthused user? This type of user may be willing to take part in research without an incentive. Whilst a member of the public who may not have heard of your brand/company before may need an incentive.
It is also important to consider how long the usability testing will take. If it is guerrilla testing, which typically takes between 10 – 15 minutes then you may not need to offer an incentive. But if you think your test will be more than 30 minutes you might need to think about offering an incentive. Again the type of incentive will depend on your user. We would recommend doing a little bit of desk research into who it is you are going to recruit to see what their interests might be.
Q5. Do you notice a difference in the quality of insight you get from an in–person test in a lab versus a moderated remote test with someone in their natural environment? Or is all insight valuable?
A5. All insight from all testing formats is extremely valuable, and the right user type and tasks will always be the most important factors in the design of the test quality. We would typically (if test design, quality and user/persona type choice are equal) sway towards usability testing in a natural environment being best. There you will normally have the context of use, natural surroundings and the user will be on a familiar device. Those circumstances can lead to the test being more natural to the user, affecting their state of mind when approaching the tasks leading to insight more directly related to the interface – and not influenced by outside factors including unfamiliar, location and devices.
Q6. Do you have tips on convincing stakeholders of the importance of usability testing?
A6. When it comes to convincing stakeholders of the importance of usability testing, one of the best starting points is to frame the conversation around the cost of bad design and the ROI that design–first organisations achieve. We delivered a recent webinar on that very topic. We would recommend that when new product development and product evolution are happening, demonstrate the risk of commercial failure that can arise from not designing a product correctly or show the positive commercial impact that can happen from good design.
Usability testing is one of the core UX research methods used by most design teams and really should be in all product design initiatives; whether tactical or transformational and could be prioritised over another initiative if time, resources or budget are an issue. Another way to help convince stakeholders, is to show them tests. Demonstrate how issues or problems were solved through either a live or recent series of usability tests, or use previous tests. This real–life evidence can often help convince stakeholders that it is worth the investment.
Q7. What would you do instead when there is no time or money to accommodate User Testing?
A7. When there is no time or money to accommodate usability testing, the most important thing to ask yourself as an individual or team is; do we have research available to understand what is happening and why it is happening? A common method to help answer these questions is the use of behavioural analysis tools such as Hotjar. Hotjar allows you to see what is happening on the screen via heatmaps and click maps and in addition allows you to get qualitative feedback via polls or surveys. Whilst not as richly insightful as usability testing where the feedback is real–time and contextual to the task undertaken this form of research can be a fall–back that requires less time, budget and resources to facilitate, administer, analyse and report on.
Q8. Do you consider recording participant’s faces while remote testing necessary or are voice and screen recording enough in some cases?
A8. We would prefer to record the user during any form of usability testing as it gives great insight into what they are thinking. Recording their facial expressions is an invaluable way that we can potentially read between the lines of what a user is saying. Even listening to the tone of the voice can provide that extra bit of insight.
Q9. During the testing journey, do you recommend the use of reflective models, such as Kolb’s Experiential Learning Cycle, to transform knowledge from the users experience of the product? If so, where in the process is it most important?
A9. This is such an interesting question. For me, I think the Experiential Learning Cycle is more suited to the entire process of research and design of a product as opposed to just one particular research methodology. Usability testing is there to test the functionality of your product and therefore can fit into the various stages of the Learning Cycle.
Find us on YouTube
You can watch more of our webinars and talks about UX and service design on our YouTube channel.
