As UX professionals we use qualitative research to gather stories behind data and to shed light on context–specific experiences and opinions. For example, a procurement officer visits a B2B company website to make a shortlist of potential suppliers. They have a totally unique set of tasks and motivations compared to an investment manager visiting the same website to scope out options for a client’s portfolio. Let’s take an e–commerce website as another example, a new user might be searching for products and considering purchase, their behaviours and motivations will be different from an existing customer on the same website looking for delivery updates or returns information on a purchased product.
We can’t begin to understand the differences in what each user needs from the interface without exploring their contexts of use.
The importance of quality data collection in qualitative research
When conducting qualitative user research, the quality of the findings is heavily dependent on the researcher’s data–collection technique. The challenge with qualitative research is that, unlike in quantitative methods, the researcher is the instrument. To put things bluntly, a survey (if well designed) carries limited risk in terms of data extraction because it only relies on one human (the respondent), but qualitative methods rely on another human to do the extracting (the researcher).
Interviews – a familiar example of qualitative research
The impact of good and bad research technique can be clearly illustrated with interviews. While there are many other qualitative methods that UXer’s need to have up their sleeves (workshops, usability testing, ethnography, diary studies, focus groups) there aren’t a lot of parallels to these in day–to–day life. However, we see interviews all the time with well–known celebrities and politicians and it’s easy to see when they go well – or horribly wrong.
Bad interview technique
Richard Ayoade is an actor and director known for his very quintessentially British wit. In 2014 he published a new book and Channel 4 interviewer Krishnan Guru–Murthy was tasked with quizzing the author about his work.
Watch the clip below and observe how Krishnan conducts the interview:
If you didn’t watch to the end, I can summarise it with Ayoade’s painfully accurate closing remark: “Don’t thank me, I’ve done nothing for you.”
Why did it go so badly? Here are a few humble critiques I would make on the interviewer’s technique:
- His first question reveals that he didn’t understand (or read?) Ayoade’s book
- He freezes because he hasn’t prepared back–up questions
- He doesn’t react confidently in the moment when new information is presented
- He admits that he wants to change the framing of the interview to something different than what was agreed in advance
- He asks blatantly loaded and leading questions that are more about his own views than those of the subject
If this was a user interview, the interviewer would be no closer to meeting the research objectives than before they began. While Ayoade’s clever quips and fluent sarcasm make it entertaining, we don’t learn anything unique, personal or poignant about the participant, and certainly nothing useful about his new book.
On a side note, if you enjoyed that clip, wait till you see poor Krishnan winding up Quentin Tarantino with even less subtle leading questions.
Good interview techniques
A good researcher invites the subject to fully articulate a lived experience, a thought process or an underlying belief. This sort of data is very useful in UX when we want to create user–centred designs. Keeping with famous interviewers, there are lots of good examples we can learn from:
- Expert inquisitors like Jeremy Paxman and Louis Theroux who don’t let awkward silences or question dodging derail them
- Entertainment hosts like Terry Wogan who lets his guests share their stories with raw authenticity like when he sat down with Paul McCartney
- Long–form interviewers like Chris Anderson (TED) and Joe Rogan who can make the most rich and powerful people in the world show their human side
9 rules for collecting qualitative data
The style and format of every research session depends on the context of the study, but there are a number of techniques that researchers can use to make sure they collect the best possible data in any scenario. The following rules apply to any qualitative research method:
- Stop talking and let the participant speak – avoid going around in circles when you ask a question or make a comment
- Listen closely and only ask meaningful follow–up questions that are pertinent to the study
- Don’t interrupt – let them get to the point in their own time (within reason)
- Don’t ask leading questions – this makes your findings less reliable
- Don’t affirm or ‘mmhmm’ the whole way through the participant’s answer – be neutral and let them change their mind as they’re speaking
- Don’t talk about yourself unless it’s essential to build rapport or drive the conversation forward
- Don’t pretend to be an expert in what the participant is talking about – it’s their experience, encourage them to explain things in simple terms
- Allow enough time to talk about everything you need to, but don’t tire participants out – 45 to 60 minutes is ample for one–on–one sessions
- Make the subject feel comfortable and important by showing a genuine interest in what they have to say
Why design needs good qualitative research
Qualitative research is an essential activity for UX design teams because it introduces real user needs, goals and expectations into our decision–making frameworks. Only a robust data collection process can produce insights that are credible, transferable and trustworthy. Quite often this boils down to our preparedness and our ability to keep quiet and listen.
Cover image: Mediedgar, N (image owner). Louis Theroux on stage being interviewed by Anne Lindmo. Retrieved from Flickr. Modified from original.