If the corporate brochure is to be believed, researchers in pristine labs carry out perfectly designed research projects devoid of even a hint of bias, observing users on the latest technology who look just like the beautiful people in the persona definition picture from earlier in the project, and match the target customer demographic to the exact scintilla. These test subjects have the exact interests, social class, income, family background, gender, level of web use and expertise, technical prowess, professional and personal motivations as the company’s target customers. The icing on the cake is that the subjects are perfectly motivated to provide the researchers with as much time as they need to squeeze every last piece of information from them with the beautiful result that they provide the researchers with near–perfect results, free of noise and inaccuracies.
Meanwhile in the real world people of all shapes, sizes, stress levels, foibles, preconceptions, expectations, and web expertise on hardware of diverse ages and capability are using broadband connections of assorted speeds seeking to fulfil tasks using websites.
And somewhat importantly it is the latter scenario rather than the former into which a website seeks to bring value.
The impact of this dynamic on user–testing is that researchers need to work harder than ever before to recreate real–world scenarios in real–world contexts. How users are observed, more than ever, needs to be indicative of how they use websites in the wild.
We are fortunate to work with a client who assists sportspeople with game practice, by more accurately mimicking match play in practice play. The premise upon which their business is built is that the vast majority of game practice (particularly in solo sports such as golf and tennis) is ineffective because it recreates only the technical context of match–play, but crucially not the psychological context, which arguably has a greater bearing on performance. They achieve this by recreating pressure, context, the potential for social embarrassment, competition and finality.
Using technology and scenario planning, they mimic as far as humanly possible what it must feel like to take a penalty at Wembley, serve for Championship point at Wimbledon or putt for the winning birdie at the US Open.
Learning, whether it be from practice or observation, has to be based on experiences as close to the real world environment as possible.
A leading American airline wanted to test its evacuation procedures and to do so simulated a crash in a plane full of test subjects. These subjects intently listened to the various announcements as the plane imitated a crash situation, and patiently and politely carried out the instructions delivered over the speakers in accordance with the wishes of the captain and cabin crew.
The conclusion from the test? That the safety procedures were exactly correct and appropriate for the situation.
The problem with that? The test bore no resemblance to real life.
Aware of the shortcomings of the test, the airline designed another test, which this time fashioned the behavioural motivations of a crash situation, which frankly cannot be created via a simulated crash. Rather these motivations were recreated by getting a plane full of 20–something testosterone–filled American footballers into the cabin, and telling them that on the trigger of various announcements they were to leave the plane according to the instructions. Oh yes, and did we mention that the first man out got $1,000, the next five got $500 and the following ten got $100 each?
The outcomes of the test? Very different indeed to the previous test scenario. The safety procedures fell some way short of dealing with the human emotion involved in a real–life crash.
Testing and research are not possible from ivory towers. Empathy cannot be expressed from a safe distance. Smart designers, committed to human–centred design processes, are prepared to roll up their sleeves, find the right test subjects the hard way, and observe them in real life situations. Because that’s the battleground where the fruits of their labour will stand or fall.