What makes live observation of a usability test a must?

January 28, 2009

Usability_testing_lab_experience_dy Usability testing gains it's strength from direct observation. Like listening to someone describe water vs. putting your hands under a tap to directly experience water, direct live observation of a usability test is more "real".  Moreover, direct observation is critical for accelerating understanding and decision-making.

Live usability testing observation  can not be underestimated. All too often stakeholders (and even clients who hire usability consultants) will skip the test, in favor of reading the report or watching the videos later.

Bottom line: You are wasting a learning opportunity (and your companies money) if you do not attend a usability test for a design of which you have stakeholder say in direction of a site or product.

What makes a usability test live observation so compelling?

Usability testing in-person lets you see, hear and feel your users as they succeed and fail. This "direct experience" makes usability lab testing powerful over other methods (see discussion below).

The dynamics of the user in person with all their human problems (twitches, sneezes, strained vision aka body language) seem to be part of it. The direct contact with the user as you see clearly their facial expressions, hear their sighs and breathing, get a feel for who they are- all seem to factor into a rich observation experience.

First-hand observation of your customer as they "think aloud", can save you a dozen meetings later. Understanding how they make sense out of your design in all the fidelity of their physical presence can leave a powerful impression on you.There's just something very different from "being there" and watching it on video later.

I often meet marketers, developers, project and program managers who have witnessed a usability test. They usually smile when they recall a usability test, with that "I get it" look and they usually want more. Rarely do I meet someone who has seen a real (professional) usability test who said, "oh that's boring, I think I will attend a meeting instead on testing day..." Yet I have worked with clients who do not show up to usability tests.

Top 10 Excuses for NOT Attending a Usability Test

1. Our senior managers can't make it to the usability test. If senior management is involved in decision making, which invariably they are, attending at least 2-3 sessions can potentially save countless hours spent trying to understand issues and back-track on poorly informed decisions.

2. I'll watch the video highlights.Watching the game on TV and being in the stadium are two different things. If your company is paying to host a game, go to the game!

3. We can't bring our whole team out. 1-3 Stakeholders at the minimum should be in attendance.

4. It's too expensive to travel. Consider remote testing to save budgets. Remote testing gives you everything you need minus a few dynamics such as user facial expressions, and the "in-person" factor. Remote testing is great for iterative, rapid testing with a team of familiar players.

5. I'll just read the report. Test reports are important however, adding the context gained from live observation will make the report many times more meaningful.

6. I'm in meetings and won't make all the sessions. If you are managing the project, you should be there for all or almost all sessions. If you are a peripheral stakeholder, try to make 2-3 sessions.

7. I'll wait to hear what the usability experts say. The behavioral interpretation from the usability expert is important, however you will be able to more readily understand decisions by having been to the user sessions.

8. It's not my site (or direct responsibility) that's being tested. You can always learn from user behavior no matter what site they are using.

9.  I've attended usability tests before and I don't need to be there. To keep current and avoid the known problems of generalization and assumption, attend a few sessions and then decide that.

10. I'm not really sure how it works, so it's all a little intimidating... What better way to learn from the experience itself!

Conclusion: Attending and getting as much live Usability Testing exposure as possible is crucial to making better decisions, understanding the context of user problems or issues and for strengthening your user empathy and advocacy skills.

Happy Usability Testing!
Frank Spillers, MS (Usability Consultant)