Why you need to do more usability testing...

September 03, 2009

Users women cultureUsability testing has a hidden benefit that is not apparent unless you do a lot of usability testing. The benefit is cultural absorption. By cultural absorption, I am not referring to culture as in a field study (contextual inquiry, ethnographic study), instead I am talking about user culture.

User culture is what you gain from understanding how users think, walking in their shoes, taking their perspective on issues like:

User culture, situated outside of companies, is usually abstracted, misunderstood and distorted (in that order!). User culture typically runs counter to your development culture. Regular usability testing helps bridge this culture gap.

Development cultures are technological sub-sets of company culture:

  • Dev groups typically think: users know more than they do, that users can do what they normally do (or most of it), that shortcuts are helpful, that features are more helpful than they usually are, for example. 
  • Company culture is typically characterized by big picture thinking CEO's that get user experience but don't terraform their company culture (see Why you can't innovate like Apple), with a few or single grassroots user experience champions. Middle management is usually too busy to get up to speed on usability research and as a result contact with customers is limited or not existent.

Usability testing is usually thought of as offering benefits to understanding how users use your design. That's all good, but one of the under-noticed benefits of usability testing is exposure to user cognition. User cognition is characterized by rules, behaviors, habits, values, beliefs, attitudes and patterns. That is why I consider it and call it a culture. It's unique to your users and it's typically different to the way your organization functions and thinks about its user experience problems.

Usability testing is typically believed to be good on a regular basis because it helps designers and developers vet usability problems. While this is true, it is rarely recognized that iterative, rapid usability testing is good because it puts you in closer contact with users, their culture and their cognitive requirements you design for. This is why I believe you need to do more usability testing!

Happy Usability Testing!
Frank Spillers, MS (Usability Consultant)

Announcing: Experience Capture Studio- new usability testing software (Beta)

June 16, 2009

ECS-Software-Package After 6 months of hard work, we are pleased to announce our latest version of usability testing software. Experience Capture Studio or "ECS" is officially available!

The software dramatically updates our previous LiveLogger product with integrated video viewing and logging notes (in one environment). This provides a more powerful solution allowing for observational logging to be contexted with captured video during a usability test.

This Beta release adds numerous improvements to the overall usability testing software solution: Video file sizes are dramatically smaller and export of video is very quick. The software can import up to 4 video feeds; ideal for mobile or consumer device usability testing. More enhancements aimed at making usability testing logging easier are planned for future releases.

Experience Dynamics & Usability Lab Rental partnered with New Zealand based Intranel to develop the software, utilizing their extensive ethnographic software package experience. 

New Features:


1) Live note taking: Take notes and flag "events" (usability metrics such as error rates) as markers in the live video stream.
2) Playback a session and find or edit flagged events. Metrics such as success rate can be altered in real-time if accidentally recorded or if the user gets it seconds after hitting the fail button.
3) Quick video exporting: Find incidents (observed metrics such as a user getting confused by navigation) and export an instant highlight clip.
4) Instant Reporting: Metrics from the observed test are logged as graphs and exportable tables. Common Excel or JPEG and WMP video formats are supported.

Learn more about key features in Experience Capture Studio

Usability Labs Whatever your Scenario

With the release of ECS, our labs are now suitable for specific usability testing scenarios such as mobile device (iPhone, tablet, BlackBerry, Nokia etc) or medical or healthcare devices (glucose monitoring; patient record-keeping) as well as Web/Software based usability testing.

Explore your usability testing scenario or get in touch and we can talk about how we can help you with your desired usability lab. 

Happy Usability Testing!
Frank Spillers, MS (Usability Consultant)

Formal vs. Informal Usability Tests

April 09, 2007

Wireframe What type of usability tests should you be conducting and why?

Formal Usability Testing

Also called "High Fidelity" usability testing.

Where it gets it's name: Design concepts are typically more finalized. Formal testing can take place in pre-release design, but not always. Websites in their current state (before a re-design) are considered Hi-Fi tests.

How common is it? Very common- even more common than informal usability testing. 

Advantages: Click-able prototypes are easier to follow (for stakeholders). Formal usability testing is often the test of choice for including developers, project managers, executives etc.

Drawbacks: A certain level of HTML "smoke and mirrors" design needs to be created (for web sites). More level of coding complexity is involved in testing software applications- in this case Informal testing is better.

Informal Usability Testing

Also called "Low Fidelity" usability testing or "paper prototype" testing.

Where it gets it's name: Design concepts are tested in draft "wireframe" or unpolished state. No coding or graphic design has occurred at this level. The focus is solely testing the "information architecture" or the "interaction design".

How common is it? Very common, contrary to what most marketers might think. Usability guru, Jakob Nielsen called this testing "Guerilla HCI" to refer to the fast and frequent use of this technique in corporate environments. However, most low-fi testing is usually done by usability engineers, with users behind closed doors or "in the trenches".

Advantages:  Yes! You can get design feedback early on. Feedback can be rapidly acquired in less than two weeks and inserted into the development lifecycle rapidly. A benefit for the quick turn Agile development cycles.

Drawbacks: You can't always test dynamic page level interactions. This will become more of a problem as "Web 2.0" interface design elements become more mainstream (such as fading and hovering elements).

Mixed Fidelity Usability Testing

Mixed-Fi? Mixed fidelity tests are more common for us at Experience Dynamics. We typically test with low-fi concepts that are "taped together" with HTML and some JavaScript. This gives us a rapid "cut and paste" site that can be iterated and refined on the fly. Informal HTML prototypes allow us to prototype and test quickly thereby keeping costs down, but still including stakeholders in our usability labs or remote testing sessions (more on Remote Usability Testing in a future post).

Conclusion: Usability testing can and should be done early on and throughout the product design lifecycle. It is very common for usability practitioners to test concepts that only exist on paper or as static PhotoShop files. Moreover, with basic HTML, a hybrid fidelity can be achieved bringing both the need for speed and user validation to a design.

Happy Usability Testing!
Frank Spillers, MS (Usability Consultant)