Examine study results on the benefits and downside of remote usability testing in this two-part article. Pervasive Computing specialists Velda Bartek and Deane Cheatham share the experience gained by conducting a number of remote usability studies using application-sharing technology.
Experience remote usability testing, Part 1
The article provides a context for remote usability testing by detailing and describing the benefits and pitfalls of remote usability evaluations and the application-sharing tools that were evaluated.
Experience remote usability testing, Part 2
The second article describes some of the experiences and lessons learned as the authors planned for and conducted remote usability evaluations for software products.
Experience remote usability testing, Part 1
Experience remote usability testing, Part 2
For product suppliers
- Do: Explain the CIF to your development team. Make sure they understand how this is different from a formative evaluation and why the report looks like it does.
- Do: Use the CIF for the benchmark studies of your product. The CIF is designed for these types of summative studies.
- Do: Encourage your customers to do their own testing of your product, using their specific user populations and tasks.
- Do: Consider hiring an external firm to conduct your product’s benchmark study. Doing so adds credibility to the results with your customers.
- Do: To avoid having two reports for internal vs. external audiences, isolate sections of the report that are for internal use only vs. those for product consumers.
- Do: Make the connection between the measures in the CIF and success for the business case of the product, both for suppliers and customers.
- Do: Get the necessary management /legal approvals to release the CIF to customers.
- Don’t: Use the CIF for iterative usability studies. The CIF is not designed for studies whose aim is to find and fix problems with a product feature.
- Don’t: Rely on a single study of your product using the CIF at the end of the product development cycle. Because the CIF is not designed to find problems during development, you need to do iterative usability studies throughout the product’s creation to find and fix problems.
More information at Top ten things you should know about the CIF.
Produced by Industry Usability Reporting (IUSR)project of the National Institute of Standards and Technology (NIST), the overall purpose of the Common Industry Format (CIF) for Usability Test Reports is to promote incorporation of usability as part of the procurement decision-making process for interactive products. Examples of such decisions include purchasing, upgrading and automating. It provides a common format for human factors engineers and usability professionals in supplier companies to report the methods and results of usability tests to customer organizations.
Common Industry Format for Usability Test Reports V1.1
Designers are continuously engaged in elicitation of user data, prior to new product conceptualisation, as well as during the testing of prototypes. When such user data collection happens across different cultures, data is often collected and analysed ignoring the rich qualitative cues embedded in non-verbal communications such as gestures.
In cross-cultural situations, gestures can yield additional information from the user. This paper analyses non-verbal kinesthetic cues of users engaged in a verbal protocol-based testing situation.
Influence of Cultural Background on Non-verbal Communication in a Usability Testing Situation
OK–Cancel or Cancel–OK?
By Jakob Nielsen
Summary
Should the OK button come before or after the Cancel button? Following platform conventions is more important than suboptimizing an individual dialog box
OK–Cancel or Cancel–OK?
OK and Cancel Buttons- What’s the Right Order?
By Tom Tullis
Takeaways
- Avoid using grouped buttons labelled as “OK” and “Cancel” on the web. There’s too great a chance that your users may have a different expectation from yours about which button is which. If they’re in a hurry, they might accidentally choose the wrong button.
- It’s probably better to use buttons that are visually separated.
- With the buttons visually separated, putting the action to continue (e.g., OK, Save, Submit, etc) on the right is more likely to match your users’ expectations.
OK and Cancel Buttons- What’s the Right Order?
This NASA site provides a guide to color design for information visualization. It includes:
- a step by step process for designing color usage in complicated interface graphics
- two detailed examples of design of aerospace displays
- a new color selection tool to support the recommended design process
- information about color usage standards and guidelines
- information about applied color science, and reference resources
Aerospace graphics get special attention, but much of the information should be useful for other color graphics as well.
Using Color in Information Display Graphics
You’ve spent several days setting up a usability test, recruiting the participants and running it. Then you’ve poured over the data. What next?
If you are writing up your dissertation then skip this paper. You’ll need to follow the rules of your university and every detail is likely to be important. But if you are doing usability testing as part of user centered design within a business setting, then there are many ways that you can communicate the results.
This paper looks at reports and then considers presentation and observation as alternatives to reports.
Better Reports: How to Communicate the Results of Usability Testing (PDF, 250kb)
Interfaces is a quarterly magazine with features, events, reviews and jobs. Interfaces is available in print for members and as downloads for everyone else.
Interfaces Magazine
A comparison of QUIS, CSUQ, SUS, product reaction cards and other questionnaires used by Fidelity Investments usability group.
Conclusion
- One of the simplest questionnaires studied, SUS (with only 10 rating scales), yielded among the most reliable results across sample sizes.
- It was also the only one whose questions all address different aspects of the user’s reaction to the website as a whole.
- For the conditions of this study, sample sizes of at least 12-14 participants are needed to get reasonably reliable results.
A Comparison of Questionnaires for Assessing Website Usability (PDF, 170 kb)
.
The Research-Based Web Design & Usability Guidelines (Guidelines) were developed by the U.S. Department of Health and Human Services (HHS), in partnership with the General Services Administration. This edition contains 209 guidelines.
The guidelines were developed to assist those involved in the creation of Web sites to base their decisions on the most current and best available evidence. The guidelines are particularly relevant to the design of information-oriented sites, but can be applied across the wide spectrum of Web sites.
Research-Based Web Design & Usability Guidelines (PDF)