The Journal of Usability Studies (JUS) is a peer-reviewed, international, online publication dedicated to promote and enhance the practice, research, and education of usability engineering.
Its aim is to provide usability practitioners and researchers with a forum to share:
- Empirical findings
- usability case studies (research case studies, not business case studies)
- opinions and experiences (regarding the practice and education of usability engineering)
- Reports of good practices in usability engineering
Journal of usability studies
Summary
Enemies of usability claim that because “the experts disagree,” they can safely ignore user advocates’ expertise and run with whatever design they personally prefer.
Building Respect for Usability Expertise
Abstract
When the authors company chose to adopt an Agile development process for new products, her User Experience Team took the opportunity to adjust, and consequently improve their user-centered design (UCD) practices. Their interface design work required data from contextual investigations to guide rapid iterations of prototypes, validated by formative usability testing. This meant that they needed to find a way to conduct usability tests, interviews, and contextual inquiry— both in the lab and the field— within an Agile framework. To achieve this, they adjusted the timing and granularity of these investigations, and the way that they reported their usability findings.
This paper describes their main adaptations. They found that the new Agile UCD methods produced better-designed products than the ‘waterfall’ versions of the same techniques. Agile communication modes allowed them to narrow the gap between uncovering usability issues and act on those issues by incorporating changes into the product.
Adapting Usability Investigations for Agile User-Centered Design
It is common to think of time-on-task data gathered only during summative evaluations because, during a formative evaluation, the focus is on finding and fixing problems, or at least finding the problems and delivering a report. For a variety of reasons, time-on-task measures often get left out of the mix. This article shows that time-on-task can be a valuable diagnostic and comparative tool during formative evaluations.
Task Times in Formative Usability Tests
Abstract
Traditional in lab usability evaluation has been used as the ’standard’ evaluation method for evaluating and improving usability of software user interfaces (Andre, Williges, & Hartson, 2000). However, traditional in lab evaluation has its drawbacks such as availability of representative end users, high cost of testing and lack of true representation of a user’s actual work environment. To counteract these issues various alternative and less expensive usability evaluation methods (UEMs) have been developed over the past decade. One such UEM is the Remote Usability Evaluation method.
Remote evaluation is a relatively new area and lacks empirical data to support the approach. The need for empirical support was addressed in this study. The overall purpose of this study was to determine the differences in the effectiveness of the two evaluation types, the remote evaluation approach (SREM) and the traditional evaluation approach, in collecting usability data. This study also compared the effectiveness between the two methods based on user type, usability novice users and usability experienced users. Finally, the hypothesis that users, in general, will prefer the remote evaluation approach of reporting to the traditional in-lab evaluation approach was also tested. Results indicated that, in general, the synchronous remote approach is at least as effective as the traditional in lab usability evaluation approach in collecting usability data across all user types. However, when user type was taken into consideration, it was found that there was a significant difference in the high severity negative critical incident data collected between the two approaches for the novice user group. The traditional approach collected significantly more high severity negative critical incident data than the remote approach. Additionally, results indicate that users tend to be more willing to participate in the same approach as the one they participated previously. Recommendations for
usability evaluators for conducting the SREM approach and areas for future research are identified in the study.
Comparative Study of Synchronous Remote and Traditional In-Lab Usability Evaluation Methods (PDF, 400 kb)
Abstract
This paper presents the results of two studies comparing traditional lab-based usability testing and remote web-based usability testing of Web sites. Two sites were tested: an employee benefits site and a financial information site.
The remote tests used an automated technique whereby the users participated from their normal work locations using their normal browser, and there was no real-time observation. Tasks were presented to the user in a small browser window at the top of the screen that was also used to capture their input. Results showed high correlations between lab and remote tests for the task completion data and the task time data. Subjective ratings of the sites did not correlate as well. The most critical usability issues with the sites were identified by both techniques, although each technique also uniquely uncovered other issues. In general, the results indicate that both the lab and remote tests capture very similar information about the usability of a site. Each type of test appears to offer its own advantages and disadvantages in terms of the usability issues it can uncover.
An Empirical Comparison of Lab and Remote Usability Testing of Web Sites (PDF, 1 mb)
A key aspect of successful User-Centered Design (UCD) is evaluating early and throughout the UCD process. The Background: Accessibility & User-Centered Design (UCD) chapter introduces the User-Centered Design process.
This section provides information on incorporating accessibility into the following evaluation methods:
- Importance of Comprehensive Accessibility Evaluation
- Standards Review
- Heuristic Evaluation
- Design Walkthroughs
- Screening Techniques
- Usability Testing
Evaluating for Accessibility
Purpose
This is a case-study comparing remote and lab-based usability testing. Bolt | Peters conducted two parallel Usability Studies on the corporate web site of a fortune 1000 software company in January 2002. Both studies used identical test plans but one was executed in a traditional usability laboratory and the other was conducted remotely, using an online screen-sharing tool to observe behavior.
Remote Testing versus Lab Testing
As more usability practitioners start conducting remote usability testing, there seems to be a demand for some tips and guidelines around this technique. New screen-sharing tools like Breeze, Co-Pilot, and GoToMeeting, and remote usability tools like Ethnio and UserVue, make it easier to conduct moderated remote usability testing. Dealing with video and audio recordings keeps getting simpler as well. But observing people remotely presents a unique set of obstacles, so this is a guide to what we’ve learned from conducting 149 remote studies with 1,213 participants over the last seven years. We can’t get that time back, but hopefully some of what we’ve picked up will be helpful.
Guide to Remote Usability Testing
Examine study results on the benefits and downside of remote usability testing in this two-part article. Pervasive Computing specialists Velda Bartek and Deane Cheatham share the experience gained by conducting a number of remote usability studies using application-sharing technology.
Experience remote usability testing, Part 1
The article provides a context for remote usability testing by detailing and describing the benefits and pitfalls of remote usability evaluations and the application-sharing tools that were evaluated.
Experience remote usability testing, Part 2
The second article describes some of the experiences and lessons learned as the authors planned for and conducted remote usability evaluations for software products.
Experience remote usability testing, Part 1
Experience remote usability testing, Part 2