Articles & research papers

Usability Engineering

Wherefore Art Thou O Usability? – Cognitive Lock-in to the Rescue

The author has enjoyed teaching HFI's series of CUA Certification Courses for many years now. Over 20 years of teaching, he got to meet a lot of people who were serious about usability – their organizations paid good money to get them trained. The organizations were "in love" with usability, so to speak. O usability, O usability.

But when he asked the class what their most serious issues were, their replies surprised him. The problem was more "political" than "operational". Their problem was convincing managers that usability offered true value – and should be incorporated into the design life cycle as a routine activity.

Yes, organizations send people to classes. Yes, organizations employ usability specialists. But NO, usability often has no priority and (surprisingly) remains ignored amidst the other pressures of business. Turns out some of these relationship prejudices can be overcome.

Wherefore Art Thou O Usability? – Cognitive Lock-in to the Rescue

Outliers and Luck in User Performance

Summary: 6% of task attempts are extremely slow and constitute outliers in measured user performance. These sad incidents are caused by bad luck that designers can — and should — eradicate.

Outliers and Luck in User Performance

Discount Usability: 20 Years

Simple user testing with 5 participants, paper prototyping, and heuristic evaluation offer a cheap, fast, and early focus on usability, as well as many rounds of iterative design.

Discount Usability: 20 Years

Education in HCI

The HCI Education page is a collection of resources for students and educators interested in Human-Computer Interaction. The following are key resources for HCI Education:

  1. Curriculum: The SIGCHI Curriculum Development Group report Curricula in Human-Computer Interaction.
  2. Affordable Textbook: Clayton Lewis and John Rieman's shareware book Task-Centered User Interface Development.
  3. Readings: Gary Perlman's Suggested Readings in Human-Computer Interaction (HCI), User Interface (UI) Development & Human Factors (HF).
  4. Educator's Mailing List: The SIGCHI Mailing List: CHI-Educators ( CHI-Educators is archived on the Web
  5. Student's Mailing List: The SIGCHI Mailing List: CHI-Students ( CHI-Students is archived on the Web
  6. Program Ratings: Because it is so often asked, I have created: Gary Perlman's Ratings of HCI Education Programs

Education in HCI

Fill Your Portfolio With Stories

On the trail of exploring our next career move, it’s likely we’ll need to show the path we’ve been on. As part of a design team, that usually means displaying our work.

However, if we didn’t make proper arrangements before we took the job, it’s very likely we can’t show much of our work to anyone. Consultants, contractors, and full-time employees are usually covered (in the US at least, but most other places as well) by a "work for hire" agreement, which means that the people we work for own all the work product we produce.

Wireframes, sketches, and other deliverables are not ours to show. If the final design isn’t publicly visible, such as internal application, there might not be any evidence of what we’ve done.

This puts us in an uncomfortable position when it comes time to show our work to a prospective employer. How do we show what we’re capable of when we don’t have access to our work? What can you put into your portfolio when your work is all locked up? The simple answer: Fill your portfolio with stories.

Fill Your Portfolio With Stories

234 Tips and Tricks for Recruiting Users as Participants in Usability Studies

A well-managed recruiting program at an organization allows teams to quickly find quality participants for usability studies.

This free 190-page report from the Norman Nielsen group gives you 234 guidelines on how to set up and manage a recruiting program. It also presents advice on when to outsource to a recruiting agency and when to use in-house recruiting.

Topics covered

  • Learn how to set up and manage a recruiting program to get the right users for usability studies
  • Know when it’s appropriate to outsource to a recruiting agency or use in-house recruiting
  • Planning for recruiting
    • Recruiting criteria
    • Incentives
    • Going to participants vs. having them come to you
    • Screening script and questionnaire
  • Screening and scheduling participants
    • Recruiting on your own
    • Working with an outside recruiting agency
    • Reusing Participants
  • Running the test sessions
    • Ensuring participants’ safety, privacy, and physical comfort
    • Preparing session materials
    • Dealing with unqualified participants
  • Building and maintaining a participant database and recruiting staff
  • Sample scripts and forms

234 Tips and Tricks for Recruiting Users as Participants in Usability Studies (1.3 mb, PDF)

No More No Shows — How to Make Sure Your Research Participants Actually Show Up

“No shows” stink. A few startups recently complained to the author that after diligently planning UX studies and recruiting a great batch of customers, some of their participants just didn’t show up. That’s incredibly frustrating, can be embarrassing in front of the team, and wastes everyone’s time.

Here are a few habits that have dramatically reduced “no shows” at the author's studies:

  1. Avoid scheduling interviews on Mondays or immediately before or after holidays
  2. Offer an incentive that’s big enough to motivate people to show up
  3. Don’t start recruiting too far in advance
  4. Send recruits clearly written confirmation emails
  5. If parking is difficult in your neighborhood, give them specific instructions and assistance
  6. Ensure all communication (phone calls, emails, etc.) to your participants is respectful, professional, and organized
  7. Warn recruits ahead of time that the sessions will be 1-on-1 interviews
  8. Call participants to remind them about their appointments the day before
  9. Elicit several responses from your recruits in the days leading up to the study

No More “No Shows” — How to Make Sure Your Research Participants Actually Show Up

How to Measure Visual Appeal

Is a beautiful website more usable? Psychological literature has discussed, for some time, the "what is beautiful is good" phenomenon. That is, we ascribe positive attributes to things that are more attractive.

This applies to people and likely to products and websites, as well. But does that positive halo also carry over to our impressions of website usability? It's a bit of an open research question, but first, it needs to be considered: how reliable are impressions of website beauty?

In reviewing the literature on rating aesthetics, beauty and visual appeal, researchers often generate their own set of questions and scales to measure these somewhat fuzzy and overlapping constructs. There's nothing wrong with creating new scales, but without any validation, there's a risk that the way the items are worded may generate misleading or less accurate results than those from scales which have been subjected to psychometric validation.

How to Measure Visual Appeal

Pitting Usability Testing Against Heuristic Review

Consider this scenario: You are managing the Intranet applications for a large company. You've spent the last year championing data-driven (re-)design approaches with some success. Now there is an opportunity to revamp a widely used application with significant room for improvement. You need to do the whole project on a limited dollar and time budget. It's critical that the method you choose models a user-centered approach that prioritizes the fixes in a systematic and repeatable way. It is also critical that the approach you choose be cost-effective and convincing. What do you do?

Independent of the method you pick, your tasks are essentially to:

  • Identify the problems
  • Prioritize them based on impact to use
  • Prioritize them based on time/cost benefits of fixing the problems
  • Design and implement the fixes
  • In this situation, most people think of usability testing and heuristic (or expert) review. Empirical evaluations of the relative merit of these approaches outline both strengths and drawbacks for each. Usability testing is touted as optimal methodology because the results are derived directly from the experiences of representative users… The tradeoff is that coordination, testing, and data reduction adds time to the process and increases the overall man- and time-cost of usability testing… As such, proponents of heuristic review plug its speed of turnaround and cost-effectiveness… On the downside, there is broad concern that the heuristic criteria do not focus the evaluators on the right problems (Bailey, Allan and Raiello, 1992). That is, simply evaluating an interface against a set of heuristics generates a long list of false alarm problems. But it doesn't effectively highlight the real problems that undermine the user experience.

    There are many, many more studies that have explored this question. Overall, the findings of studies pitting usability testing against expert review, lead to the same ambivalent (lack of) conclusions.

    Pitting Usability Testing Against Heuristic Review (Link leads to a cached Google page since the original link is dead, good piece of content none the less)

    Severity Ratings for Usability Problems

    Severity ratings can be used to allocate the most resources to fix the most serious problems and can also provide a rough estimate of the need for additional usability efforts. If the severity ratings indicate that several disastrous usability problems remain in an interface, it will probably be unadvisable to release it. But one might decide to go ahead with the release of a system with several usability problems if they are all judged as being cosmetic in nature.

    The severity of a usability problem is a combination of three factors:

    • The frequency with which the problem occurs: Is it common or rare?
    • The impact of the problem if it occurs: Will it be easy or difficult for the users to overcome?
    • The persistence of the problem: Is it a one-time problem that users can overcome once they know about it or will users repeatedly be bothered by the problem?

    Severity Ratings for Usability Problems