Thirty one usability labs from all over the world, all recently built and in active use, take you on a photographic tour and show you how they designed their rooms, what equipment they use, and which software tools are used for data collection and analysis. The usability lab photo gallery covers a wide range of application domains, organizations and technology: from desktop software to mobile applications, from healthcare to military systems, from consumer electronics to office hardware, from focus groups and qualitative testing to physiological measurements and eye tracking, from corporate to educational facilities, from stationary to portable labs. Enjoy the tours!
Tours of usability labs across the world
Summary
Although measuring usability can cost four times as much as conducting qualitative studies (which often generate better insight), metrics are sometimes worth the expense. Among other things, metrics can help managers track design progress and support decisions about when to release a product.
Usability Metrics
Abstract
Tracking the user’s eye-gaze information has been techno-logically possible for several decades. However, systems that track eye-gaze are still very expensive. The exorbitant price tag on commercial systems has resulted in limited use of eye-tracking technology. In this paper we examine the factors which contribute to the high costs of eye-tracking systems. We then propose several techniques and strategies which can be used to reduce the cost of these systems, ultimately resulting in more widespread use of the technology.
Reducing the Cost of Eye Tracking Systems (PDF, 97 kb)
.
The list of questions on this page is a compilation of the questions and answers the author has heard most often about the use of questionnaires in usability engineering and includes answers to the following questions:
- What is a questionnaire?
- Are there different kinds of questions?
- What are the advantages of using questionnaires in usability research?
- What are the disadvantages?
- How do questionnaires fit in with other HCI evaluation methods?
- What is meant by reliability?
- What is meant by validity?
- Should I develop my own questionnaire?
- What’s wrong with putting a quick-and-dirty questionnaire together?
- Factual-type of questionnaires are easy to do, though, aren’t they?
- What’s the difference between a questionnaire which gives you numbers and one that gives you free text comments?
- Can you mix factual and opinion questions, closed and open ended questions?
- How do you analyse open-ended questionnaires?
- What is a Likert-style questionnaire? One with five response choices to each statement, right?
- How can I tell if a question belongs to a Likert scale or not?
- How many response options should there be in a numeric questionnaire?
- How many anchors should a questionnaire have?
- My respondents are continually complaining about my questionnaire items. What can I do?
- What other kinds of questionnaires are there?
- Should favourable responses always be be checked on the left (or right) hand side of the scale?
- Is a long questionnaire better than a short one? How short can a questionnaire be?
- Is high statistical reliability the ‘gold standard’ to aim for?
- What’s the minimum and maximum figure for reliability?
- Can you tell if a respondent is lying?
- Why do some questionnaires have sub-scales?
- How do you go about identifying component sub-scales?
- How much can I change wordings by in a standardised opinion questionnaire?
- What’s the difference between a questionnaire and a checklist?
- Where can I find out more about questionnaires?
Questionnaires in Usability Engineering- A list of FAQs
Abstract
Usability does not exist in any absolute sense; it can only be defined with reference to particular contexts. This, in turn, means that there are no absolute measures of usability, since, if the usability of an artifact is defined by the context in which that artifact is used, measures of usability must of necessity be defined by that context too. Despite this, there is a need for broad general measures which can be used to compare usability across a range of contexts. In addition, there is a need for ‘quick and dirty’ methods to allow low cost assessments of usability in industrial systems evaluation. This chapter describes the System Usability Scale (SUS) a reliable, low-cost usability scale that can be used for global assessments of systems usability.
The System Usability Scale (SUS) (PDF, 150 kb)
Does a PhD pay off financially? The author recently helped conduct the statistical analysis of the UPA 2009 salary survey, and used this opportunity to look into the data to see if he could calculate how much a PhD affects salaries in this profession. The data set contained salary information for a wide range of jobs in the profession—usability engineers, designers, managers and information architects.
How much a PhD is worth- the price of a PhD in the usability profession
Published in the ‘Ergonomics in Design’ journal in 1997. He collected and created this list of 34 thumb rules (given below in order of priority) that were found particularly useful during the design process by colleagues working in the human-computer interface (HCI) design field.
- Know thy user, and YOU are not thy user.
- Things that look the same should act the same.
- Everyone makes mistakes, so every mistake should be fixable.
- The information for the decision needs to be there when the decision is needed.
- Error messages should actually mean something to the user, and tell the user how to fix the problem.
- Every action should have a reaction.
- Don’t overload the user’s buffers.
- Consistency, consistency, consistency.
- Minimize the need for a mighty memory.
- Keep it simple.
- The more you do something, the easier it should be to do.
- The user should always know what is happening.
- The user should control the system. The system shouldn’t control the user. The user is the boss, and the system should show it.
- The idea is to empower the user, not speed up the system.
- Eliminate unnecessary decisions, and illuminate the rest.
- If I made an error, let me know about it before I get into REAL trouble.
- The best journey is the one with the fewest steps. Shorten the distance between the user and their goal.
- The user should be able to do what the user wants to do.
- Things that look different should act different.
- You should always know how to find out what to do next.
- Don’t let people accidentally shoot themselves.
- Even experts are novices at some point. Provide help.
- Design for regular people and the real world.
- Keep it neat. Keep it organized.
- Provide a way to bail out and start over.
- The fault is not in thyself, but in thy system.
- If it is not needed, it’s not needed.
- Color is information.
- Everything in its place, and a place for everything.
- The user should be in a good mood when done.
- If I made an error, at least let me finish my thought before I have to fix it.
- Cute is not a good adjective for systems.
- Let people shape the system to themselves, and paint it with their own personality.
- To know the system is to love it.
Expert Ratings of Usability Maxims (article access requires purchase)
The author desribes A/B tests and multivariate tests (MVT), the difference between them and how one can choose which best fits their needs. A comparison between the techniques is mentioned, taking into consideration the overall use of the testing technique, coding needs, design needs, granularity of results and other considerations.
Landing Page Testing: Choosing Between A/B Or Multivariate Approaches
Heuristic evaluation (Nielsen and Molich, 1990; Nielsen 1994) is a usability engineering method for finding the usability problems in a user interface design so that they can be attended to as part of an iterative design process. Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the “heuristics”).
In general, heuristic evaluation is difficult for a single individual to do because one person will never be able to find all the usability problems in an interface. Luckily, experience from many different projects has shown that different people find different usability problems. Therefore, it is possible to improve the effectiveness of the method significantly by involving multiple evaluators.
How to Conduct a Heuristic Evaluation
This study presents a cross-user usability test approach and analysis technique that extends beyond merely identifying the existence of a usability problem to introducing an empirical basis for identifying the type of usability problem that exists. For experimental purposes, 60 users were tested with three levels of user-competency determined by experience in using: (1) computers, and (2) the tested application. Applying the Tukey honestly significant difference (HSD) test to each test element provided statistical comparison between different experience levels.
Analysis results between experience levels suggested which levels encountered usability problems. The authors demonstrate that statistical calculations of cross-user data can render empirical support for categorizing usability problems.
Cross-user analysis: Benefits of skill level comparison in usability testing (300 kb)