Usability Study

Usability studies are aimed at determining the ease of use of a particular device, software, or technology. They are conducted in order to inform product developers of barriers to user interface and design errors. Usability studies take place in controlled conditions or natural settings, where researchers can observe subjects attempting to use a device or technology for its intended purpose. Many factors are taken into account when measuring usability, including: the ease in which novice users can accomplish basic tasks, the length of time it takes users to accomplish basic tasks, the types of mistakes users make and how frequently they make them, and the attitudes users take towards the technology. The technology’s compatibility with various assistive devices, features, and software programs is also a consideration.

• What are the benefits of conducting a usability study?

Usability studies let product developers understand barriers to user interface and design errors before a product goes onto the market. By identifying these barriers early, developers have the opportunity to rectify them. Properly addressing design flaws before the product goes into production or to the market saves time and money, increases customer satisfaction with the product and thereby increases sales and revenues. By building with the widest population of users in mind and ensuring universal access features and compatibility with assistive devices, features, and software, the product can also meet more consumer needs.

• When should I conduct a usability study?

Usability studies allow product developers the chance to see how the intended customers will interact with their product. By doing so, these studies can help developers understand potential barriers their customers would experience after purchasing their product. For assistive technologies this is particularly important. Some examples of usability questions are:

• How easy is it for people to use our product?
• How easy is it for caregivers or service providers to use and integrate our product into a natural setting?
• What mistakes do people make when using our product?
• What problems occur when people use our product?

Usability study questions are answered through direct observation, interviews, surveys, and questionnaires. It is critical to have real users, representative of your consumer base, involved in the usability tests. Choose testers wisely so that you get the widest possible slice of your market. For example, if you plan to market a magnifying software tool to both the blind/low vision education community and to the elderly, then you need to have both consumer groups in your study.

• What are the resources needed to conduct a usability study?

Unlike some rigorous academic forms of research, usability studies can be—and in fact ought to be—conducted by people within your organization. Often in research, it is important to get an outside organization to conduct studies in order to prevent bias or the appearance of bias. With usability studies, however, this is not a concern. The prime purpose of a usability study is not to determine that a product works, but rather what design problems there may be with a product, particularly in the early build stages. Having people who are involved in the design process help implement the usability study is beneficial, since it is important that the information gained from the usability study be directly understood by the design team.

Usability studies are relatively inexpensive. Researchers are encouraged to use no more than five study participants for each round of usability testing. Furthermore, testing itself takes a short period of time to complete. However, as discussed above, it is important that different target audience segments are included as testers, so a thorough study may involve multiple rounds of users.

Elements of a usability study

• Study participant recruitment

Study participants should be recruited from the group of consumers to whom you will be marketing your product. With usability studies, it is important that the study participants enter into the study with the same amount of knowledge that the average consumer would have. That is, using members of the design team as study participants would give you flawed results, since they would have prior experience with the product. Similarly, using study participants who have less experience with your type of technology than the average consumer would give you flawed results as well.

When conducting a usability study, create a protocol or scenario to have study participants attempt to use your product for appropriate, natural tasks. It is important that the study itself best mirrors how your likely customers will use your product in the natural setting.

Remember, usability studies are aimed at pointing product designers to the barriers to use and the product design errors, not to the solutions of those problems. While developing possible solutions may not be difficult, getting the right solution is necessary. It is important to conduct usability tests on the redesigned product in order to ensure that you have solved the problem, and that you haven’t made it worse, or created additional problems. When conducting usability tests on a redesigned product or version, it is important to use a different group of consumers as your study participants in order to get a fresh perspective.

Article: Seven Common Usability Testing Mistakes

• Observation data

Gathering data from observing study participants is an important part of a usability study. Often, study participants are placed in realistic situations where they are asked to perform specific tasks using the product that is being tested. People administering the usability study observe and take notes on how well the study participants are performing, on how long it takes for them to complete specific tasks, and on any difficulties the participants appear to be having. Logging software that records user paths and click patterns can be installed to provide additional data from a user test. Having participants think aloud or talk through what they are doing provides insight into their planned and executed actions. Take the opportunity while you have consumers with you to ask, “Can you think of a different or better way you could use this device?” Consumers may surprise you with setting realities and applications.

• Survey data/questionnaires

Open-ended questions (e.g., “What difficulties, if any, did you have in completing task A?”) can be qualitatively analyzed by looking at trends and themes in the data or tallying up the most frequent topics across all survey respondents.

Examples and additional resources

• Further resources

See these sites for further information on usability testing and design:

User Interface Engineering

Usability Testing: How to plan, design, and conduct effective tests

Usability.gov

Useit.com (Jakob Nielsen)

• A real world example

My School Day Online. With an NCTI Tech in the Works award, this team compared the ease of use for students and teachers of Bridge Multimedia’s My School Day Online scheduler to ease of use of Microsoft’s Outlook scheduler. The study was conducted on site at the Tennessee School for the Blind, and findings and user feedback informed ongoing technology development.