This chapter is from the book
This chapter is from the book
This chapter is from the book
Data Gathering and Testing Techniques
Usability data gathering and testing are some of the most valuable tasks your usability team can do. While the phrase "run a usability test" is a general term, keep in mind that there is not a single type of usability test—there are many different types. For example, there are tests of branding, early paper prototyping tests for conceptual design, and later tests on robust working prototypes. You must select the type of test needed for where you are in your development process and then create the correct type of test questionnaires to support the testing.
You can save a lot of money and time by having an initial set of questions and then customizing them as needed for each test. Having a list of standard tests helps to quickly plan the testing, but each test needs its own set of forms, such as video consent forms, facilitator scripts, task instructions, and so on. Defining and creating predesigned templates can save countless hours. While no template for a given type of test works for all situations, there is certainly value in having a template as part of your infrastructure. Some example template forms include those listed below.
-
The screener is an essential questionnaire used to select participants for a study. The screener can help eliminate participants who are too sophisticated or too inexperienced. In some cases, a template can be developed and used repeatedly for each study that will access those types of users, though typically the template must be modified for each test.
-
Usability testing routine forms are a family of forms you need when running usability tests. They are not very exciting, but they are quite necessary. For example, you must have an informed consent form to get the participant's agreement to participate. Without this form in place, you are in violation of ethics in human research and can be sued. You may also need demographics forms and forms to acknowledge compensation.
As mentioned, there is more than one type of usability test. Below are descriptions of different tests. Which one you use depends on what questions you are trying to answer.
-
Brand perception tests let you see how the user perceives the current Web site or application. One version of this test is for a single design, and a variant of the test can also be run as a comparison with competitors' designs. Another version of this type of test involves the selection of the best among suggested designs. This test can be conducted with designs from different graphic artists or even different agencies. Regardless of the scenario, the questionnaire for this test must be customized to reflect the company's target brand values. You need to pick the brand values you are interested in testing. What brand values are you looking for, and which do you want to make sure to avoid? Trendy, warm, friendly, sophisticated, "tech-y"...you need to customize the questionnaire to get at the data you are interested in.
-
If you ask users if they want a given function, they almost always say yes. If you give them a list of potential functions and ask them to rate how important they are, they rate most as very important. But if you give them a list of possible functions and say they can have only three, you get interesting results. This test, called a functional salience test, is a great way to identify the relative importance of functions.
-
A test of affordance determines whether users can tell what they can select on a page. You simply give users a printed copy of a page and tell them, "Circle the items you think you can select and click on." You will see if there are selectable items that users cannot tell are selectable. You will also see if there are items that are not selectable that make users think they can select them.
-
"Think aloud" tests consist of a whole family of tests where the user is told to do a series of tasks, which are observed. Users are asked to read out loud as they work and tell the facilitator what they are thinking. This is a great way to find problems in a design. You can also estimate how long it will take users to complete tasks.
-
The card sort test is a useful method if you are trying to find how users categorize the topics in a Web site or application. You create stacks of cards with one item on each card, and then the participants group the cards in a way that makes sense to them. Software can help collect and analyze the groupings used by different participants. The software uses cluster analysis and gives results that can guide the information structure of the design.3
-
While the card sort test can help guide the design, you can use the reverse card sort method to check whether the design worked. You give the participants a list of items and see if they can figure out where to go to find them. If they can find them, the navigational structure is self-evident.
-
Subjective ratings are a large family of tests that allow users to describe how they feel about your site or application. They decompose or break down the perceptions to allow you to more easily track the cause of problems. For example, you might find that people love the colors but feel that the site is very slow. These findings need to be carefully considered. You might find lots of users saying they want a search facility, but this may actually indicate that there is a problem with the structure of the site. The stated desire for a search facility is often just a symptom of being lost in a poor navigational structure.