IT Services Usability - Card Sorting Usability Study

Executive Summary
Jennifer Ward
Christopher Holstrom

From 13 November 2001 to 6 December 2001 IT Services conducted a card sorting usability study. The goals of the study were to determine how users organized the features on the Gateway, how users viewed the significance of Gateway features, and finally what words users use to describe features. Five users participated in the study; one undergraduate, one graduate student, one faculty member, and two staff members. The users' weekly Internet use ranged from 2 hours per week to 70 or more hours per week, with an average use of about 25 hours per week.

Participants in the study were asked to perform two sets of tasks with 46 note card representing current and potential features on the Information Gateway. First, participants grouped note cards according to their perceived importance: critical, very important, important, slightly important, not important. Second, participants grouped note cards according to their own navigational structure. They were asked to clump together features they considered similar and would expect to find together on the Information Gateway.

Navigation Structure

The broad message from this study relates to navigational structure. Users clumped the Gateway features into groups resembling those employed in the current Gateway navigation structure. Looking from top to bottom at the cluster analysis diagram reveals two groups of news and library information features ("About the Libraries" and "Alerts"), a group related to patron information ("Get It"), a group of features dedicated to searching resources ("Find It"), a group for user help ("Help"), a group for site navigation, and a group for services and library information ("Services For" and "About the Libraries"). These groups and the features included in them mapped well to the existing groups on the Gateway.

The one feature that users placed in a significantly different place was "Connecting." Users were not certain of the meaning of "Connecting" and they ended up loosely clumping the feature with news and library monetary issues. This grouping probably resulted from misunderstanding the feature name instead of a firm belief that "connecting" belonged with these features. Also noteworthy is the users' inclination to separate electronic resources from print resources, as evidenced by E-Books, E-Journals, and Image Collections being tightly clumped together. Finally, users grouped Interlibrary Loan, Cascade, Links to Other Libraries, and Reference Tools in a group that seemed to represent looking for information in other places. These features are not particularly well represented and definitely not grouped together on the current Gateway design.

Importance

One question entering this study was whether or not the features users saw as important were prominently featured on the Gateway. The study finds that features users find important are prominently displayed. For the most part the features toward the top and the left of the Gateway received higher scores for importance than the features listed in the less prominent positions near the lower right. Some users suggested that these less important features should not even be on the first page. One feature that had a particularly high importance rating (4.0) also has low prominence in the current Gateway design. Some users wanted to be able to more easily find information about library fines.

Wording

Users found some of the words used to described features ambiguous or confusing. The biggest problem was with Cascade. Some users didn't know the "brand name" at all, while one user thought it only referred to the Bothell campus. This misunderstanding suggests that Cascade is not being used to its full potential because of confusion over the name and a lack of explanation for users. Nearly as confusing for users was the feature "Connecting." It was unclear that this description refers to connecting using the proxy server. Users also found "Starting Points" to be vague. They did not know what to expect on a Starting Points page. When the content of the page was described to users, many suggested changing the name to the popular term FAQ's. Finally, users also found the "News" and "Current Events" features confusing because they did not expect to find either of these on the Gateway.

New Features

Some of the features listed on note cards were not available on the Gateway at the time of the test. These features were proposed based on suggestions from previous survey results. The most popular proposed feature was a Database Selection Wizard (3.6), This feature appealed to all users except for one who shunned any Gateway feature she saw as a crutch. Users considered Online/Live Reference Service (2.25), New Acquisitions and What's On Order (2.2), and Highlighted Library Service (2.0) much less important.