Evaluate the usability gains (if any) afforded by the redesign of the individual pages which make up the “Get It” section of the Information Gateway.
|
Product |
Problem
Statements |
|
Information Gateway |
|
|
“Get It” Section |
|
|
“Borrowing/Delivery” Sub-Section |
|
|
Other “Get It” Sub-Section Pages |
|
Participants selected for this study will come from the
following groups: students, faculty and staff. All participants will have at
least a High School education and will ideally be either pursuing or have
completed an advanced degree. Participants will be intellectually curious and
proactive when it comes to finding information. Participants are expected to
have some experience with the Information Gateway and with web searching.
The test will consist of a performance test made of four sections:
Each participant will be personally greeted by the test monitor and made to feel comfortable and relaxed. The participants will be asked to fill out a short pre-study questionnaire which gathers background information. (See pre-test questionnaire)
Participants will each receive a scripted, verbal introduction and orientation which will explain the purpose and goals of the test. They will also be assured that the website is what is being tested, not themselves. Participants will then be introduced to the equipment and told that they will be recorded with video/audio recording, and by tracking mouse clicks and movement. They will then be asked to sign standard consent forms, if they haven’t already.
The performance test consists of a series of tasks that the participants will carry out while being observed. These tasks will be conducted in the following manner:
a. Participant will be handed a script, which they will be asked to read out loud, detailing the task to be accomplished. They will then attempt that task.
b. After concluding the tasks, the participant will complete a short survey to gauge their responses to the tasks. (See post-test questionnaire)
During the performance test the monitor will make notes on elapsed time and participant errors. The monitor will also make note of any unusual circumstances that may not be picked up by the recording devices.
Each participant will be debriefed by the test monitor, with the session being audiotaped. The topics of the questions will be:
· Participant’s overall agreement with the Information Gateway design.
· Participant’s comments and recommendations regarding the Information Gateway design.
·
Participant’s responses to probes from the
monitor about specific errors or problems that occurred during the test.
Task
1
|
Description
|
|
Task |
Make a guess as to what resources would be available
behind key-term page links. |
|
Starting State |
Paper questionnaire |
|
Successful Completion Criteria |
Survey completed |
|
Benchmark |
7 minutes |
|
Script |
Please take a look at the following words and phrases. Imagine that each of these words is a hyperlink to resources on the libraries website. Write down in the space provided what you would expect to find behind each link.
|
Task
2
|
Description
|
|
Task |
Label a webpage according to its content |
|
Starting State |
Image of web page on screen with paper survey |
|
Successful Completion Criteria |
Survey completed |
|
Benchmark |
4 minutes |
|
Script |
You will be shown two images
of two different web pages. If you were asked to give a title to each of
these pages, what would it be? Please write your answer in the boxes on the
right. |
Task
3
|
Description
|
|
Task |
Renew checked out books. Reset pin number. |
|
Starting State |
Information Gateway Home |
|
Successful Completion Criteria |
Test subject locates page |
|
Benchmark |
3 Minutes |
|
Script |
You've made it through to the
end of fall quarter and are now enjoying a vacation skiing in |
Task
4
|
Description
|
|
Task |
Locate fine-paying options |
|
Starting State |
Information Gateway Home |
|
Successful Completion Criteria |
Test subject locates page |
|
Benchmark |
1 Minute |
|
Script |
Your mind must have been on
the slopes when you renewed those books! When you return to school you find a
notice stating that you owe fines for one of the items and your registration
for the next quarter is blocked until it’s paid. What options are available
for paying fines? |
Task
5
|
Description
|
|
Task |
Identify how resources would be acquired (ideally by using
the resources in “Get It”) |
|
Starting State |
Information Gateway Home |
|
Successful Completion Criteria |
Test subject identifies how book is acquired or locates “Get a Book” status definition page |
|
Benchmark |
5 Minutes |
|
Script |
An assignment for Professor West’s Russian Studies class requires that you provide a short summary of each of the following items, all of which are in the UW Libraries online catalog. Write down how you would obtain each of the following items in the spaces below. You may return to the website to search for more information if necessary. a) Steel
sector restructuring in b) Restifo, Frank. 2001. Guide to Soviet Bibliographies. Library of Congress, General Reference and Bibliography Division. 1950. [View Image 3] c) Pale
fire: a novel. Nabokov, Vladimir Vladimirovich. 1962. [View Image 4] |
Task
6
|
Description
|
|
Task |
Locate “Get library notices via email” page |
|
Starting State |
Information Gateway Home |
|
Successful Completion Criteria |
Test subject locates page |
|
Benchmark |
1 Minute |
|
Script |
After searching the catalog,
you found a few more items that you’ve requested be transferred from the
Bothell library to the Odegaard circulation desk and held for you. You have a
paper due in two weeks, need those books ASAP, and don’t want to wait for a
notice in the mail to let you know when they’re available. Is there any way
to know if those items are at Odegaard yet? Is there a way to get notices
faster? |
Task
7
|
Description
|
|
Task |
Locate information for extension students. |
|
Starting State |
Information Gateway Home |
|
Successful Completion Criteria |
Test subject locates page |
|
Benchmark |
1 Minute |
|
Script |
Now imagine that you’re new to
|
Task
8
|
Description
|
|
Task |
Locate card activation page/link. |
|
Starting State |
Information Gateway Home |
|
Successful Completion Criteria |
Test subject locates page |
|
Benchmark |
1 Minute |
|
Script |
As part of your new student
orientation, you’ve been told that you can access library databases and
resources from home, but you first have to have your library card activated.
What are the options for activating your card? |
The test environment consists of a separate office with a computer station and two observer chairs. The computer is set up with Internet Explorer as the browser and sauk.lib.washington.edu as its home page. Monitoring the test is a video camera mounted behind and to the right of the test subject and focused on the computer monitor. Monitoring mouse movement and keystrokes is Camtasia.
The test monitor will sit in the room with each participant
while conducting the test. The test monitor will not volunteer help unless the
participant is seriously off-course and will only answer questions if they
don’t threaten to color the test results. The test monitor will be on hand in
case of equipment failure or other problems. Participants will be instructed to
use the test server home page, and in particular the “Get It” section, to
answer test questions.
Performance
Preference
Quotes
The report will include the following sections: