By Subjects Usability Study: User Summaries
User 1: Graduate Student in Paper Science and Engineering
Questionnaire: Spends more than 10 hours a week on the Internet, but has only recently started doing serious research in her field. Uses the Gateway weekly and prefers to research using the Databases and Catalogs.
Typical Task: You are researching the chemical and fiber composition of genetically engineered trees. You want to determine if there is a significant difference between the composition of modified and non-modified trees, and more importantly how any difference might require changes in the paper production process. Find two resources pertinent to your research on the effect of genetic engineering on the paper process.
Tendencies: Didn’t want to use Subject pages. Went to Databases and Catalogs to find Web of Science because she had used them before and heard about them from colleagues. After it was established that she wouldn’t use the Subject pages unless forced, I made her use those pages. She was relatively successful and found some resources she might have otherwise missed.
Interview: She doesn’t use the Subject pages because she went there once and didn’t find what she wanted. She prefers the current hierarchy to an A-Z list; the list is long and requires too much reading. She said she would be more likely to click on “By Subject” than “Browse Subjects.” Didn’t like “Core Resources.” Would prefer “Quick Links.” Didn’t think the three choices for her field were pertinent – preferred to go with what her peers said.
User 2: Sophomore in Computer Science
Questionnaire: She had never been to the Information Gateway and had never done college-level research. She spent more than 10 hours on the Internet weekly.
Typical Task: You are studying biology. You are in Biology 101: General Biology. Once again you have been assigned a research paper. The professor wants you to write a paper on any current topic in biology. Find a resource that will help you write your paper.
Tendencies and Interview: She immediately wanted to leave the Subject pages. She felt uncomfortable and wanted to search on Yahoo, Looksmart, and HotBox. Always searched with the word “information” in her string because she wanted information on a topic. Wanted to find a search feature after I encouraged her to stay in Subject pages. She generally skipped right over “Core Resources” and read the links below…didn’t understand “Core Resources.” Once a database returned one bad article, then she rejected the whole database and started over. Wanted to find resources by her class name, not subject. Wanted student ratings for each resource to help her choose: ****. Multiple times she went to department pages looking for info – including looking for a reasonable specific search term to help her narrow her general research topic. Would prefer “By Topic” to anything with “Subject” in it. Would prefer “Top Picks” or “Best ____” to “Core Resources.”
User 3: Professor in Paper Science and Engineering
Questionnaire: He is a long-time researcher (more than 10 years) who does research for more than 10 hours per week. He is familiar with the Gateway and uses it 2-3 times per week. He uses the Internet for 5-10 hours per week. Favorite Gateway research sections: 1. Databases and Catalogs, 2. UW Catalog, 3. E-Journals.
Typical Task: See User 1
Tendencies: He was very skilled in using the databases and catalogs, but he never uses the Subject pages. Instead he has the major databases in his field bookmarked on his computer so he can go directly to those pages. Loves E-Journals. Has research advantage of knowing major researchers in the field; can search by author name.
Interview: Would only use Subject pages if he felt uncomfortable in the field – like when he has to do chemistry research. Didn’t think “By Subject” or “Browse Subjects” was good, but he couldn’t think of anything better. He skipped over the “Core Resources” largely because they didn’t apply to his sub-discipline. I asked him if he thought his undergrads would use it…he thought the wording was too formal for undergrads.
User 4: Graduate Student in History
Questionnaire: She uses the Gateway daily and has used it for over a year. She spends less than 5 hours per week online and less than one hour per week doing research. Top Gateway resources: 1. UW Catalog, 2. Databases and Catalogs
Typical Task: You are researching the life and culture of Native Americans in the Pacific Northwest. Specifically you want to find information on the impact of European fur traders on tribes in the area. Find two resources on the tribes of the Pacific Northwest and how the influx of European traders changed their way of life.
Tendencies: She is the only user who didn’t seem inclined to leave the “Subject” pages. She was, however, one of the weakest researchers…or maybe she had the hardest tasks. She made peculiar choices and didn’t understand how search engines worked. These problems were revealing; however, as she faithfully stayed with dead ends for a long time (ex. She pursued architecture resources for a long time and to no avail when researching the history of the Great Wall of China. She tried other subject pages before going to History (ex. Went to comparative literature to find out about Irish Monks and Illuminated Manuscripts.)
Interview: She said she would use the Subject pages if she were unfamiliar with the research topic, otherwise she would go right to the databases she knows. She would prefer “Most Used” or “Popular” instead of “Core Resources.” The History page had a more customized layout, so we put “Core Resources” to the right of the rest of the links. She always found a good link before she read to that side of the page, so she hardly noticed those resources. Subjects that she would have used if she saw them: “Literature” and “Chinese Studies.”
User 5: Senior in Biology
Questionnaire: He is familiar with the Information Gateway, as he has used it for more than a year and uses it weekly. Top Gateway resources: 1.E-Journals, 2.UW Catalog, 3. Top 20 Databases. He spends 5-10 hours per week on the web and 1-5 hours per week doing research. He is an experienced researcher for an undergraduate, having done college-level research for between 2 and 5 years.
Typical Task: See User 2
Tendencies: This user never left the Subject pages, largely because he had no need to search elsewhere. He immediately scanned the list to find the subject he was looking for and quickly found a resource in every task. Each research task took him only 2-3 minutes. On the first two tasks he skipped over the Core Resources in favor of Indexes: Finding Articles. He felt he had to guess a little to find a good resource, but he was skilled at that. The third task asked him to do Biology 101 research. He found his trusty BIOSIS under core resources. From that point forward he trusted the listing under Core Resources. When he was out of his subject area (art, business, etc.), he chose a core resource and found what he wanted with ease. Because he finished so quickly, I asked him to do some additional tasks. First, he said he would be taking a graduate seminar in urban planning. When I asked him to find a resource, he scanned the subject list for urban planning. When he didn’t find it he went right to architecture and quickly found resources. Then I asked him to do a more advanced research task in biology. He used the Subject page because he remembered there was an easy link to BIOSIS. He found plenty of resources.
Interview: He said that in general he rarely uses the Subject pages because he most often researches in biology and goes directly to BIOSIS. He did, however, like what he found on the Subject pages when he was outside of his field. He didn’t see much difference between “By Subject” and “Browse Subjects,” as both of them made sense to him. The wording “Core Resources” made sense to him. He knew that meant the most useful resources or the resources that will most likely have what you want. He saw it like a gamble: you can make a pretty strong bet that what you want will be found in the core resources. In that vein he suggested “Best Bets,” but then thought it was too informal. He felt that the alphabetical list was slightly better than the hierarchy because he was more willing to scan for what he was looking for than click through to find it. He didn’t, however, consider it a major difference. We discussed how the undergraduate tasks were maybe a little easy for him because he is an experienced researcher. He said that he would have probably found the resources even if we threw up roadblocks – which suggested to me that an experienced researcher might not have a need for the Subject Pages.
User 6: Sophomore just accepted to English Dept.
Questionnaire: He has used the Gateway for more than a year and uses it on a weekly basis. He has limited his research to the UW Catalog. He easily spends more than 10 hours per week on the web, but is not nearly as experienced as a researcher. He admits that he really hasn't started doing college-level research.
Typical Task: See User 2
Tendencies: This user tended to stay in the subject pages, except for a couple of occasions when he felt lost and desperate and then returned to his more typical research tool: Google. From the A-Z list of subjects, he had no problem finding the specific subject page he needed. He did, however, occasionally have problems finding resources. He succeeded when he was looking for general information on a topic. However, when faced with more specific tasks he struggled because he expected to find a specific link within the subject pages. He did not make the connection that he should first find a business index and then search for Internet Business within that journal. As a beginning researcher, he struggled to find a suitable index for his research. He spent significant time reading the descriptions of the indexes on the "Finding Articles" pages, but still felt he needed more guidance. Once he asked of the indexes, "What is this stuff?" Ironically, he completely ignored the guidance that was aimed directly at him. He never considered using the "Core Resources." He just skipped over these recommended resources because they didn't seem important to him. He said he was drawn to the resources listed below with folder graphics next to them. He perceived these as more important and more promising.
Interview: The user admitted that he had never used the By Subject pages, largely because he does research using Google. When he does come to the Gateway to do research he immediately clicks on the UW Catalog, assuming that the other links under Find It offer pretty much the same information. He preferred the A-Z list to the hierarchy because he wouldn't have known where to find what he was looking for with the hierarchical structure. He probably would have looked somewhere else when he was confronted with the By Subject page. He claimed that he skipped over the "Core Resources" because of their relative position on the page and not because of their name. He did, however, suggest that something like "Librarian Recommendations" would have been more meaningful to him. Finally, he said he preferred "Browse Subjects" slightly more than "By Subject." He noted that "By Subject" matched up grammatically with "Find It," but that none of the other resources on the list matched that way -- so it didn't really help that one matched. The word "Browse" was meaningful to him.
User 7: Undergraduate applying to the Business Department
Questionnaire: Not particularly experienced in using the Internet, the Gateway, or doing research. She spends between 1 and 5 hours a week on the Internet, and has used the Gateway for less than a year. She does use the Gateway multiple times a week. She prefers to use the UW Catalog to do her research.
Typical Task: See User 2
Tendencies: This user was not comfortable doing research. She even admitted at the end of the exercise that she did not like doing research and felt she wasn't very good at it. She did struggle to find information, but not as much as her lack of comfort might have indicated. She spent more time than other users on the Browse Subjects page looking for her topic. She rarely found the subject she wanted right away because she did not think to locate it alphabetically. Instead she scanned until she found something close to her topic (Health for biology, Women's Studies for Virginia Woolf and postmodernism) and went with that. She was quick to back out of her choice when she realized that it wasn't what she wanted. When she did get to a subject page, she never chose the Core Resources. She considered them on a couple occasions, but generally skipped right over them. She most often used the "Finding Articles" link on the subject pages and had reasonable success choosing from that page. She did have trouble searching databases when she was outside of her knowledge domain because she didn't know any search terms. On at least three occasions she grew frustrated and opted to use Yahoo to search for information. She succeeded, but realized that what she found through Yahoo wasn't very academic in nature.
Interview: The interview revealed that she had never used the Subject Pages. She preferred to use search engines to do her research. She did not like or completely understand the term "Core Resources." However, she said that she didn't pay attention to the Core Resources because of where they were located on the page; the long, bold descriptions of the resources; and the folder icons next to the other resources lower on the page. When other names were suggested for Core Resources she preferred "Top Picks" and "Library Recommendations." She did not mind the long lost of subjects on the Browse Subject page and preferred it to the hierarchical structure because felt like the hierarchy might be confusing. She did not have a strong opinion about "Browse Subjects" or "By Subject."
User 8: Senior History Major
Questionnaire: This user has worked in the library for three years and is advanced in his history research, so he is skilled with research and using the Gateway. He uses the Gateway at least weekly and spends 5 to 10 hours per week doing research. He also spends 5 to 10 hours per week on the web. he prefers to use the UW Catalog, but also uses the Top 20 Databases and Databases and Catalogs.
Typical Task: See User 2
Tendencies: This user was very efficient as he moved through his research tasks. his most typical progression through a task was to find the subject he wanted on the Browse Subjects page, select a core resource, and then enter a useful search term to find a resource. He had success following this pattern for most of his research tasks. Occasionally, because of his expertise in a field he would use the UW Catalog or go directly to a database to find information. He also used Google on one occasion. He was looking for information on Renee Descartes, but not finding any because he was spelling the name wrong. Google suggested the correct spelling and gave him the resources he wanted. When looking for information on thermodynamics, he used the Electrical Engineering Subject page. This page had not been modified for the test because users were not expected to use it. He noticed right away that there were no Core Resources, so he left the page right away.
Interview: It seemed very obvious to this user that the core resources were links that were selected as being especially useful for research in a particular field. The main reason for this distinction was their prominence on the page. He claimed that he didn't even read the words "Core resources" because it wasn't necessary. When asked about the term "Core Resources" he thought it made sense and didn't really need to be changed. He also liked the alternatives we have been discussing and thought they might be more user friendly. He preferred the A to Z list of subjects to the hierarchy because he felt that he didn't know what he would find behind a link like "Humanities." He suggested combining the two methods by providing all of the links on one page, but ordering them based on higher-level heading. He said this might cut down on reading of long lists and on uncertain clicking. He felt that "Browse Subjects" was an improvement on "By Subject" and didn't suggest any other alternatives.
User 9: Sophomore Drama Major
Questionnaire: This user has been usign the gateway for more than a year and uses it multiple times each week. She is most comfortable using the UW Catalog, but occasionally uses the Top 20 Databases or Databases and Catalogs. She is on the web over ten hours per week, but is not quite as experienced as a researcher. She claims to spend less than an hour per week doing research.
Typical Task: See User 2.
Note: User 9 was the first user to use the modified Browse Subject pages.
Modifications:
1. "Core Resources" was changed to "Best Bets."
2. "Best Bets" marked with a star graphic.
3. "Best Bets" descriptions no longer in bold.
Tendencies: This user found her way from the "Browse Subjects" page to specific subject pages with very little difficulty. Once she reached the specific subject pages, she was equally efficient and successful at finding a resource. For all but one of her tasks she chose a "Best Bet," often remarking that "Best Bets" and the stars made her think she would find something useful. She only struggled when she was researching something she knew very little about. For example, she found good databases for computer science, but once she was there she had no idea what to enter into the search engine. One of the pages, the English page, had not been updated to include stars and the "Best Bets" wording. When she reached this page, she chose a Core Resource because she had already realized that links located in that position on the page were likely to be successful for her.
Interview: Although She had never used them before her session, she felt like the Subject Pages were incredibly helpful and useful. She thought that finding resources was easier in the Subject pages than in the Top 20 Databases or Databases and Catalofgs, largely because the "Best Bets" made things easy for her. When she was shown the existing Subject pages, she had an immediate negative reaction. She said she would probably leave the page right away if she saw the hierarchical listing. She was confused as to what Subject went under what heading. She much prefered the alphabetical listing because it was easy to scan. We discussed different terms for "Best Bets." She prefered "best Bets" to everything except possibly "Recommended Resources." She slightly prefered "By Subject" to "browse Subjects" because it palced more emphasis on the word "Subject" and made grammatical sense under the "Find It" heading. She felt that the Subject pages should be the second listing under "Fidn it" because they were so helpful.
User 10: Senior English Major (transfer from community college)
Questionnaire: This user was inexperienced using the Information Gateway. He has used it for less than a year and only uses it on a monthly basis. He prefers to use the UW Catalog, E-Journals, or Databases and Catalogs when he does use the Gateway. This user spends 1-5 hours on the web per week and less than one hour per week doing research. He has been doing research for between 1 and 2 years.
Typical Task: See User 2
Tendencies: This user made extensive use of the "Finding Articles" pages within specific subjects, but only used a Best Bet once. He used the Finding Articles pages because he didn't completely trust the Best Bets and felt he could make a better choice on his own. For the most part he succeeded with his own choices, but he spent alot of time on the Finding Articles pages guessing which link to follow and wishing that he had some more guidance available to help him make his choice. On the one occasion that he used a Best Bet (Engineering...very much out of his knowledge area) he found an article quickly and successfully. When he was researching in English he quickly turned to the UW Catalog to find a resource and he was successful. On many occasions he made use of the Expanded Academic Index because he was comfortable using that database based on previous experience.
Interview: He admitted that he did not use "By Subject" pages and probably would not have used subject pages for most of his tasks. Now that he is aware of the subject pages he might consider using them for topics like Engineering or Biology -- where he otherwise would have no idea where to start. When he was shown the existing "By Subject" page he had an instantly negative reaction. He did not like th short list of general topics. He much preferred the A-Z list. Despite not using them much, he thought adding the Best Bets was an improvement on the existing subject pages. He liked the stars next to the Best Bets and stated that they drew his attention. He did not, however, like the term "Best Bets." He thought it was too informal and he did not trust links with that label. He prefered "Core Resources" even though he realized that it was a little "stuffy." He also liked the alternative "Library Recommendations," but definitely not "Top Picks" for the same reason he rejected "Best Bets." Finally, he was asked about the difference between the terms "By Subject" and "Browse Subjects." He much preferred "Browse Subject" because it gave a clear indication of how he would find information if he followed that link.
|