44 iNFOrMAtiON tecHNOlOGY AND liBrAries | MArcH 2011 Jennifer Emanuel Usability of the VuFind Next-Generation Online Catalog VuFind incorporates many of the interactive web and social media technologies that the public uses online, including features from online booksellers and commer- cial search engines. The VuFind search page is simple, containing only a single search box and a dropdown menu that gives users the option to search all fields or to search by title, author, subject, or ISBN/ISSN (see figure 1). To combine searches using Boolean logic or to limit to a particular language or format, the user must use the advanced search feature (see figure 2). The record- results page displays results vertically, with each result containing basic item information, such as title, author, call number, location, item availability, and a graphical icon displaying the material’s format. The results page also has a column on the right side displaying “facets,” which are links that allow a user to refine their search and browse results using catalog data contained within the result set (see figure 3). VuFind also contains a vari- ety of Web 2.0 features, such as the ability to tag items, create a list of favorite items, leave comments about an item, cite an item, and links to Google Book previews and extensive author biographies data mined from the Internet. Corresponding to the beginning of the VuFind trial at UIUC, the university library purchased reviews, synopses, and cover images from Syndetic Solutions to further enhance both VuFind and the existing WebVoyage catalog. An additional appealing aspect of VuFind was its speed; the CARLI installation of WebVoyage is slow to load and is prone to time out while conducting searches. The UIUC library first provided VuFind (http:// www.library.illinois.edu/vufind) at the beginning of the 2008 fall semester and expected it to be trialed through the end of the spring semester 2009. Use statistics show that throughout the fall semester (September through December), there were approximately six thousand unique visitors each month, producing a total of more than thirty-eight thousand visits. Spring statistics show use averaging more than ten thousand visitors a month, an increase most likely from word-of-mouth. Librarians at both UIUC and CARLI were inter- ested in what users thought about VuFind, especially in relation to the usability of the interface. With this in mind, the library launched several forms of assessment during the spring semester. The first was a quantita- tive survey based on Yale’s VuFind usability testing.3 The second was a more extensive qualitative usability test that had users conducting sample searches in the interface and telling the facilitator their opinions. This article will discuss the hands-on usability portion of this study. Survey responses that support the results pre- sented herein will be reported in a separate venue. While this article only discusses VuFind at a single institution, it does offer a generalized view of next-generation catalogs and how library users use such a catalog compared to a traditional online catalog. The VuFind open–source, next-generation catalog system was implemented by the Consortium of Academic and Research Libraries in Illinois as an alternative to the WebVoyage OPAC system. The University of Illinois at Urbana-Champaign began offering VuFind alongside WebVoyage in 2009 as an experiment in next generation catalogs. Using a faceted search discovery interface, it offered numerous improvements to the UIUC catalog and focused on limiting results after searching rather than limiting searches up front. Library users have praised VuFind for its Web 2.0 feel and features. However, there are issues, particularly with catalog data. V uFind is an open–source, next-generation catalog overlay system developed by Villanova University Library that was released to the public as beta in 2007 and version 1.0 in 2008.1 As of July 2009, four institu- tions implemented VuFind as a primary catalog interface, and many more are either beta or internally testing it.2 More information about VuFind, including the technical requirements and compatible OPACs, is available on the project website (http://www.vufind.org). In Illinois, the state Consortium of Academic and Research Libraries in Illinois (CARLI) released a beta installation of VuFind in 2008 on top of its WebVoyage catalog database. The CARLI installation of VuFind is a base installation with minor customizations to the CARLI catalog environment. Some libraries in Illinois utilize VuFind as an alternative to their online catalog, including the University of Illinois at Urbana-Champaign (UIUC), which currently adver- tises VuFind as a more user friendly and faster version of the library catalog. As a part of the evaluation of next- generation catalog systems, UIUC decided to conduct hands-on usability testing during the spring of 2009. The CARLI catalog environment is very complex and comprises 153 member libraries throughout Illinois, rang- ing from tiny academic libraries to the very large UIUC library. Currently, 76 libraries use a centrally managed WebVoyage system referred to as I-Share. I-Share is com- posed of a union catalog containing holdings of all 76 libraries as well as individual institution catalogs. Library users heavily use the union catalog because of a strong culture of sharing materials between member institutions. CARLI’s VuFind installation uses the records of the entire union catalog, but has library-specific views. Each of these views is unique to the member library, but each library uses the same interface to view records throughout I-Share. Jennifer emanuel (emanuelj@illinois.edu) is digital Services and reference librarian, university of illinois at urbana-champaign. usABilitY OF tHe VuFiND Next-GeNerAtiON ONliNe cAtAlOG | eMANuel 45 not simply find them.6 As a result, the past five years have been filled with commercial OPAC provid- ers releasing next-generation library interfaces that over- lay existing library catalog information and require an up-front investment by libraries to improve search capabilities. As these systems are inherently commercial and require a significant investment of capital, several open–source, next-gener- ation catalog projects have emerged, such as VuFind, Blacklight, Scriblio, and the eXtensible Catalog Project.7 These interfaces are often developed at one institution with their users in mind and then modified and adapted by other institutions to meet local needs. However, because they can be locally customized, libraries with sig- nificant technical expertise can have a unique interface that commercial vendors cannot compete against. One cannot discuss next-generation catalogs without mentioning the metadata that underlie OPAC systems. Some librarians view the interface as only part of the problem of library catalogs and point to cataloging and metadata practices as the larger underlying problem. Many librarians view traditional cataloging using Machine-Readable Cataloging (MARC), which has been used since the 1960s, as outdated because it was devel- oped with nearly fifty-year-old technology in mind.8 However, because MARC is so common and allows cataloging with a fine degree of granularity, current OPAC systems still utilize it. Librarians have developed additional cataloging standards, such as Dublin Core (DC), Metadata Object Description Schema (MODS), and Functional Requirements for Bibliographic Records (FRBR), but none of these have achieved widespread adoption for cataloging printed materials. Newly devel- oped catalog projects, such as eXtensible Catalog, are beginning to integrate these new metadata schemas, but currently others continue to use MARC.9 Many librarians also advocate to integrate folksonomy, or user tagging, into library catalogs. Folksonomy is used by many library websites, most notably Flickr, Delicious, and LibraryThing, each of which store user-submitted content that istagged with self-selected keywords that allow for easy retrieval and discovery.10 VuFind integrates tagging into individual item records ■■ Literature Review Librarians have complained about the usability of online catalogs since they were first created.4 When Amazon.com became the go-to site for books and book information in the early 2000s, librarians and their users began to harshly criticize both OPAC interfaces and metadata standards.5 Ever since North Carolina State University announced a partnership with the commercial-search corporation Endeca in 2006, librarians have been interested in the next generation of library catalogs and more broadly, discovery systems designed to help users discover library materials, Figure 1. VuFind Default Search Figure 2. VuFind Advanced Search Figure 3. Facets in VuFind 46 iNFOrMAtiON tecHNOlOGY AND liBrAries | MArcH 2011 searching the library’s online catalog and were eager to see changes made to it. The test used was developed from a statewide usabil- ity test of different catalog interfaces usedin Illinois. The test was adapted using the same sample searches, but was customized to the features and uses of VuFind (see appendix). The VuFind test was similar to the original test to allow a comparison of other catalog interfaces to VuFind for internal evaluation purposes. I designed the test to allow subjects to perform a progressively compli- cated series of sample searches using the catalog while the moderator pointed out various features of the catalog interface. Subjects were also asked what they thought about the search result sets and their opinions of the interface and navigation; they also were asked to perform specific tasks using VuFind. The tasks were common library-catalog tasks using topics familiar at undergradu- ate–level students. The tasks ranged from a keyword search for “global warming” to a more complicated search for a specific compact disc by the artist Prince. The tasks also included using the features associated with cre- ating and using an account with VuFind, such as adding tags and creating a favorite items list. Through complet- ing the test, subjects got an overview of VuFind and were then asked to draw conclusions about their experience and compare it to other library catalogs they have used. The tests were performed in a small meeting room with one workstation set up with an install of the Morae software, a microphone, and a web camera. Morae is a very powerful software program developed by TechSmith that records the screen on which the user is interacting with an interface, as well as environmental audio and video. Although the study did not utilize all the features of the Morae software, it was invaluable to the researcher to be able to review the entire testing experience with the same detail as when the test actually occurred in person. The study was carried out with the researcher sitting next to the workstation asking subjects to perform a task from the script while Morae recorded all of their actions. Once all fifteen subjects completed the test, the researcher watched the resulting videos and coded the answers into various themes on the basis of both broad subject catego- ries and individual question answers. The researcher then gathered the codes into categories and used them to fur- ther analyze and gain insight into both the useful features of and problems with the VuFind interface. ■■ Analysis Participants generally liked VuFind and preferred it to the current WebVoyage system. When asked to choose which catalog they would rather use, only one person, a faculty member, stated he would still use WebVoyage. This faculty but does not pull tags from other sources; rather, users must tag items individually. Additionally, next-generation catalogs offer a search mechanism that focuses on discovery rather than simply searching for library materials. Users, accustomed to new ways of searching both on the Internet and through com- mercial library indexing and abstracting databases, now search in a fundamentally different style than they did when OPACs first became a part of library services. The online catalog is now just one of many tools that library users use to locate information and now covers fewer resources than it did ten to fifteen years ago. Library users are now accustomed to using a single search box, such as with Google; they also use nonlibrary online tools to find information about books and no longer view library cata- logs as the primary place to look for books.11 As users are no longer accustomed to using the con- trolled language and particular searching methods of library catalogs because they have moved to discover- ing materials online, libraries must adapt to new way of obtaining information and focus not on teaching users how to locate library materials, but give them the tools to discover on their own.12 VuFind is one option among many in the genre of next-generation or discovery-catalog tools. ■■ Methods The study employed fifteen subjects who participated in individual, hands-on usability test sessions lasting an average of thirty minutes. I recruited volunteers though several methods, including posting to a university faculty and staff e-mail discussion list, an e-mail discussion lists aimed toward graduate students, and flyers in the under- graduate library. All means of recruitment stated that the library sought volunteer subjects to perform a variety of sample searches in a possible new library catalog inter- face. I also informed subjects that there was a gift card as a thank you for their time. All subjects had to sign a human subjects statement of informed consent approved by the University of Illinois Institutional Review Board. I sought a diverse sample, and therefore accepted the first five volunteers from the following pools: faculty and staff, graduate students, and undergraduate students. I felt that these three user groups were distinct enough to war- rant having separate pools. The number of five users in each group was chosen because of Jakob Nielsen’s statement that five users will find 85 percent of usability problems and that fifteen users will discover all usability problems.13 Although I did not specifically aim to recruit a diverse sample, the sample showed a large diversity in areas including age, library experience, and academic discipline. All subjects stated they had some experience usABilitY OF tHe VuFiND Next-GeNerAtiON ONliNe cAtAlOG | eMANuel 47 though there were questions as to how results were deemed relevant to the search statement as well as how they were ranked. Participants were then asked to look at the right sidebar of the results page, which contains the facets. Most users did not understand the term “facets,” with faculty and staff understanding the term more than graduate and undergraduate students did. One faculty member who understood the term facet noted that “facets are like a diamond with different sides or ways of viewing something.” However, when asked what term would be better to call the limiting options other than facet, several users suggested either calling the facets “categories” or renaming the column “Refine Search,” “Narrow Search,” or “Sort Your Search.” Participants were then asked to find how to see results for other I-Share libraries. Only two faculty members found I-Share results quickly, and just half of the remain- ing participants were able to find the option at all. When asked what would make that option easier to find, most said they liked the wording, but the option needed to stand out more, perhaps with a different colored link or bolder type. Two users thought having the location integrated as a facet would be the most useful way of seeing it. Participants, however, quickly took to using the facets, as they were asked to use the climate change search results to find an electronic book published in 2008. No user had problems with this task, and several remarked that using facets was a lot easier than limiting to format and year before searching. The next task for participants was to open and exam- ine a single record within their original climate change results (see figures 4 and 5). Participants liked the layout, including the cover image with some brief title infor- mation, and a tabbed bar below showing additional information, such as more detailed description, holdings information, a table of contents, reviews, comments, and a link to request the item. Several users remarked that they liked having information contained under tabs, but VuFind organized each tab as a new webpage that made going back to previous tabs or the results page cumber- some. The only problem users had with the information contained within the tabs was the “staff view,” which contained the MARC record information. Most users looked at the MARC record with confusion, including one graduate student who said, “If the staff view is of no use to the user, why even have it there?” One other useful feature that individual records in VuFind contain is a link to an overlay window containing the full citation infor- mation for the item in both APA and MLA formats. Users were able to find this “Cite This” link and liked having that information available. However, several participants noted that citation information would be much more ben- eficial if it could be easily exported to Refworks or other bibliographic software. The next several searches used progressively higher-level member thought most of his searches were too advanced for the VuFind interface and needed options that VuFind did not have, such as limiting a search to an individual library or call number searching. This user did, however, specify that VuFind would be easier to use for a fast and simple search. Other users all responded very favorably to VuFind, liking it better than any other online catalog they have used, with most stating that they wanted it as a permanent addition to the library. The most common responses to Vufind were that the layout is easier on the eyes and displayed data much better than the WebVoyage catalog; there were no comments about actual search results. Several users stated that it was nice to be able to do a broad search and then have all limiting options presented to them as facets, allowing users to both limit after search- ing and letting them browse through a large number of search results. One user, an undergraduate student, stated she liked VuFind because it “was new” and she always wants to try out new things on the Internet. The first section of the usability test asked users to examine both the basic and advanced search options. Users easily recognized how the interface functioned and liked having a single search box as the basic interface, noting that it looked more like a web search engine. They also recognized all of the dropdown menu options and agreed that the options included what they most often searched. However, four users wanted a keyword search. Even though there is not a keyword search in WebVoyage and there is an “all fields” menu option, participants seemed to think of the one box search universally as a keyword search and wanted that to be the default search option. One participant, an international graduate stu- dent, remarked that keyword is more understood by international students than the “all fields” search because, internationally, a field is not a search field but a scholarly field such as education or engineering. In the advanced search, all users thought the search options were clear and liked having icons to depict the var- ious media formats. However, two users did remark that it would be useful to be able to limit by year on the advanced search page. The advanced search also is where the user can select one of seven languages, all of which are consid- ered western languages, including Latin and Russian. Two users, both international graduate students, stated that more languages would be beneficial, especially Asian and more Slavic languages. The University of Illinois has sepa- rate libraries for Asian and Slavic materials, and these two participants said it would be useful to have search options that include the languages served by the libraries. The first task that participants were asked to do was an “all fields” search for “climate change.” They were instructed to look at the results page and an individual record to give feedback as to how they liked the layout and what they thought of the search results. Upon looking at the results, all participants thought they were relevant, 48 iNFOrMAtiON tecHNOlOGY AND liBrAries | MArcH 2011 to items in which James Joyce is the author, no par- ticipant had any problems, though several pointed out that there were three fac- ets using his name—Joyce, James; Joyce, James Avery; and Joyce, J. A.—because of inconsistencies in cataloging (see figure 6). Participants were next asked to search for an audio recording by the art- ist Prince using the basic (single) search box. Most participants did an “all fields” search for Prince and attempted to use the facets to limit by a particu- lar format. All but one was confident that they achieved the proper result, but there was confusion about the for- mat. Some participants were confused as to what format an audio recording was because the correspond- ing facet was for a music recording. A couple of users thought “audio recording” could be a spoken-word record- ing. Most participants preferred that the format facets be more concrete toward a single actual physical format, such as a record, cassette, or a compact disc (see figure 7). Physical formats appeared to resonate more with users than the broad cataloging term of “music recording.” A more specific format type (i.e., compact disc) is contained in the call number and should be straightforward to pull out as a facet. It appears VuFind pulls the format informa- tion from MARC field 245 subfield $h for medium rather than the call number (which at Illinois can specify the format) or the 300 physical description field or another field such as a notes field that some institutions may use to specify the exact format. However, when participants were asked to further use facets to find Prince’s first album, 1978’s For You, limitations with VuFind became more apparent. Each par- ticipant used a different method to search for this album, and none actually found the item either locally or in I-Share, though the item has multiple copies available in both locations. Most participants tried initially limiting by date because they were given that information. However, VuFind’s facets focus on eras rather than specific years, which participants stated was frustrating as many items can fall under a broad era. Also, the era facets brought up many more eras than one would consider an audio research skills and showed problems with both VuFind and the catalog record data. The first search asked participants to do an “all fields” search for James Joyce. All were able to complete the search, but there was notable confusion as to which records were written by James Joyce and which were items about him. About half of the first-page results for this search did not list an author on the results page. VuFind appears to pull the author field on the results page from the 100 field in the MARC record, so if the 700 field is used instead for an editor, this information is not displayed on the results page. Individual records do substitute the 700 field if the 100 field is not present, but this should also be the case on the initial results screen as well. Several users thought it was strange that the results page often did not list the author, but an author was listed in the individual record. Additionally, when asked to use the facets to limit Figure 4. Results Set Figure 5. Record Display Figure 6. Author Facet Figure 7. Format Facet usABilitY OF tHe VuFiND Next-GeNerAtiON ONliNe cAtAlOG | eMANuel 49 about both the reviews and comments that could be seen in the various records participants were asked to examine. Many of the participants wanted more information as to where the reviews came from because this information was not clear. They also wanted to know whether the reviews or comments from catalog users had any type of moderation by a librarian. For the most part, participants liked having reviews inside the catalog records, but they liked having a summary even more. Several users, all graduate students, expressed concern about the objective- ness of having reviews in the catalog, especially because it was not clear who did the review and feared that reviews may interject some bias that had no place in a library cata- log record. One of these participants stated, “If I wanted reviews, I would just go to Amazon. I don’t expect reviews, which can be subjective, to be in a library catalog—that is too commercial.” Several undergraduate participants stated that reviews helped them decide whether the book was something that would be useful to them. The final task of the usability test asked participants to create an account with VuFind because it is not connected to our user database. Most users had no problems finish- ing this task, though they found some problems with the interface. First, it was not clear that users had to create an account and could not log in with their library number as they did in the library’s OPAC. Second, the default field asks users for their barcode, which is not a term used at UIUC (users are assigned a library number). Once logged in, participants were satisfied with the menu options and how their account information was displayed. Finally, participants were asked, while logged in, to search for a favorite book and add it to their favorites list. All users liked the favorites-list feature, and many already knew of ways they could use it, but several wished they could create multiple lists and have the ability to arrange lists in folders. ■■ Discussion Participants thought favorably of the VuFind interface and would use it again. They liked the layout of informa- tion much more than the current WebVoyage interface and thought it was much easier to look at. They also had many comments that the color scheme (yellow and grey) was easier than the blues of the primary library OPAC. VuFind also had more visual elements, such as cover images and icons representing format types that partici- pants also commented on favorably. When asked to compare VuFind to both the WebVoyage catalog and Amazon, only one participant indicated a preference for Amazon, while the rest preferred VuFind. The user who specified Amazon, a faculty member, stated that that was where he always started searching for books; he would then search for specific titles in the recording, such as the 15th century. Granted, the 15th century probably brings up music that originated in that era, not recorded then, but participants wanted the date to correspond to when an item was initially published or released. It appears that VuFind pulls the era facet information from the subject headings and ignores the copyright or issue year. To users, the era facets are not useful for most of their search needs; users would rather limit by copyright or the original date of issue. Another search that further highlighted problems searching for multimedia in VuFind is the title search participants did for Gone with the Wind. Everyone thought this search brought up relevant results, but when asked to determine whether the UIUC library had a copy of the DVD, many users expressed confusion. Once again, the confusion was based on the inability to limit to a specific format. Participants could use the facets to limit to a film or video, but not to a specific format. Several participants stated that they needed specific formats because when they are doing a comparable search, they only want to find DVDs. However, because all film formats are linked together under “Film/Video,” they must to go into indi- vidual records and examine the call number to determine the exact format. Most participants stated clearly that “DVD” needed to be it’s own format facet and that enter- ing a record to find the format required too much effort. Participants also expressed frustration that the call num- ber was the only place to determine specific format and believed that this information should be contained in the brief item information and not buried in the tabbed areas. The frustrations with the lack of specific formats also were evident when participants were asked to do an advanced search for a DVD on public speaking. All users initially thought the advanced search limiter for film/video was sufficient when they first looked at the advanced search options. However, when presented with an actual search (“public speaking”), they found that there should be more options and specific format choices up-front within the advanced search. Another search that participants conducted was an author search for Jack London. They then used the fac- ets to find the book White Fang. This search was chosen because the resulting records are mostly for older mate- rials that often do not contain a lot of the additional information that newer records contain. Participants looked at a specific record and then were asked what they thought of the information that was displayed. Most answered that they would like as much informa- tion as you can give them, but were accepting of missing information. Several participants stated that most people already know this book and thus did not need additional information. However, when pressed as to what informa- tion they would like added to the record, several users stated a summary would be the most useful. Additionally, several users asked for more information 50 iNFOrMAtiON tecHNOlOGY AND liBrAries | MArcH 2011 the simplicity of the favorites listing feature, the difficulty of linking to other I-Share library holdings, and the dif- ficulties in using the facet categories. ■■ Implications I intend to continue to perform similar usability tests on next-generation catalogs on a trial basis to examine one aspect regarding the future of online catalogs at UIUC. UIUC is looking at various catalog interfaces, of which VuFind is one option, to see which best meets the needs of our users. Users stated multiple times during testing that they find the current WebVoyage interface to be very frustrating and will accept nearly anything that is an improvement, even if the new interface has some usabil- ity issues. VuFind is not perfect for all searches, as shown by a lack of a call number search and the limitations in searching for multimedia options, but it does provide a more intuitive interface for most patrons. The future of VuFind at UIUC is still open. Development is currently stalled because of a lack of developer updates and internal staffing constraints both at UIUC and CARLI. However, because VuFind is open–source, and the only ongoing cost is that of server maintenance, both CARLI and the library are continuing to display it as an option for searching the catalog. Both CARLI and UIUC are closely examining other options for catalog interfaces that would provide patrons with a better search experience, but they have taken no further action to permanently adapt either VuFind or to demo other options. Despite its limitations, VuFind is still a viable option for libraries with substantial technology expertise that are interested in a next-generation catalog interface at a low price. Although it does have limitations, it has a bet- ter out-of-the-box interface than traditional OPACs and should be considered alongside commercial options for any library thinking of adapting a catalog interface overlay. This usability test focused on one institution’s installation of VuFind, which may or may not apply to other installa- tions and other institutional needs. It would be interesting to study an installation of VuFind at a smaller, nonresearch institution, where users have different searching needs and expectations related to a library’s OPAC. References 1. John Houser, “The VuFind Implementation at Villanova University,” Library Hi Tech 27, no. 1 (2009): 96–105. 2. VuFind, “VuFind: About,” http://www.vufind.org/about .php (accessed Sept. 10 2009). 3. Kathleen Bauer, “Yale University VuFind Test— Undergraduates,” http://www.library.yale.edu/libepub/ usability/studies/summary_undergraduate.doc (accessed Mar. 20, 2010). library catalog to check availability. Other participants who made comments about Amazon stated that it was commercial and more about marketing materials, while the library catalog just provided the basic information needed to evaluate materials without attempting to sell them to you. Several participants also stated they checked Amazon for book information, but generally did not like it because of its commercial nature; because VuFind pro- vides much of the same information as Amazon, they will use VuFind first in the future. Participants also thought Amazon was for a popular and not scholarly audience, making it not useful for academic purposes. Most users did not have much to say about the WebVoyage OPAC, except it was overwhelming, had too many words on the result screen, and was not pleasantly visual. Participants were also asked to look at VuFind, Amazon, and WebVoyage from a visual preference. Again, participants believed that VuFind had the best layout. They liked that VuFind had a very clean and uncluttered interface and that the colors were few and easy on the eye. They also commented about the visuals contained (cover art and icons) in the records and the vertical orientation of VuFind (WebVoyage has a horizontal orientation) to display records. They also liked how the facets were dis- played, though two users thought they would be better situated on the left side of the results because they scan websites from the left to the right. The one thing that was mentioned several times was VuFind’s lack of the star rating system that Amazon uses to quickly rate an item. Participants thought such a system might be better than reviews because it allows users to quickly scan through the item and not have to read through multiple reviews. When asked to rate the ease of use for VuFind, with 1 being easy and 5 being difficult, participants rated it an average of 1.92. Faculty rated the ease at 1.6, graduate stu- dents at 1.75, and undergraduates at 2.8. Undergraduates were more likely to get frustrated at media searching and thought that some of the facets related to media items were confusing, which they used to explain their lower scores. However, when asked if they would rather use VuFind over the current library catalog (WebVoyage), all but one participant enthusiastically stated they would use VuFind. Most users stated that although VuFind was not perfect, it was still much better than the other library catalog because of the better layout, visuals, and ability to limit results. The only user that specified they would still rather use the WebVoyage catalog believed it had more options for advanced search, such as call number search- ing, which VuFind lacked. There are, however, several changes that could make VuFind more useful to our users that came out of usabil- ity testing. Some of these are easy to implement on a local level, and others would improve the base build of VuFind. A number of issues arose from usability testing, but the largest issues are the lack of Refworks integration, usABilitY OF tHe VuFiND Next-GeNerAtiON ONliNe cAtAlOG | eMANuel 51 9. Jennifer Bowen, “Metadata to Support Next-Generation Library Resource Discovery: Lessons from the eXtensible Catalog, Phase 1,” Information Technology & Libraries 27, no. 2 (2008): 6–19. 10. Tom Steele, “The New Cooperative Cataloging,” Library Hi Tech 27, no. 1 (2009): 68–77. 11. Ian Rowlands and David Nicholas, “Understanding Information Behaviour: How do Students and Faculty Find Books?” Journal of Academic Librarianship 34, no. 1 (2008): 3–15. 12. Ja Mi and Cathy Weng, “Revitalizing the Library OPAC: Interface, Searching, and Display Challengers,” Information Technology & Libraries 27, no. 1 (2008): 5–22. 13. Jakob Nielsen, “Why You Only Need to Test with 5 Users,” http://www.useit.com/alertbox/20000319.html (accessed Mar. 20, 2010). 4. Christine Borgman, “Why are Online Catalogs Still Hard to Use?” Journal of the American Society for Information Science 47, no. 7 (1996): 493–503. 5. Georgia Briscoe, Karne Selden, and Cheryl Rae Nyberg, “The Catalog versus the Home Page: Best Practices for Connecting to Online Resources,” Law Library Journal 95, no. 2 (2003): 151–74. 6. Kristin Antelman, Emily Lynema, and Andrew K. Pace, “Toward a Twenty-First Century Library Catalog,” Information Technology & Libraries 25, no. 3 (2006): 128–39. 7. Marshall Breeding, “Library Technology Guides: Discovery Layer Interfaces,” http://www.librarytechnology. org/discovery.pl?SID=20100322930450439 (accessed Mar. 2010). 8. Karen M. Spicher, “The Development of the MARC Format,” Cataloging & Classification Quaterly 21, no 3/4 (1996): 75–90. Appendix. VuFind Usability Study Logging Sheets I. The Look and Feel of VuFind A. Basic Screen (the VuFind main page) 1) Is it obvious what to do? Yes _____ No _____; What were you trying to do? 2) Open the drop down box, examine the options. Do you recognize theseoptions? Yes _____ No _____ Some _____ (If some, find out what the patron was expecting and get suggestions for improvement). Comments: B. Click on the Advanced Search option—take a minute to allow the participants to look around the screen 1) Examine each of the Advanced Search options a) Are the advanced search options clear? Yes_____ No_____ b) Are the advance search options helpful? Yes_____No_____ 2) Examine the Limits fields, open the drop-down menu boxes a) Are the limits clearly identified? Yes _____ No _____ b) Are the pictures helpful? Yes _____ No _____ c) Are the drop-down menu box options clear? Yes _____ No _____ Comments: II. (Back to the) Basic Search Field A. Enter the phrase—climate change (search all fields)—examine the search results 1) Do the records retrieved appear to be relevant to your search statement? Yes _____No _____Don’t Know _____ 2) What information would you like to see in the record? How should it be displayed? 3) Examine the right sidebar. Are the “facets” clear? Yes _____No _____Some, not all _____ 4) If you want to view items from other libraries in your search results, can you find the option? Yes _____No _____ 5) Can you find an electronic book published in 2008? Yes _____No _____Don’t Know _____ Comments: B. Click on the first book record in the original Climate Change search results 1) Is information about the book clearly represented? Yes _____ No _____ 2) Is it clear where to find item? Yes _____ No _____ 3) Look at the Tags. Do you understand what this feature is? Yes _____ No _____ Comments: C. Look at the brief item information provided on the screen 1) Is the information displayed useful in determining the scope and content of the item? Yes _____No _____ 2) Are the topics in the record useful for finding additional information on the topic? Yes _____No _____ Comments: D. Click on each button below the brief record information 1) Is this information useful? Yes _____ No _____ 2) Are the names for the tabs accurate? What should they be named? E. Can you easily determine where the item is located and how to request it? Yes _____No _____ Comments: F. Go back to the basic search box and enter the author James Joyce (all fields) as a new search 1) Is it easy to distinguish items by James Joyce from items about James Joyce? Yes _____No _____ 2) Using the facets, can you find only titles with James Joyce as author? Yes _____No _____ 3) Can you find out how to cite an item? Yes _____ No _____ Comments: 52 iNFOrMAtiON tecHNOlOGY AND liBrAries | MArcH 2011 G. Now try to find an audio recording by the artist Prince using Basic Search Were you successful? Yes _____No _____ H. Find the earliest Prince recording ( “For You”; 1978). Is it in the local collection? Yes _____ No _____ If not, can you get a copy? Comments: III. In the Advanced Search Screen: A. Use the title drop down to find the item: Gone with the Wind 1) Were you successful? Yes _____ No _____ Not Sure _____ 2) Can you locate a DVD of the same title? Yes _____ No _____ 3) Are copies of the DVD available in the University of Illinois Library? Yes _____ No _____ Comments: B. Use the author drop down in the Advanced Search to locate titles by: Jack London Using the facets, find and open the record for the Jack London novel, White Fang. Explore each of the: Description, Holdings, and Comments tabs: 1) Is this information useful? Yes _____ No _____ 2) Would you change the names of the tabs or the information on them? 3) Other than your local library copy of White Fang, can you find copies at other libraries? Yes _____ No _____ Comments: C. Using the Advanced Search, find a DVD on public speaking (Hint: use the limit box to select the film/video format) Are there instructional videos in the University of Illinois library? Yes _____ No _____ 1) Identify the author that’s responsible for one of the DVDs 2) Can you easily find other works by this author? Yes _____ No _____ Comments: IV. Exploring the Account features: A. Click on Login in the upper right corner of the page. On the next page, create an account. Is it clear how to create an account? Yes _____ No _____ B. Once you have your account and are logged in to VuFind, look at the menu on the right hand side. Is it clear what each of the menu items are? Yes _____ No _____ C. While still logged in, do a search for your favorite book and add it to your favorites list. Is this tool useful, would you consider using it? Yes _____ No _____ Comments: V. Comparing VuFind to other resources: A. Open three browser windows (this is easiest in Firefox by entering Ctrl-T for each new window) with 1) Your Library Catalog 2) VuFind 3) Amazon.com Enter global warming in each website in the basic search window of each. Based on your initial reactions, which service appears the best for most of your uses? Library Catalog _____ VuFind _____ Amazon _____ Comments: C. Do you have a preference in the display formats? Library Catalog _____ VuFind _____ Amazon _____ Comments: Debriefing Now that you have used VuFind, how would you rate it—on a scale from 1–5, from easy to confusing to use? Comments? How does it compare to other library catalogs you’ve used? If VuFind and your home library catalog were available side-by-side, which would you use first? Why? Are you familiar with any of these other products: Aquabrowser _____ GoogleBooks _____ Microsoft Live Search _____ LibraryThing _____Amazon.com _____Other preferred service _____ That’s it! Thank you for participating in our usability. You will be receiving one other survey through email, we appreciate your opinions on the VuFind product. LITA covers 2, 3, and 4 Index to Advertisers