College and Research Libraries Research Notes The "Known" in Known-Item Searches: Empirical Support for User-Centered Design Barbara M. Wildemuth and Ann L. O'Neill Traditionally, the catalog record for an item has been based on the cataloger's inspection of the item and has contained a complete description of the item. An alternative approach-user-centered design-would require the study of user behaviors and cognition related to interaction with the catalog and using the results of those studies to inform design decisions. To support known-item searches, one would need to study users' conceptions of the item being sought, what the user knows about the item, and which pieces of known information are viewed by the user as most appropriate for inclusion in a search. A pilot study was conducted to develop methods that can be applied to these questions. During the three phases of the study, 103 catalog users described 386 searches. Any written information known by the searchers was photocopied. The search- ers generally knew the title, publication date, page numbers (particularly for journals), and/or the author (particularly for books). The information known by the searcher was usually accurate. Results from the study indicated that the method was feasible and valid, and provided a preliminary picture of known- item searching in one library's catalog. orne people who approach a library catalog have a particu- lar item in mind, and they · want to determine whether the library holds that item and where in the library it is located. Such a person would conduct a known-item search. A known- item search may include the author, the title, the subject, or a combination of these and other pieces of information to identify the item in the catalog. Inclusion of a piece of information in the known- item search presumes that the searcher knows that piece of information. Barbara M. Wildemuth is an assistant professor at the University of North Carolina School of Information and Library Science at Chapel Hill. Ann L. O'Neill is an instructor at the University of South Carolina's College of Library and Information Science. The authors would like to express their gratitude to Frederick G. Kilgour for raising the issues addressed in this paper and for his encouragement during the completion of this study. Funding for this study was provided by a Junior Faculty Development Grant from the University of North Carolina at Chapel Hill. 265 266 College & Research Libraries Traditionally, the catalog record has been conceptualized as a surrogate for an item. Since all possible searches for that item cannot be anticipated, the goal is comprehensive description of the item based on the cataloger's inspection of it. Though attempting to serve the user's goals, the design of such a catalog is system- oriented, i.e., the designers use knowledge of the system (including the items being described) to design the catalog. An alternative approach is user-cen- tered design. 1 Applying this approach would require the study of user behav- iors and cognition related to use of the catalog and using the study results to inform design decisions. To support known-item searches, one would need to study users' conceptions of the item being sought (e.g., how vague/ clear is the · user's image of the item?), what the user knows about the item, and which known information is viewed by the user as most appropriate for inclusion in a search. The pilot study described here was conducted to develop and test methods for determining (1) which data elements related to known-item searches are pos- sessed by catalog users and (2) the utility of those data elements in conducting catalog searches. This study is a meth- odological precursor to a large-scale study of known-item searches. In addi- tion, this article reports preliminary re- sults from the pilot study. LITERATURE REVIEW Most studies of online public access catalogs (OPACs) have focused on user satisfaction, system features, interface design, system response time, or subject searches conducted. There have been only a few investigations of known-item searching, and even fewer studies of the information brought to the catalog to conduct a known-item search. The litera- ture on known-item searches addresses three questions: • What percentage of catalog searches are known-item searches? • What types of errors occur in known- . item searches? • What information do people bring to the catalog to search? May1995 Transaction log analyses and surveys have been used most often to obtain data on OPAC use. Academic libraries have been investigated more frequently; however, a few public libraries were cov- ered also. Many of the studies that address known-item searching were conducted in the early 1980s. ·of particular note are the cross-institutional studies spon- sored by the Council on Library Re- sources (CLR). 2 At the time of these studies, OPACs were relatively new. Many search features have been im- proved in today's online catalogs and library users are more accustomed to OPACs. Therefore, the results reported in this review may not apply to today's catalogs and their users. What Percentage of Catalog Searches Are Known-Item Searches? Because transaction logs generally are examined after the search, it is impossi- ble to determine the type of search reli- ably.3 Some people use either the author or title of a known work as a starting point in a subject search. The interpre- tation of keyword searches is also am- biguous. For example, MELVYL, the University of California OPAC, assumes that all title searches are keyword searches unless an exact-title com- mand is specified. Some title keyword searches may actually be subject searches. John Akeroyd claims that searches per- formed solely to identify items on a par- ticular subject may account for as much as 24 percent of all author/title searches. 4 One study found that, of fifty searches, three author and two title searches (10 percent of the total) were actually subject searches, and another found that 43 percent of the catalog searches for a specific item were subject searches or hybrid subject/known-item searches.5•6 At Yale University, it was found that 73 percent of the card catalog users stated that they were looking for a specific item, but almost one-third of them believed they could find the de- sired information in some other publica- tion.7 Conversely, a subject search could be conducted to identify a known item. The "Known" in Known-Item Searches 267 A study based on researcher-specified queries found that 27 percent of the card catalog searches for the "Warren Report" began with a subject search, and a natural- istic study found that 6 percent of known- item searches in a card catalog began with a search for a subject heading.8'9 In spite of the ambiguity involved, several studies have categorized the type of search conducted based on ex- amination of transaction logs (see table 1). In the table, author, title, and call number searches are considered known- item searches. The row totals represent the proportion of all searches in the transaction logs that were known-item searches; the remaining proportion (not shown) were subject searches. The re- sults are reviewed here, beginning with the most recent. At North Carolina State TABLEt Study Akeroyd, 1990 Akeroyd, 1990 Akeroyd, 1990 Chang, 1986 Holmes and Bulger, 1988 Hunter, 1991 Matthews, Lawrence and Ferguson, 1983 Matthews, Lawrence and Ferguson, 1983 Matthews , Lawrence and Ferguson, 1983 Matthews, Lawrence and Ferguson, 1983 Matthews, Lawrence and Ferguson, 1983 Matthews, Lawrence and Ferguson, 1983 Matthews, Lawrence and Ferguson , 1983 Peters, 1989 Tolle, 1983* Zink, 1991 PERCENTAGE OF SEARCHES BY TYPE, BASED ON TRANSACTION LOG ANALYSIS Library South Bank Polytechnic Stirling University Polytechnic of Central London University of North Carolina University of Ottawa NC State University University of California (command mode) University of California (lookup mode) Northwestern University Claremont Colleges Mankato State University Mission/West Valley Colleges Pikes Peak Library District University of Missouri, Kansas City Dallas Public Library University of Nevada; Reno Type of Known-item Search Title or Author Title Call Row Author Series Title and Title Keyword Number Total 24% 55% 79% 35% 21% 16% 72% 18% 39% 21% 78% 27% 27% 54% 23% 34% 8% 3% 3% 71% 21% 26% 0% 47% 28% 24% 52% 30% 41% 71% 31% 38% 69% 35% 18% 2% 55% 12% 16% 7% 3% 38% 16% 19% 0% 2% 37% 13% 40% 53% 23% 1% 34% 1% 59% 9% 10% 2% 3% 24% 13% 19% 32% ,. In addition to subject searches (38 percent of the total), Tolle's data included Begin and End commands (18 percent each) and errors (2 percent). _ 268 College & Research Libraries University 47 percent of the searches were for authors or titles, while 32 per- cent of the searches at University of Ne- vada, Reno, were for authors or titles. 10•11 Author, series, and title searches for known items accounted for 56 percent of the searches at Stirling University, while searches for authors and titles accounted for 79 percent at South Bank Polytechnic, and searches for authors accounted for 18 percent at the Polytechnic of Central London (where all title searches were by keyword). 12 Of the searches conducted on the University of Missouri Information Network, 24 percent were for authors or titles. 13 Fifty-seven percent of the searches at the University of Ottawa were for an author or a title. 14 In a preliminary study of the online catalog at the University of North Carolina at Chapel Hill, 27 percent of the commands were for author searches and 27 percent were for title searches.15 CLR study results indicated that over half the searches were for authors or titles at Northwestern University, the University of California, the Claremont Colleges and Pikes Peak [public] Library District, while only 28 percent of the searches at Mankato State University and 35 percent of the searches at the Mission/West Valley [com- munity] Colleges were for authors or titles. 16 Only 19 percent of the searches at the Dallas Public Library were based on authors or titlesP As summarized in ta- ble 1, different studies have found quite different results in the number of author and title searches conducted. Some systems support searches of author I title combinations. For example, at Ohio State University, derived author/title search keys (e.g., 4,4 or 4,3,3,2) make possible the author/title search feature. Combination author/ti- tle searches made up 39 percent of the searches at the Polytechnic of Central London, but were much less common at other libraries (8 percent of the searches at the University of Ottawa, 2 percent of those at the Dallas Public Library, and less than 1 percent of those at the Mis- sion/West Valley Colleges).1S-21 Other systems have a title keyword feature, allowing users to search for any word in a title, rather than being re- May1995 stricted to words at the beginning. The classification of these searches as known- item searches particularly is open to ques- tion. Title keyword searches accounted for 16 percent of the searches at Stirling Uni- versity, 21 percent at Polytechnic of Central London, and 34 percent at the University of Missouri-Kansas City.22- 24 The same searches accounted for only 3 percent at the University of Ottawa and at Mankato State University.25•26 Numeric searches for known items (e.g., using call number, ISSN, or ISBN) are a very small percentage of the searches. In most transaction log studies, numeric searches are attributed to staff and discounted from further analysis. When the percentage of numeric known- item searches is reported, it ranges from less than 1 percent to 3 percent. Surveys and interview studies of the types of catalog searches conducted are listed in table 2. The percentages in this table represent the proportion of all the respondents that reported conducting each type of search; the remaining searches (not reported) were subject searches. These studies were more consistent in th~ proportion of known-item searches found: 48 percent at Ohio State University; 33-36 percent in the University of Califor- nia system; and 41-50 percent across the libraries participating in the CLR study.27- 29 For comparison, an earlier study of card catalog use found that 56 percent of the users were looking for a specific docu- ment and 11 percent of the users were conducting searches for a particular author or for bibliographic verification.30 In summary, transaction logs have in- dicated that between 24 percent and 78 percent of OPAC searches are for authors or titles. Surveys have found that be- tween 33 percent and 67 percent of the searches are for known items. Based on these data, it can be concluded that known-item searching is an important function that should be supported by a library catalog. What Types of E"ors Occur in Known-Item Searches? In the studies cited here, a search state- ment generally is categorized as an error The "Known" in Known-Item Searches 269 TABLE2 PERCENTAGE OF SEARCHES BY TYPE, BASED ON INTERVIEW DATA Library Study (Question Asked) Author Alzofon and Van Ohio State 14% Pulis, 1984 University (command used) Lawrence, Graham University of 14% and Presley, 1984 California (information brought to search) Lawrence, Graham University of and Presley, 1984 . California (purpose of search) Lawrence, Graham Multiple libraries 14% and Presley, 1984 (information brought to search) Lawrence, Graham Multiple libraries and Presley, 1984 (purpose of search) Lipetz, 1970 Yale University 6% card catalog (purpose of search) if it retrieves no citations. Most studies have not tried to determine whether zero-retrieval is due to the fact that the library does not hold the desired item. When such collection failures were taken into account, an adjusted error rate is re- ported. A few studies have defined er- rors based on an evaluation of the quality of the search statements, rather than the outcome of the search.31.32 Overall failure rate for known-item searches varies from study to study. In the CLR study, users ·were asked whether their searches were successful. Sixteen percent said that they did not find any of the items sought.33 Other studies estimated success based on transaction logs. Naturalistic studies conducted at Ohio State University found that 10 to 15 percent of the search sessions contained only errors, and that 17 to 25 percent of the title searches in departmental library catalogs were un- successful.34.35 Errors occurred in 37 per- cent of the title searches and 22 percent of the author searches conducted at Northwestern University.36 For 44 per- cent of the known-item searches in an online circulation system at the Univer- Type of Known-item Search Author Known Bibliographic Row Title and Title Document Verification Total 18% 16% 48% 6% 13% 33% 36% 36% 12% 18% 44% 50% 50% 56% 5% 67% sity of Illinois, the item was not found, even though it was in the system.37 Several researchers examined search failures in more detail. In most studies, typographical errors or misspellings were frequent. At Northwestern Univer- sity 54 percent of the errors in title searches, excluding collection failures, were typographical or spelling errors, and a reanalysis of a portion of that data found that 36 percent of the errors in author searches were typographical, ex- cluding collection failures. 38.39 Excluding collection failures, 60 percent of the er- rors in title searches and 30 percent in author searches on the University of Mis- souri Information Network were typo- graphical errors or misspellings; the comparable error rates at the University of Nevada, Reno, were 37 percent of the un- successful title searches and 42 percent of the unsuccessful author searches.40•41 Even in a smaller study of experienced catalog users, 5 percent of the errors were typo- graphical.42 Based on analyses of both failed and successful searches, two stud- ies found that typographical errors oc- curred in 10 percent of the searches and another found that such errors oc- 270 College & Research Libraries curred in 7 percent of the searches.43·44 In summary, typographical errors or misspellings accounted for approxi- mately one-third of the errors made in conducting known-item searches in on- line catalogs. Another common error involved the search mode. In most catalogs the type of search must be specified prior to en- tering the search term, and users experi- ence confusion about search mode (e.g., they enter an author's name while in subject search mode). Two studies con- ducted at the University of North Caro- lina found that the user's failure to specify a search mode accounted for 18 percent and 14 percent of the errors, re- spectively.45 At Northwestern University such errors accounted for 8 percent of the title search errors and 6 to 7 percent of the author search errors, excluding col- lection failures. 46 The comparable rate at the University of Missouri was 16 per- cent.47 At the University of Nevada, Reno, mode errors occurred even more fre- quently, accounting for 60 percent of the errors among title searches, excluding col- lection failures. 48 The wide variation in the rate of mode errors is most likely due to differences in catalog design, i.e., the vis- ible cues for specifying search type. In author searches, a frequent error was to leave the name uninverted (e.g., Mark Twain instead of Twain, Mark). Excluding collection failures, 36 percent of the errors in author searches at the University of Nevada, Reno, could be attributed to uninverted names.49 The comparable error rate at Northwestern University was 22 to 24 percent; and at the University of Missouri was 19 per- cent.so,st At the University of Ottawa, name inversion errors occurred in 6 per- cent of the author searches.52 A frequent error in title searches was the inclusion of an initial article in the search term, accounting for 20 to 26 per- cent of the errors at the University of North Carolina at Chapel Hill, and for 26 percent at Northwestern University, ex- cluding collection failures. 53,s4 Only these three studies examined this type of error, but the results are strikingly consistent and show a high failure rate. May 1995 In summary, users are unsuccessful in approximately one-quarter of their known-item searches. Researchers attrib- ute these search failures to a variety of causes. The most consistent finding is that many typographical and spelling errors are made. In addition, users experience difficulty in expressing bibliographic in- formation in the form required by the cata- log and in handling command syntax. What Do Users Bring to the Catalog to Search? Empirical data concerning the biblio- graphic information brought to a search of an online catalog are almost nonex- istent. The CLR study examined this is- sue in the most detail, asking catalog users what information they possessed (author, partial author, title, partial title, subject heading, etc.) and which infor- mation they used in their search. 55 David Holmes and Derrick Bulger reported that very few searches incorporated more data than that included in a brief cataloging record, i.e., bibliographic ele- ments other than author, title, date, or call number.56 Jerry Specht asked sub- jects what information they brought to the search, but reported this information only as "known-item" or "location" search. 57 Two earlier studies, included in this review, were conducted of the biblio- graphic information users brought to searches of a card catalog.58 The CLR survey indicated that, across both known-item and subject searches, 50 percent of the users knew the author's name, and 48 percent knew the title.59 Earlier studies of card catalog use delved more deeply into the completeness and accuracy of the bibliographic data pos- sessed by users. At Yale University 77 percent of those conducting known-item searches knew the author, 97 percent knew the title, and 59 percent had date information. However, only 42 percent had completely accurate author infor- mation, only 62 percent had accurate ti- tle information, and only 29 percent were within one year of the correct pub- lication date. 60 In a study of ~own-item searches at three university libraries and one public library, 70 percent of the titles The "Known" in Known-Item Searches 271 were complete and accurate and 60 per- cent of the authors' last names were complete and accurate. 61 These results should be applied to OPAC design with caution, however, because "minor dis- crepancies" in spelling were disregarded. 62 These discrepancies might not disturb card catalog searches but could result in failure of an online catalog search. From these studies, it can be con- cluded that people often bring basic bib- liographic data to the catalog, but that there are often inaccuracies in the data, some of which have significant negative effects on the search outcomes. RESEARCH QUESTION The research reviewed above indi- cates that known-item searches account for a significant proportion of online catalog searches. Therefore, it is worth- while to try to improve users' effective- ness in conducting such searches. It is also clear that typographical and spell- ing errors are often the cause of search failures. Based only on an examination of transaction logs, it cannot be deter- mined whether these spelling problems are related to the user's typing skills or the inaccuracy of the bibliographic data they possess. Studies of card catalog use indicate that users often have basic bib- liographic data available to support their searches, but their data often con- tain inaccuracies. In a user-centered approach to catalog design, all these findings are helpful. However, gaps in our knowledge re- main. The current study was intended to test a method for addressing three re- search questions: (1) Of the many data elements that could be used to describe a bibliographic entity, which data ele- ments do users bring with them to sup- port their known-item searches? (2) How accurate is their recording or mem- ory of those data elements? and (3) How successful are OPAC searches that ern- ploy those data elernents?63 METHOD In order to develop a method for ad- dressing these research questions, an in- terview protocol was developed to determine what type of search the user was conducting, which data elements the user possessed prior to beginning the search, and whether the user considered the search successful. (The interview re- sults will be integrated with transaction log analysis when the full-scale study is conducted in order to evaluate the rela- tionship between the bibliographic data possessed and that used in the search.) This section describes the final instru- ment and the evolution of the interview protocol over three phases of data collec- tion. In the next section, the preliminary results generated during the pilot study are reported. All the interviews were conducted in Davis Library at the University of North Carolina at Chapel Hill (UNC). Data collection periods were approximately 1.5 to 2 hours and were staggered to cover class changes. Data collection was scheduled at different times be- tween 10:00 a.m. and 8:00p .m. to deter- mine periods of heavy use. The online catalog software was a customized ver- sion of the catalog available from Data Research Associates (DRA). Public online catalog terminals were available on the main floor of the library and on each floor of the library stacks. During each three- day data collection phase, one of the investigators was stationed near the catalog terminals. Respondents were selected from those who approached the terminal area but had not yet started their search. With the exception of library staff, repeat users, and students working in groups, every person that approached a terminal while the investigator was not occupied with another respondent was invited to par- ticipate in the study. Only those who said they were searching for known items ("an author," "a book," "a jour- nal") were asked all questions. The first draft of the survey instru- ment was derived mainly from questions on the CLR study survey.64 Questions were open-ended to accommodate the full range of responses. The interview cov- ered the type and purpose of the re- spondent's search(es), the bibliographic information possessed by the respon- 272 College & Research Libraries dent (eitherwrittenorremembered), the respondent's evaluation of the success of the search, and the discipline and aca- demic status of the respondent. The first set of interviews was con- ducted in mid-October 1993. If the re- spondent brought written information to the terminal, permission was asked to photocopy that information (a desktop copier was moved to the online terminal ar~a for this purpose). If the respondent did not have written information, all in- formation known about the desired item(s) was recorded, spelled as reported by the respondent. After the search was completed, the respondent was asked whether the desired item was found and, if not, whether other items of inter- e~t were identified. The second set of interviews was con- ducted in early November. In this phase, response categories were specified for all questions, call number .verification was added as a type of search, and the interviewer asked in more detail about the bibliographic information known and the source of that information. The third and final set of interviews was conducted in mid-November. By this time, all questions were closed- ended. For several questions, an "other" response category was still available and, where appropriate, the interviewer specified the user's response. The final form of the interview protocol is in- cluded in the appendix. May1995 In addition to the interviews, the . data reported in the next section include the out- comes from a replication of each search by a member of the research team, based on the information possessed by the user at the time of the interview. The outcomes of these searches can then be compared with the outcomes reported by there- spondents at the completion of their searches. RESULTS One hundred eighty-three people were invited to participate in the study. Of this number, 58 (32 percent) were con- ducting subject searches and 22 (12 per- cent) declined participation, resulting in 103 interviews of people conducting known-item searches. Table 3 presents the academic status and academic departments represented among the 103 respondents. Approxi- mately three-quarters of the respon- dents were students, split fairly evenly between undergraduate and graduate students. The individual· academic de- partments represented most frequently were English (9), psychology (9), educa- tion (7), political science (7), and sociology (7). The prevalence of departments in the social sciences and humanities can be at- tributed to the fact that these departments are primarily served by Davis Library, while many of the departments in the natural sciences (including the health sciences) have departmental libraries. TABLE3 ACADEMIC STATUS AND DEPARTMENT OF RESPONDENTS Undergraduate Graduate LocaV ExtemaV Department Faculty Student Student Library Staff Total Humanities* 3 6 17 0 26 Social sciencest 5 22 29 0 56 Natural sciences:j: 6 2 0 9 Undeclared/not applicable 4 6 12 Total 10 38 49 6 103 * Departments in the humanities category included English, Slavic languages, Romance languages , German, classics, history, art, art history , music, philosophy, and religious studies. t Departments in the social sciences category included anthropology, sociology, psychology, clinical psychology, business, economics, political science, international studies, womens' studies, education, information and library science, journalism, RTVMP (radio, television, and motion pictures) , and leisure studies. :j: Departments in the natural sciences category included biology, physical education, geography, medicine, nursing, psychiatry, and pharmacy. The "Known" in Known-Item Searches 273 Because this was a pilot study in- tended to design an interview protocol, the questions asked in the three phases varied slightly. Wording was changed or response categories were added as data were collected. Whenever possible, ear- lier open-ended responses were catego- rized based on the final form of the interview schedule. The fact that this was a pilot study also affected analysis of the data. Initially respondents were considered the unit of analysis, but later it became clear that the item sought was also an appropriate unit of analysis. Un- fortunately, some per-item data from the first phase was unavailable. Of the people conducting known-item searches, 57 had written information de- scribing 338 items. They were . catego- rized as having hand-written notes; in- formal bibliographies, including class reading lists; or published references, bibliographies, and search printouts. The other 46 people, searching for 48 items, did not have any written descrip- tion of the item(s). The type of item sought and the form of the information known is displayed in table 4. Most of the items sought were books and jour- nals. Most of the journal citations were drawn from published bibliographies or CD-ROM searches. All the videos sought were from a list provided by a faculty member. Respondents knew the title for 94 per- cent of the items sought (see table 5). Publication date was known for 70 per- cent of all the items and for 97 percent of TABLE4 TYPE OF ITEM SOUGHT BY FORM OF INFORMATION KNOWN Bibliographic Search Type of Recalled from Hand-written Informal Results or Published Item Sought Memory Notes Bibliographies References Total Book 41 60 78 61 240 Journal 6 20 11 74 111 Video 0 0 33 0 33 Other 0 0 2 Total 48 80 122 136 386 TABLES BIDLIOGRAPHIC INFORMATION KNOWN ABOUT THE DESIRED ITEM Recalled Bibliographic Search from Hand-written Informal Results or Published Items from All Data Element Memory Notes Bibliographies References Sources Author(s) 28 58% 48 60% 65 53% 41 31% 182 48% Editor(s) 2 4% 4 5% 14 11% 11 8% 31 8% Title or partial title 33 69% 73 91% 122 100% 132 100% 360 94% Publisher* 3 6% 22 28% 36 30% 40 31% 101 27% Date of publication * 4 8% 45 56% 92 75% 128 97% 269 70% Subject* 9 19% 5 6% 33 27% 47 36% 94 25% Page number(s) 1 2% 10 13% 11 9% 78 59% 100 26% Other 6 13 % 3 4% 35 29% 7 5% 51 13% Total items from 48 80 122 132 382 each sourcet Total number of items analyzed = 382. Data from four of the respondents in the ftrst phase could not be analyzed. * Data on publisher, subject and other information from a ftfth respondent could not be analyzed, so the base number of items in those categories was 381. t A respondent may know more than one data element per item, so the column total will be greater than the total number of items. The percentages reported use the total number of items from each source as the denominator. 274 College & Research Libraries the items for which the respondent had a published record of the citation. The author's name was known for almost half the items. The page number was known for over half of the items where the respondent was consulting a pub- lished reference list or the output of a computer-assisted bibliographic search. Other frequently known data elements included the subject and publisher. All of these data elements would be available to someone conducting an OPAC search. May1995 Data elements known for books dif- fered from those known for journals (see tables 6 and 7). Titles were known for virtually all items-both books and jour- nals. Authors were usually known for books, but were not relevant when seek- ing the location of a journal. Publication date was almost always known for jour- nals, but was known for only about half of the books. Page numbers were known for 80 percent of the journal items sought, but for only 4 percent of the TABLE6 BIBLIOGRAPHIC INFORMATION KNOWN ABOUT BOOKS Bibliographic Search Recalled from Hand-written Informal Results or Published Items from Data Element Memory Notes Bibliographies References All Sources Author(s) 28 68 % 47 78 % 65 83% 41 72% 181 77% Editor(s) 2 5% 4 7 % 14 18% 10 18% 30 13% Title or partial title 26 63 % 53 88% 78 100% 57 100% 214 91% Publisher 2 5 % 22 37% 36 46% 38 67% 98 42% . Date of publication 3 7% 25 42% 50 64% 55 96% 133 56% Subject 8 20% 3 5% 25 32% 40 70% 76 32% Page number(s) 1 2% 2 3% 1 1% 6 11% 10 4% Other* 5 13 % 2 3% 2 3% 5 9% 14 6% Total from each 41 60 78 57 236 sourcet Total number of items analyzed = 236. Data from four of the respondents in the first phase could not be analyzed . * Data on other information known about journals from one additional respondent could not be analyzed. t A respondent may know more than one data element per item, so the column total will be greater than the total number of items. The percentages reported use the number of items from each source as the denominator. TABLE7 BIBLIOGRAPHIC INFORMATION KNOWN ABOUT JOURNALS Bibliographic Search Recalled from Hand-written Informal Results or Published Items from All Data Element Memory Notes Bibliographies References Sources Author(s) 0 1 5% 0 0 1 1% Editor(s) 0 0 0 0 0 Title or partial title 6 100% 20 100% 11 100% 74 100% 111 100% Publisher 1 17 % 0 0 1 1% 2 2% Date of publicatio~ 1 17 % 20 100% 9 82% 72 97% 102 92% Subject 0 2 10% 8 73 % 7 9% 17 15% Page number(s) 0 8 40% 10 91 % 71 96% 89 80% Other 1 17% 1 5 % 0 2 3% 4 4% Total from each 6 20 11 74 111 source* Total number of items analyzed = 111. * A respondent may know more than one data element per item, so the column total will be greater than the total number of items. The percentages reported use the number of items from each source as the denominator. The "Known" in Known-Item Searches 275 books. When considering the user's pur- poses in searching the online catalog for these types of items, such differences are not surprising. The origin of the information known by the user was analyzed in table 8. One- quarter of the items were identified through CD-ROM searches. Professors/ teachers (16 percent) and class reading lists (13 percent) also were mentioned fre- quently. For those people without written information, prior use of the item was mentioned frequently. In addition to these common information sources, responses such as overdue notices and publishers' flyers were categorized as "other." Each respondent reported the pur- pose of the search (see table 9). Com- pleting a class assignment was the reason mentioned most frequently for needing an item, accounting for al- most half the items. Other research, such as proposal writing, presenta- tions, or editing a work, was also com- mon, as were preparing a dissertation or thesis and studying for comprehensive exams. Personal use or leisure reading was most common among people who did not hav~ written information, and studying for comprehensive exams was most common among people using informal bibliographies. TABLES ORIGIN OF BIBLIOGRAPHIC INFORMATION Bibliographic Search Origin of Recalled from Hand-written Informal Results or Published Items from All Bibliographic Data Memory Notes Bibliographies References Sources Professor/teacher 9 21% 13 17% 35 29% 2 1% 59 16% Class reading list 6 14% 12 16% 30 25% 0 48 13% Reference in book 2 5 % 14 19% 0 4 3% 20 5% Reference in journal 2% 23 31% 0 4 3% 28 7% CD ROM search 1 2% 10 13% 0 82 60% 93 25% Friend 2 5% 2 3% 0 0 4 1% Used item before 7 17% 0 0 3 2% 10 3% Saw item before 3 7% 1 1% 1 1% 0 5 1% Other 11 26% 0 56 46% 41 30% 108 29% Total number 42 75 122 136 375 of items Total number of items analyzed = 375. Information about. the origin of the information was not provided for 11 of the items. TABLE9 PURPOSE OF SEARCH Bibliographic Search Recalled from Hand-written Informal Results or Published Items from All Purpose of Search Memory Notes Bibliographies References Sources Class assignment 22 46% 38 48% 29 24% 91 67% 180 47% Other research 5 10% 25 31% 25 21 % 11 8% 66 17% Dissertation/thesis 2 4% 12 15% 0 26 19% 40 10% Comprehensive exams 1 2% 0 32 26 % 0 33 9% Preparation for class 4 8% 0 1 1% 4 3% 9 2% Verify a reference 1 2% 2 3% 1 1% 0 4 1% Personal use 8 17% 3 4% 0 0 11 3% Other 5 10% 0 33 27% 4 3% 42 11% Total number 48 80 121 136 385 of items Total number of items analyzed = 385. Data from one respondent in the first phase could not be analyzed. 276 College & Research Libraries Each respondent was asked to return after completing the search and report whether the desired items were found. Almost all the respondents complied with this request, so the searcher's assess- ment of his or her success was known for 92 percent of the items. Most catalog users (70 percent) said they found the items they were seeking (see table 10). Surprisingly, the more formal the source of the biblio- graphic data, the less likely that the user conducted a successful search, with the highest success rate reported for items re- called from memory. To verify the success rates of the re- spondents, a member of the research team replicated the search for each item. For 14 items, the search could not be replicated because the respondent did not share the citation details with the May1995 researchers. The results from the repli- cated searches are presented in table 11. The researcher's success rate was very close to that of the original searcher when the original searcher's informa- tion matched the catalog record. Four author names, one editor name, and six titles provided by respondents were in- accurate. Additionally, 14 search failures can be attributed to the evolving me- dium of the catalog, i.e., they were in the collection but had not yet been added to the online catalog. It can be concluded that, with accurate citation data, respon- dents successfully used the current on- line catalog for known-item searches. DISCUSSION The purposes of this study were two- fold: first, to develop and validate a TABLElO SELF-REPORTED SEARCH SUCCESS Bibliographic Search Recalled from Hand-written Informal Results or Published Items from All Memory Notes Bibliographies References Sources Successful 38 84% 51 72% 74 69% 85 65% 248 70% Not successful 7 16% 20 28% 33 31% 46 35% 106 30% Total items for 45 71 107 131 354 which success was reported Items for which 3 9 15 5 32 success was not reported TABLEll SUCCESS OF REPLICATED SEARCHES Bibliographic Search Recalled from Hand-written Informal Results or Published Items from Memory Notes Bibliographies References All Sources Successful replicatio!ls Citation correct 31 82% 57 71% 84 69% 79 60% 251 67% Citation incorrect 6 16% 5 6% 0 0 11 3% Citation in prior catalog 0 2 3% 0 12 9% 14 4% Unsuccessful replications Item not in collection 3% 16 20% 38 31% 41 31% 96 26% Total searches replicated 38 80 122 132 372 Items not replicated 10 0 0 4 14 (citation not known) Column percentage totals may not equal 100 percent, due to rounding error. The "Known" in Known-Item Searches 277 method for collecting data concerning the information known by online catalog users and, second, to provide preliminary data concerning that infor- mation. It was successful in each of these two objectives. The methodological result of the study is a structured interview protocol that can be used to gather data concern- ing the information possessed by online catalog users. The protocol (in the appen- dix) first identifies those users intending to conduct a known-item search, then asks about the information possessed by the respondent, the source of that informa- tion, and the purpose(s) of the search. Next, it covers the academic status and discipline of the respondent. Finally, it asks for the respondent's perspective on the success of the search. The use of a desktop photocopier in conjunction with the interview was found to be an efficient and cost-effective means of ac- curately capturing written or printed ci- tation data possessed by the respondent. Because this was a pilot study of a small sample, the results may not be gen- eralizable beyond the current respon- dents. Some preliminary conclusions about catalog use can, however, be drawn. First, of the 160 catalog users who did not decline participation, only 36 percent were conducting subject searches; the remaining 64 percent were conducting known-item searches. Sec- ond, the information about an item may be recalled from memory or recorded in hand-written notes, but most often (for 6 7 percent of the items) it is more formal, such as a class reading list or output from a bibliographic search. Third, known-item searches are primarily for books (about two-thirds) and journals (about one-third), and the data elements known about an item vary by the item's form. Titles are known for either type of item, but authors are known primarily for books, while date and page numbers are known primarily for journals. Fourth, over half the items originated with a professor or teacher, on a class reading list, or on the output from a CD-ROM search. Almost half the searches were conducted in connection with a class assignment. Finally, most (70 percent) of the ·searches were suc- cessful. Only a small proportion of the search failures (3 percent) could be at- tributed to inaccuracies in the informa- tion possessed by the respondents, and such inaccuracies occurred only when the re- spondent depended on his or her memory or hand-written notes. The fact that these respondents were searching on recently implemented soft- ware had no detectable negative effect. Excluding collection failures, the few search failures that occurred can be attrib- uted to inaccuracies in the citations-not to inadequacies in the catalog software. On the other hand, some people may have reported finding the desired item(s), even though they did not find exactly what they wanted, because of frustration with learn- ing a new system. The availability of key- word searching, a new feature to· UNC catalog users, may also lead to changes in the information that users bring to the catalog in the future. A large-scale study based on the method described here could address several issues, such as the variability in the information possessed by catalog us- ers and the accuracy of that information. However, one important question can- not be addressed by an interview: Of the information available, which is most likely to be used in a catalog search? The analysis of transaction logs would com- plete the picture by allowing connec- tions to be made between the data elements available, the data elements in- cluded in the search, and the success of the search. Such an analysis should be integrated with future interview studies of catalog use. IMPLICATIONS AND CONCLUSION The results reported here provide a preliminary picture of the respondents' catalog use. A more complete picture would be provided by the large-scale study described above. Results from the large-scale study could be used to im- prove online catalogs by focusing our attention on those data elements most likely to be included in searches. In essence, the results could provide the 278 College & Research Libraries basis for decisions concerning the re- duction of effort expended in descrip- tive cataloging. However, there is one major short- coming with this reasoning: it assumes that the primary use of the online catalog is to search for items-either for known items, as discussed in this paper, or for items on a given subject. Analysis of catalog use is incomplete without gath- ering additional data concerning non- search uses of the catalog, some of which may be specific to subsets of the user audience (e.g., a humanities scholar's use of details about the edition of an item). This study has not taken into account the existence of such vari- May 1995 ations in catalog use, and the method described here will not be effective in studying the frequency or quality of alternative uses. It is important to keep in mind that these results are limited to an examination of catalog searches. Studies of catalog use from the user's perspective are important for the im- provement of the services that libraries can offer. A long tradition of practice, based on the perspectives of profession- als, is not a strong basis for the design of online catalogs. Instead, a user-oriented perspective should be adopted, so that we can design catalogs that further our primary goal: providing access to the in- tellectual content sought by our clients. REFERENCES AND NOTES - 1. Ruth C. T. Morris, "Toward a User-Centered Information Service," Journal of the American Society for Information Science 45 Gan. 1994): 20-30. 2. Joseph R. Matthews, Gary S. Lawrence, and Douglas K. Ferguson, eds., Using Online Catalogs: A Nationwide Survey. A Report of a Study Sponsored by the Council on Library Resources (New York: Neal-Schuman, 1983); Gary S. Lawrence, Vicki Graham, and Heather Presley, "University of California Users Look at MELVYL: Results of a Survey of Users of the University of California Prototype Online Union Catalog," in Advances in Library Administration and Organization: A Research Annual, vol. 3, ed. Gerard B. McCabe and Bernard Kreissman (Greenwich, Conn.: JAI Pr., 1984), 85-208. 3. Micheline Hancock-Beaulieu, Stephen Robertson, and Colin Neilson, "Evaluation of Online Catalogues: Eliciting Information from the User," Information Processing & Management 27 (1991): 523-32. 4. John Akeroyd, "Information Seeking in Online Catalogues," Journal of Documentation 46 (Mar. 1990): 33-52. 5. Grace Agnew, Albert Camp, Mary Nell Maule, and Jane Richards, "The Online Catalog and Patron Search Strategies at Georgia State University," Georgia Librarian 23 (May 1986): 42-44. 6. Hancock-Beaulieu, Robertson, and Neilson, "Evaluation of Online Catalogues." 7. Ben-Ami Lipetz, User Requirements in Identifying Desired Works in a Large Library, Final Report for Grant No. SAR/OEG-1-71071140-4427 (Washington, D. C.: U.S. Department of Health, Education and Welfare, Office of Education, Bureau of Research; New Haven, Conn.: Yale University Library, June 1970). 8. JamesKrikelas, "Searching the LibraryCatalog-AStudyofUsers' Access," Library Research 2 (Fall1980): 215-30. 9. Renata Tagliacozzo, Lawrence Rosenberg, and Manfred Kochen, "Access and Recognition: From Users' Data to Catalogue Entries," Journal of Documentation 26 (Sept. 1970): 230-49. 10. Rhonda N. Hunter, "Successes and Failures of Patrons Searching the Online Catalog at a Large Academic Library: A Transaction Log Analysis,"RQ 30 (Spring 1991): 395-402. 11. Steven D. Zink, "Monitoring User Search Success through Transaction Log Analysis: The WolfPAC Example," Reference Services Review 19 (Spring 1991): 49-56. 12. Akeroyd, "Information Seeking in Online Catalogues." 13. Thomas A. Peters, "When Smart People Fail: An Analysis of the Transaction Log of an Online Public Access Catalog," Journal of Academic Librarianship 15 (Nov. 1989): 267-73. 14. David Holmes and Derrick Bulger, "A Day in the Life of a Public Terminal-A Transaction Analysis of an Online Catalogue Terminal in a Bilingual Environment," Canadian Journal for Information Science 13 (Dec. 1988): 21-33. The "Known" in Known-Item Searches 279 15. Nien-tzu Nancy Chang, "User's Search Behavior on an Online Catalog: A Preliminary Transaction Log Study" (master's paper, University of North Carolina at Chapel Hill, November 1986). 16. Matthews, Lawrence, and Ferguson, Using Online Catalogs. 17. John E. Tolle, Current Utilization of Online Catalogs: Transaction Log Analysis, Final Report to the Council on Library Resources, vol. 1 (Dublin, Ohio: OCLC Online Computer Library Center, 1983). 18. Akeroyd, "Information Seeking in Online Catalogues." 19. Holmes and Bulger, "A Day in the Life of a Public Terminal." 20. Tolle, Current Utilization of Online Catalogs. 21. Matthews, Lawrence, and Ferguson, Using Online Catalogs. 22. Akeroyd, "Information Seeking in Online Catalogues." 23. Ibid. 24. Peters, "When Smart People Fail." 25. Holmes and Bulger, "A Day in the Life of a Public Terminal." 26. Matthews, Lawrence, and Ferguson, Using Online Catalogs. 27. Sammy R. Alzofon and Noelle Van Pulis, "Patterns of Searching and Success Rates in an Online Public Access Catalog," College & Research Libraries 45 (Mar. 1984): 110-15. 28. Lawrence, Graham, and Presley, "University of California Users." 29. Lawrence, Graham, and Presley, "University of California Users"; Matthews, Lawrence, and Ferguson, Using Online Catalogs. 30. Lipetz, User Requirements in Identifying Desired Works. 31. David Barnes Bennett, "BIS Online Catalog Use: A Transaction Log Analysis" (master's paper, University of North Carolina at Chapel Hill, April1987); Laurie L. Weakley, "Online Catalog Use: A Transaction Log Analysis" (master's paper, University of North Carolina at Chapel Hill, July 1989). 32. Mary Noel Gouke and Sue Pease, "Title Searches in an Online Catalog and a Card Catalog: A Comparative Study of Patron Success in Two Libraries," Journal of Academic Librarians hip 8 (July 1982): 137-43. 33. Matthews, Lawrence, and Ferguson, Using Online Catalogs. 34. Christine L. Borgman, End User Behavior on the Ohio State University Libraries' Online Catalog: A Computer Monitoring Study, Research Report Prepared for OCLC (Dublin, Ohio: OCLC Online Computer Library Center, 1983). 35. Gouke and Pease, "Title Searches in an Online Catalog and a Card Catalog." 36. Jean Dickson, "An Analysis of User Errors in Searching an Online Catalog," Cataloging & Classification Quarterly 4 (Spring 1984): 19-38. 37. Jerry Specht, "Patron Use of an Online Circulation System in Known-Item Searching," Journal of the American Society for Information Science 31 (Sept. 1980): 335-46. 38. Dickson," An Analysis of User Errors." 39. Arlene G. Taylor, "Authority Files in Online Catalogs: An Investigation of Their Value," Cataloging & Classification Quarterly 4 (Spring 1984): 1-17. 40. Peters, "When Smart People Fail." 41. Zink, "Monitoring User Search Success." 42. Alexandra Dimitroff, "Mental Models Theory and Search Outcome in a Bibliographic Retrieval System," Library & Information Science Research 14 (April/June 1992): 141-56. 43. Holmes and Bulger, "A Day in the Life of a Public Terminal"; Janet Kinsella and Philip Bryant, "Online Public Access Catalog Research in the United Kingdom: An Overview," Library Trends 35 (Spring 1987): 619-29. 44. Borgman, End User Behavior. 45. Bennett, "BIS Online Catalog Use"; Weakley, "Online Catalog Use." 46. Dickson, "An Analysis of User Errors"; Taylor, "Authority Files in Online Catalogs." 47. Peters, "When Smart People Fail." 48. Zink, "Monitoring User Search Success." 49. Ibid. 50. Dickson, "An Analysis of User Errors"; Taylor," Authority Files in Online Catalogs." 51. Peters, "When Smart People Fail." 52. Holmes and Bulger, "A Day in the Life of a Public Terminal." 53. Bennett, "BIS Online Catalog Use"; Weakley, "Online Catalog Use." 54. Dickson, "An Analysis of User Errors." 280 College & Research Libraries May1995 55. Matthews, Lawrence, and Ferguson, Using Online Catalogs; Lawrence, Graham, and Presley, "University of California Users." 56. Holmes and Bulger, "A Day in the Life of a Public Terminal." 57. Specht, "Patron Use of an Online Circulation System." 58. Tagliacozzo, Rosenberg, and Kochen, "Access and Recognition"; Lipetz, User Requirements in Identifying Desired Works. 59. Matthews, Lawrence, and Ferguson, Using Online Catalogs. 60. Lipetz, User Requirements. 61. Tagliacozzo, Rosenberg, and Kochen, "Access and Recognition." 62. Ibid., 234. 63. Ann T. Curran and Henriette D. Avram, The Identification of Data Elements in Bibliographic Records, Final Report of the Special Project on Data Elements (United States of America Standards Institute, Sectional Committee on Library Work and Documentation (Z-39), Subcommittee on Machine Input Records (SC-2), May 1967). 64. Matthews, Lawrence, and Ferguson, Using Online Catalogs. APPENDIX Interview Form for Study of Known-Item Searches Date: ----,-...'-----':-------- Su~ey#: ________________ _ We are conducting a research project on how people use the online catalog. Would you be willing to participate? It will take less than five minutes. 1. What are you looking for today? 0 Subject search (Something on ... ) Thank you and stop. 0 Known item (A book, a book by ... ) Give the respondent the full consent form. 0 Book 0 Journal 0 Verify call number Do you have any questions about the research? I would like to remind you that you may withdraw from the project at any time. 2. Did you bring any written information about the book/article with you? 0 Yes. May I see the information and make a copy of it? 0 No. What do you know about the item? 0 Author 0 Title 0 Author and title 0 Part of the title 0 Editor 0 Publisher 0 Su~ect: ---------------------------------------------- Is there anything else you know about the item? 3. Where did you get this information? 0 Professor/teacher told me about it 0 Class reading list 0 Reference in: 0 Book 0 Friend told me about it 0 Journal 0 Citation from : 0 CD-ROM search 0 Online search Database: 0 Other:------------------------,-----------------------------,- 4. What will you be using the book/article for? 0 Class assignment D Dissertation/thesis 0 Course preparation D Verify reference(s) for publication 0 Personal 0 Other research 0 Other: ----------------------------------------------------- The "Known" in Known-Item Searches 281 5. I need to get some basic information about you. Are you a student or faculty member? D Undergraduate student D Graduate student D Student from another school: ------------------ What department are you in? -------------------- D Faculty D Independent researcher/scholar D Faculty from another school: ------------------ What department are you in? ----------~~-------- D Member of local community D Library staff D Oilier: -------------------------- 6. When you're finished using the online catalog, will you please come back and let me know whether you found the book/article in ilie catalog? Time for start of search: -------~ Mter the search: 7. Did you find what you were looking for? D Yes Did you get: D More than you ~eeded D Exactly what you needed Time for return : ________ _ D Not what you were looking for, but similar items that will satisfy your need. D No D It was in another library. Which? ----------------,----'- 0 Oilier: D We don't own it. D It was checked out. D Other: COLLEGE & RESEARCH LIBRARIES INDEX TO ADVERTISERS ALA 193 Biosis cover 3 Blackwell 194 Ebsco Subscription Service 189 InfoEN Associates 257 Library Technologies 248 OCLC 220 PAIS cover 2,191 Personal Bibliographic Software 197 Readmore 198,292 Todd cover 4 H. W. Wilson 285,287