College and Research Libraries Research Notes Database Selection by Patrons Using CD-ROM Gillian Allen A variety of bibliographic databases on CD- ROM are now available. This study examined the appropriateness (as determined by three in- dependent judges) of database selection by pa- trons, when a choice of databases was provided. It was found that one patron in five used the most appropriate database available, while one patron in five used an entirely inappropriate database. Patrons followed the same search procedure if they were using an appropriate database or not. Experience in searching also had no effect on database selection. Patrons us- ing appropriate or inappropriate databases had similar opinions about their need for training and about how successful their searches were. Academic libraries are increasingly adopting bibliographic databases on CD- ROM. While libraries at one time acquired one system, some libraries are now pur- chasing a number of different CD-ROM systems: Previous research has shown that patrons like to use CD-ROM and be- lieve that they reguire little training to use these databases. 1 To a great extent, these studies of CD-ROM involved surveys of patrons using Info Trac. 2 But little research has been done to date in libraries which of- fer patrons a choice of bibliographic data- bases on CD-ROM. There has been no published study on the appropriateness of the databases selected by patrons when a variety of databases is provided. The pur- pose of this study was to examine the database selection by patrons at the Un- dergraduate Library of the University of il- linois at Urbana-Champaign (UIUC), and to determine the appropriateness of that database selection in searching specific topics. BACKGROUND Few of the studies of CD-ROM have compared the use of a variety of CD-ROM databases. Jane Kleiner compared the use of Info Trac, Legal Trac and Government Pub- lications Index on CD-ROM. 3 In that study, 77 percent of the patrons searched the Info Trac database, although the majority of us- ers realized that other databases were available. Eighty-six percent of those sur- veyed considered that they had retrieved Gillian Allen is a master's degree student at the Graduate School of Library and Information Science, Univer- sity of Illinois at Urbana-Champaign, Champaign, Illinois 61820-6291. 69 70 College & Research Libraries adequate information from Info Trac. Karla Pearce found that PsycLIT pro- duced the largest proportion of satisfac- tory searches of the eight databases in her study. 4 Eighty-seven percent of the users surveyed felt that they had retrieved rele- vant citations. However, in both of those studies, the appropriateness of database selection was not determined. Appropriateness, in this context, relates to the suitability of the database in terms of its subject content for use in searching a specific topic. METHODOLOGY In August 1988, the Undergraduate Li- brary at UIUC established a CD-ROM lab area, that has a number of workstations equipped with CD-ROM bibliographic database systems. There are two worksta- tions equipped with Info Trac and Readers' Guide (Wilson), and a number of worksta- tions equipped with other Wilsondisc sys- tems. Additional copies of some of these CD-ROM databases are available else- where in the University, and a few data- \ bases not held by the Undergraduate Li- brary are held by other departmental libraries on campus. This study examined the use by patrons of Info Trac and nine Wilson bibliographic databases on CD-ROM in the lab area. During the period of the study, eighty- two searches were observed. The data- base selected by each patron and the topic of each search were recorded. Three inde- pendent judges then selected the three most appropriate databases available to patrons for each of the search topics, and the choices of the patrons and judges were compared. The Wilsondisc systems included Read- ers' Guide, Applied Science and Technology Index, Biological and Agricultural Index, MLA Bibliography, Business Periodicals In- dex, Humanities Index, Education Index, Art Index and the Index to Legal Periodicals. The subject content of the databases in the Un- dergraduate Library CD-ROM lab ranges from general, such as Info Trac or Readers' Guide, to specialized, such as Art Index and MLA Bibliography. In addition to these ten databases avail- able in the CD-ROM lab of the Undergrad- January 1990 uate Library, six other databases that are available elsewhere in the library system were added to create a list of sixteen data- bases available to patrons for end-user searching. The six databases added were ERIC, PsycLIT, Medline, ABI/Inform, Da- text, and PAIS. Each of these databases is located in a departmental library in build- ings that are adjacent to the Undergradu- ate Library. Undergraduates were able to go to the appropriate departmental library to use the database that was the most suit- able for the subject of the search. The three independent judges were ex- perienced in using all of the CD-ROM databases available within the University. Also, to aid them in selecting the three most appropriate databases, each judge was provided with a list of the sixteen databases that are available to patrons within the University, and a description of the subject content of each of the data- bases. The independent judges were asked to select the three most suitable databases from the list provided for each of the searches included in the study. Judges were to name their first, second, and third choices of databases. These se- lections by the independent judges were then compared with the databases actu- ally selected by the patrons for their searches. . If the database selected by the patron was the same database chosen as the most appropriate database for the search topic by an independent judge, the search was awarded three points. If the database se- lected by the patron was the database de- termined to be second best by an indepen- dent judge, the search was awarded two points. If the database selected by the pa- tron was a judge's third choice, the search was awarded one point. This process was repeated for the three judges for each of the eighty-two search topics. The total score on each search would therefore range from zero points (complete dis- agreement between database selected by patron and databases selected as best, sec- ond best or third best by all independent judges) to nine points (complete agree- ment between patron and independent judges: database selected by patron was considered to be the most appropriate database for the search by all three inde- pendent judges). The number of cycles (that is, the time from initial input of a search statement to the input of another search statement or end of the search) through which each pa- tron progressed during the search was also counted. A success was recorded when citations were considered valuable enough to be printed out. A failure was re- corded when citations were not found or were not printed. The amount of time taken to perform each search was also re- corded. Patrons were asked a number of ques- tions about their searches and their search results. They were asked which biblio- graphic databases they had used prior to their current search in order to determine if experience in searching had an effect on database selection. They were ask~d if they felt they had found some information on their search topics, and if they had suc- cessfully answered their research ques- tions, to determine how positively they felt about their search results. The opin- ions of the patrons were also compared with the findings of the previous studies that reported patrons were generally sa tis- 0 .000 1.000 Po1 nts Awarded 2.000 3.000 By Independent 4.000 5.000 Judges 6.000 7.000 8.000 9.000 0 5 5 Research Notes 71 fied with their search results (retrieved rel- evant citations 5 and adequate informa- tion6). Patrons were also asked if they pre- ferred using bibliographic databases on CD-ROM or comparable print reference tools. They were asked what they in- tended to do next in their research. Possi- ble responses included: look up the cita- tions found, go to another database on CD-ROM or to a print source, ask for help, or quit. It was expected that patrons using appropriate databases would be able to proceed with their research, but that pa- trons using inappropriate databases would have to look further for relevant ci- tations before proceeding with their re- search. FINDINGS The results from the study were ana- lyzed using a statistical package (Mac SS). In conducting hypothesis tests, the level of acceptable Type 1 error was set at a=O.OS. Correlations were tested using Spearman's rank order correlation. When database selection by patrons was compared with database selection by in- dependent judges, the highest percentage 10 15 20 25 19.51. 3.66. 9.761 10.98. 8.54. 4.881 7.32. 3.661 9.761 21.951 10 15 20 25 Number of Searches FIGURE 1 Appropriateness of Database Selection 72 College & Research Libraries was for absolute agreement between pa- trons and judges. The highest possible score of 9 points was received in 21.95 per- cent of the eighty-two searches. The sec- ond highest percentage was for absolute disagreement between patrons and inde- pendent judges: 19.51 percent of the searches received the lowest possible score of 0 points. ''These findings seem to indicate that patrons may not know that they are working in an inappropriate data- base, and may continue with their searches as though they were retriev- ing useful citations.'' There was no significant difference in the way patrons conducted their searches using appropriate or inappropriate data- bases. No significant correlations were found between appropriateness of data- base and the number of cycles completed in the searches, or the time taken to per- form the searches. These findings seem to indicate that patrons may not know that they are working in an appropriate data- base, and may continue with their searches as though they were retrieving useful citations. Table 1 shows the rela- tionship between appropriateness of data- base selected and the numbers of cycles, failures and successes that were experi- enced by the patrons. The numbers in the "Inappropriate" column contain the results of the searches in which 0-2 points were awarded, based on the level of agree- ment between patrons and independent judges as to appropriateness of database selection. In these searches, at least one judge considered the database selected to be inappropriate. The "Marginal" column results are for those searches that were awarded 3-6 points, and the "Appropri- ate'' column contains those searches where agreement between patrons and judges was strongest: those awarded 7-9 points. In these searches, at least one judge consid- ered the database selected to be the most appropriate. Although ANOV A showed significant differences in the number of cycles January 1990 (p < .04) and in the number of failures (p < .01), these significant differences ap- pear to have been caused by the fact that a larger proportion of searches using Info Trac were in the ''Marginal'' group of cy- cles and failures. Seven out of the ten searches using Info Trac had results that were included in the ''Marginal'' group, and it was found that Info Trac searches re- quired more cycles and produced more failures than Wilsondisc searches. The results using the Wilsondisc systems were more consistent and therefore more evenly spread out between the three classes of scores. In these searches, the number of cycles completed and the num- bers of successes and failures achieved were similar regardless of the level of ap- propriateness of database selected. This confirms the findings of the correlation analysis that appropriateness of database did not affect search characteristics. There was also no significant correlation between appropriateness of database se- lection and the number of failures or suc- cesses in the search cycles completed by the patrons. At first glance, this result im- plies that, because there was no apparent . difference in the search process measured in cycles or in the proportion of successes or failures when using appropriate or in- appropriate databases, it did not matter which database was used by the patrons. While some patrons who were in inappro- priate databases found no citations on their search topics, (for example, the pa- tron who was searching in MLA Bibliogra- phy for articles on "mattresses") and left the CD-ROM lab without printing any ar- ticles, the majority of patrons printed out citations from inappropriate databases. Given the scatter of literature in a variety of journals, and hence in a number of databases, it is not surprising that some hits worth pursuing were even found in inappropriate databases. Although subse- quent use of citations was not investigated in this research, one wonders whether the printed citations were actually used by the patrons. There was no significant correlation be- tween appropriateness of database used and the number of databases that had been used previously. This implies that patrons did not benefit from past search- ing experience in improving their data- base selection ability. There was no significant correlation be- tween appropriateness of database used and opinions of the patrons of the extent to which they found some information in their searches. However, there was a modest and nearly significant relationship between appropriateness of database se- lection and opinions of the patrons that they had been successful in answering their research questions (R=-.2063, p<.06). There was a significant positive correla- tion (R = .2182, p < .05) between appropri- ateness of database selection and patron preference for using bibliographic data- bases on CD-ROM or print reference tools (Spearman's R Correlation). Patrons us- ing appropriate databases expressed a stronger preference for using print refer- ence tools. Although these patrons were using CD-ROM appropriately, they still demonstrated a preference for using the traditional reference tools. There were no significant differences based on appropriateness of database se- lection in the subsequent activities of the patrons (Kruskal-Wallis ANOVA). The largest group of patrons (51.2%) went di- rectly from searching to looking for the cited articles. Of these, 26.2% were using the most appropriate databases, and 14.3% were using totally inappropriate databases. It is reasonable to assume that the 26.2% of patrons using appropriate databases would have found useful cita- tions to look up, and it seems reasonable to assume that the 14.3% of patrons in the wrong databases would not have found as useful citations. The remaining patrons did not move from finding citations to looking up the ci- tations, but they were going to ask for help, move to other reference tools, or quit. Of these patrons, 17.5% were using the most appropriate databases, and 25% were using totally inappropriate data- bases. It appears reasonable that the pa- trons using inappropriate databases would not continue with their searches, but it is difficult to understand why the 17.5% of patrons using appropriate data- Research Notes 73 bases did not continue by looking up the citations found. They may have, for exam- ple, found unexpected citations, or may have been using the appropriate data- bases incorrectly and retrieving citations that were not useful. The motivations for the subsequent actions of the patrons were not investigated in this research. There were significant differences in the degree of appropriateness of database se- lection attached to the bibliographic data- bases on CD-ROM that were used in each of the searches (Kruskal-Wallis ANOVA). Table 2 shows the average number of points awarded to the ten searches carried out in each of the seven databases. For ex- ample, the highest agreement between patrons and independent judges on ap- propriateness occurred with patrons us- ing the Biological and Agricultural Index (ap- propriateness rating of 6. 9 out of the possible 9 points). The average search car- ried out using the Education Index scored only 2 points (out of a possible 9 points) on appropriateness. The average score on ap- propriateness was 4.43 out of the potential 9 points. The most appropriate databases to be used in searching were those databases that are specialized in their subject con- tent, rather than general in content. The TABLE 1 SEARCH CHARACTERISTICS BY APPROPRIATENESS Inappropriate Marginal Appropriate Cycles (avg.) Failures (avg.) Successes (avg.) 2.15 1.07 1.11 5.2 3.77 1.42 TABLE2 APPROPRIATENESS OF DATABASES SELECTED Database Biological & Agricultural Index Business Perioaicals Index Applied Science & Technology Index Av_g. of all databases Info Trac Readers' Guide Humanities Index Education Index 3.35 1.79 1.55 Avg . Points Awarded 6.9 6.7 6.4 4.43 3.4 2.9 2.7 2.0 74 College & Research Libraries closest matches of opinions between pa- trons and independent judges were on the selection of the two specialized science databases and the business database. There was a low level of agreement on the appropriateness of the selection of both Readers' Guide and Info Trac, and the great- est difference of opinion was on the selec- tion of the Humanities and Education In- dexes. This could relate to the fact that both of these databases, as well as Info Trac and Readers' Guide, cover a wide range of sub- ject content. IMPLICATIONS OF THE RESEARCH The research provides information on the appropriateness of database selection by patrons using bibliographic databases on CD-ROM in the academic environ- ment. Approximately 22 percent of the pa- trons selected a database that experts felt was entirely appropriate for the search be- ing conducted. Nearly as many patrons selected a database that was not even con- sidered to be one of the three most appro- priate databases for the search topic. However, all patrons went through ap- proximately the same search process, and there was no evidence of any results of in- appropriate database selection during the search process. These findings raise some doubts about patron expectations from searching bibliographic databases on CD- ROM, as well as about their searching abil- ity. 11 Approximately 60 percent of the time, the independent judges were not unanimous in their selection of the most appropriate database to be used for searching a specific topic.'' The results also show that there were considerable differences between data- bases. Those patrons using the more spe- cialized databases received higher scores January 1990 than those patrons using the more general databases for appropriateness in their searches. The research also shows that some pa- trons selected the most appropriate data- bases without any assistance or training in database selection. But other patrons did not make adequate use of the biblio- graphic databases on CD-ROM. Approxi- mately 60 percent of the time, the inde- pendent judges were not unanimous in their selection of the most appropriate database to be used for searching a specific topic. It follows that more than one data- base could be appropriate for many searches. Yet only a few patrons moved from one bibliographic database on CD- ROM to another during their searches, in order to retrieve more relevant citations. It follows, then, that for approximately 80 percent of the patrons, some training or assistance in database selection would have been advantageous. There are a number of areas where fu- ture research would be valuable. The value of training in improving the data- base selection ability of patrons could be assessed. Aspects of patron search behav- ior could be studied, for example, why they continue with searches and how they make use of printed citations. Effects of lab area layout on database selection could be studied. Despite the presence of .signage in the lab area, many patrons did not realize there was a variety of biblio- graphic databases on CD-ROM from which to choose, and they may have used the most accessible database rather than the most appropriate database for their searches. Librarians should be aware of the signif- icant differences in database selection abil- ity of their patrons, and of the desirability of providing training or assistance in data- base selection, in situations where patrons are offered a choice in the bibliographic databases on CD-ROM that could be used in their searches. REFERENCES 1. Published studies of patron response to CD-ROM' s include: Ann Bristow Beltran, "Info Trac at In- diana University: A Second Look," Database 10:48-50 (Feb. 1987); Ann Bristow Beltran, "Use of Info Research Notes 75 Trac in a University Library," Database 9:63-66 Oune 1986); Beryl Glitz, "Testing the New Technol- ogy: MEDLINE on CD-ROM in an Academic Health Sciences Library," Special Libraries 79:28-33 (Winter 1988); Jane P. Kleiner, "Info Trac: An Evaluation of System Use and Potential in Research Libraries," RQ 27:252-63 (Winter 1987); and Mary Ann Walker and Helen Westneat, "Using Info Trac in an Academic Library," Reference Services Review 13:17-22 (Winter 1985). 2. Published studies of patron response to Info Trac include: Beltran, "Info Trac at Indiana University: A Second Look"; Beltran, "Use of Info Trac in a University Library"; and Walker and Westneat, ' ' Using Info Trac in an Academic Library.'' 3. Kleiner, "Info Trac: An Evaluation of System Use and Potential in Research Libraries." 4. Karla J. Pearce, "CD-ROM: Caveat Emptor," Library ]ournal113:37-38 (Feb. 1, 1988). 5. Ibid . 6. Kleiner, "Info Trac: An Evaluation of System Use and Potential in Research Libraries." \J~R~Nl~~~ocl£~: l-N:£0 RM-c_rl'liC~L Lis publication brings j_" .~ l t together in one place basic statistics about ~ srr~rrt 1{~ libraries in the United States that describe what .t'- 0 ~ 1 ~ }_ libraries are and do, how they are used and by C.:.- \\\1\- 1 .t'- whom, what they contain, how they are financed ~ \) .l., .l- and staffed, and how they spend their resources. It was compiled, with the aid of a grant from OCLC, for those outside the library community who wish to know more about libraries as well as for librarians and potential librarians who seek a summary of the field . • 32 pages • Paperback, 8W' x 11 " • Date: 1987 • ISBN: 0-8389-7145-8 ORDER FROM : ALA PUBLISHING SERVICES 50 East Huron Street Chicago, IL 60611 NAME ADDRESS __ Copies @ $4.00 each $ ___ Total CITY STATE ZIP WILSONDISC OFFERS • CD-ROM Access Search each database on a separate compact disc. And, with the touch of a button, you can update your search online! • Online Access For access to the most current data, subscribers to each WILSON DISC database receive unlimited online search time in that database. You pay only the nominal hourly telecommunica- tions rate-no additional charge! Three Search Modes for Users at All Levels 1 BROWSE Mode The same access points as the familiar Wilson printed indexes. 2 WILSEARCH® Mode Easy-to-follow menu options simplify search formulation. WILSONLINE® Mode The same commands available with the WILSONLINE online retrieval system . Unlimited Online Searching with Your WILSONDISC Subscription Subscribers to a WILSONDISC database can also take advantage of unlimited online searching in that database, paying only a nominal telecommunications charge. VIDEO! Now available- How to Set Up WILSON DISC (NO-CHARGE with Your Initial Subscription) How to set up WILSONDISC and tailor it to the needs of your library. WILSONDISC Databases • Applied Science & Technology Index • Art Index • Biography Index • Biological & Agricultural Index • Book Review Digest • Business Periodicals Index • Cumulative Book Index • Education Index • Essay and General Literature Index • General Science Index • Government Publications and Periodicals• GPO Monthly Catalog Index to Government Periodicals • Humanities Index • Index to Legal Periodicals • Library Literature • MLA International Bibliography• • Readers' Guide Abstracts • Readers' Guide to Periodical Literature • Religion Indexes• 1• Social Sciences Index • Database not produc ed by The H. W. Wil son Co mpa ny. To Order Call Toll-Free: 1·800·367·6770 In New York State call 1-800-462-6060; in Canada call collect 1-212-588-8400. Telefax 1-212-590-1617. THE H.W.WILSON COMPANY 950 University Ave . Bronx, NY 1 0452