College and Research Libraries The Effectiveness of a Projected Computerized Presentation in Teaching Online Library Catalog Searching Nancy L. Buchanan, Karen Rupp-Serrano, and Johanne LaGrange A computerized presentation teaching NOTIS commands and search strategy was produced using Show Partner F/X software. Students from a freshman composition class viewed the presentation and completed a NOTIS search exercise. A second group of students from the same class completed the exercise without having viewed the presentation. A comparison of the two groups' exercises and transaction logs show that the group that viewed the presentation proved better at using correct NOTIS commands and formulating effective search strategies. The greatest differences between the two groups were in subject searching (Library of Congress subject heading and keyword) and in locating specific items. II he online public access cata- log is quick! y becoming a part of life in the modem library. In the past patrons struggled with the intricacies and plain unwieldi- ness of drawer upon drawer of three-by- five-inch cards. Now they wrestle with the vagaries of computer terminals. While the online library catalog is a true blessing to practicing librarians, we want to guarantee that the library catalog is equally useful to and usable by patrons. To this end, librarians have engaged in one of our oldest and fondest pastimes: library instruction. In our eagerness to en- sure that patrons make the best possible use of computerized catalogs, we try a variety of teaching methods: handouts, lectures, de- monstrations, help screens, and computer- ized instruction. But after all our strenuous efforts, have we really had any impact? This study, conducted at Texas A&M University's Sterling C. Evans Library, is one attempt to answer this question. Using the software program Show Part- ner F /X, two reference librarians and one cataloger created a computerized presentation explaining how to use NOTIS, our online library catalog. The presentation was projected onto a large viewing screen and accompanied by an audiotape narration. Three sections of the required freshman-level composition course viewed the presentation. The stu- dents in these sections, plus students from three sections of the same course who did not view the presentation, then completed a library exercise on NOTIS use. We collected and examined the students' exercises, as well as the transaction logs from the NOTIS searches the students did while completing the exercise. The results Nancy L. Buchanan is Staff Instntction Librarian at the Sterling C. Evans Library, Texas A&M University, College Station, Texas 77843-5000; Karen Rupp-Serrano is Social Sciences Librarian at the University Libraries, University of Oklahoma, Norman, Oklahoma, 73019; and ]ohanne LaGrange is Cataloger at the Health Sciences Library at Columbia University, New York, New York, 10032. 307 308 College & Research Libraries show that while the students who viewed the presentation may not have librarian-level searching ability, they were able to search NOTIS more cor- rectly and effectively than their non- viewing counterparts. Our instructional efforts did pay off. After examining previous related stu- dies, this paper describes the presenta- tion, examines the differences in NOTIS searching abilities between the presenta- tion viewers and nonviewers, recounts students' reactions to NOTIS and the presentation, and discusses the implica- tions of this study for future in- structional efforts. PREVIOUS RESEARCH A long history of research concerning the value of user training exists. Re- searchers have explored the role of in- struction in improving patron satisfaction with OPACs and have also attempted to evaluate user-education methods. Betsy Baker and Brian Nielsen discussed the development of instructional methodolo- gies for OPACs, noting how instructional theory and library technology have merged. 1 Baker and Nielsen also re- viewed the professional literature which outlines conflicting schools of thought on the value of user education. Baker and Nielsen conducted a sub- stantive study of user-training research involving the NOTIS system at North- western University. The study, sup- ported by a grant from the Council on Library Resources, concentrates on es- tablishing an effective instructional pro- gram and on evaluating instructional models through such techniques as test- ing and transaction logs.2 Researchers particularly mention that transaction log data would be "invaluable to grounding librarians in concrete knowledge about the behavior of users." 3 However, they also note that before any conclusions might be reached "a period of consider- able experimentation and practical exami- nation of what works and doesn't work" in regard to OPAC instruction needs to be undertaken.4 Nielsen, Baker, and Beth Sandore's final report to the Council on Library Resources details information July 1992 on the project and discusses its objec- tives, design, findings, and conclusions.5 The final report also analyzes OPAC in- structional theory, concepts, planning and objectives, as well as instructional content and motivational factors. 6 Several other studies also focus on OPAC user training or behavior. Karen Markey offers an overview of patron attitudes and quantitative statistical re- sults gathered during a study of library user needs in relation to OPACs. This extensive study, which involves 29 li- braries and user surveys, transaction logs, and interviews, emphasizes library patrons' use of types of assistance and the importance of printed materials.7 Mike Berger and Katharina Klemperer provide a more general overview of OPAC instruction, as they separate in- structional goals into several broad cate- gories, including teaching catalog use, teaching patrons about the library, and teaching research methods. 8 Our instructional efforts did pay off. A number of articles are helpful in addressing the use of computerized large-group presentations for user in- struction. Susan K. Charles, Keith A. Waddle, and Jacqueline B. Hambric dis- cuss the use of Show Partner to create a computerized presentation for training InfoTrac users.9 Nancy Gusack details the use of the SHOW program at UCLA. 10 Emily J. Batista and Deborah A. Einhorn discuss PC Storyboard and its applications in creating an instructional presentation for BRS/MENUS. 11 PRESENTATION The instructional presentation was created using the software package Show Partner F /X. This software, pro- duced by Brightbill-Roberts and Com- pany, is designed to create desktop presentations on IBM PCs and compati- bles. It consists of several integrated pro- grams that allow the user to import screens from a variety of software sys- tems as well as to create unique text and image screens. The imported and created screens are then arranged and presented in sequence by the software's run-time module, creating the presenta- tion in a way similar to how edited film is put together to create a videotape. The following equipment is needed to use Show Partner F /X: • an IBM PC, PS, or compatible with one or more disk drives (at least one hard disk drive preferred); • 320K of memory, assuming at least 256K of available memory (more memory may be needed for systems using an IBM Enhanced Graphics Adapter or IBM Video Graphics Array); • MS-DOS 2.0 or newer; • one of the following graphics adapters: IBM Color, MultiColor, or Enhanced Graphics Adapter; Hercules Mono- chrome Card; IBM Video Graphi~s Array; or a system offering 100 percent compatibility with one of these adapters; • one of the following monitors: IBM PC; IBM PS/2; IBM Enhanced Color Display; or 100 percent compatible; • Microsoft mouse or 100 percent com- patible (optional, but necessary to run the Object Editor application of the software). The first step in creating the presenta- tion was the selection of the NOTIS ele- ments to be included (these are discussed below). The creators then searched NOTIS for appropriate screens to illustrate the chosen points. An effort was made to ensure that the examples covered a range of academic disciplines. Once the NOTIS screens were selected, they were downloaded using Procomm telecom- munications software. These screens be- came the presentation's imported screens. With the instructional agenda and captured screens in hand, the creators wrote a script explaining the necessary points and incorporating the examples. Some of the screens were altered using Show Partner F /X. Most of the altering involved highlighting certain parts of the screens, but it also involved moving and eliminating screen elements. Several screens used in the presenta- tion were created from scratch. The Gra- Projected Computerized Presentation 309 FIX Editor application of Show Partner FIX is a paint program similar to Mac- Paint. It has many abilities, including the creation of lines, boxes, and circles, and the use of colors and patterns. The created screens enlivened the presenta- tion and provided a visual break from the standard NOTIS screens. They also helped to illustrate certain concepts. For example, the presentation used Venn di- agrams to explain Boolean logic. The creators then decided upon the screen order and selected which visual effects would be used as one screen gave way to another. A variety of effects is possible, such as a simple replacement (the entire screen changes at the same time, simulating how one NOTIS screen changes to another) or a vertical split- screen effect (the existing screen parts in the middle and moves outward, reveal- ing the new screen). The final two steps in preparing the presentation involved timing. The script was recorded, to ensure that the narra- tion in all the presentation was identical. The creators then determined the length of time each individual screen would be visible by playing the recorded narrative while viewing the presentation and painstakingly adjusting the number of seconds each screen would be projected. The presentation was then ready for viewing. The participating English classes watched the presentation in their class- rooms. This involved transporting a portable Compaq III, an audiotape player, a Dukane MagniView 400 liquid crystal display projection system, and a portable overhead projector. The presentation gave information on NOTIS 4.6 search commands and tech- niques. The following information ex- plains the NOTIS basics covered in the presentation and asked for in the exer- cise. Users search NOTIS by entering a command, which identifies what type of search the patron wants to conduct, and then entering the search term(s). The basic commands are "a="for author, "t=" for title, "s=" for Library of Con- gress subject heading, and "k=" for key- word. The author, title, and subject 310 College & Research Libraries heading commands retrieve all authors, titles, and subject headings that begin with the entered letters. Keyword lo- cates all items with cataloging records that contain the exact entered terms in the fields that are keyword indexed. In a keyword search, search terms can be truncated by entering a word stem fol- lowed by a dollar sign. A number can be placed after the dollar sign specifying words having up to and including that · number of additional letters. In a key- word search, the Boolean operators AND, OR, and NOT can be used. At any point, a user can access a general help screen by typing "h" or view a help screen on a particular type of search by typing the command letter followed by . NOTIS 4.6 has many other features and techniques (e.g., SAME Boolean operator, field limiting, local subject term searching), but these were neither covered in the presentation nor addressed in the exercise. METHOD The presentation's testing used six sections of English 104, a required fresh- man composition class. Three sections, each having a different instructor, viewed the presentation. Three sections, each taught by one of the same three in- structors, did not view the presentation. The students then had a week in which to complete a NOTIS exercise. While com- pleting the exercise was required, the students were not graded on their per- formance. A total of 68 viewers and 69 nonviewers completed the exercise. The NOTIS exercise consisted of demographic questions, questions about NOTIS commands and search strategy, questions asking students to locate items on NOTIS, and questions about there- spondent's opinions of and perceived use of NOTIS. The viewers' question- naires also included questions about the presentation. All the questions except those on reactions to NOTIS and the pre- sentation were multiple choice. Some asked students for one answer, while others asked them to mark all applicable answers. A copy of the questionnaire is available from the authors. July 1992 The respondents were asked to use NOTIS terminals in order to answer the questions and were seated at NOTIS ter- minals while they completed the exer- cise. In order to prevent respondents from asking for help from other partici- pants, librarians, or library users, the students completed the exercise during the evening under the supervision of the librarians conducting the study in the Processing Division of the library. The data were analyzed using Statpal, ami- crocomputer program for statistical col- lection and manipulation. Transaction logs were also collected for all the participants' NOTIS searches. A transaction log records every key- stroke entered into a NOTIS terminal. The transaction logs were identified by terminal number and time, making it possible to identify a search as having been conducted by a viewer or a non viewer. The transaction logs were examined for addi- tional information and insight relating to specific questions and issues. RESULTS A comparison of the questionnaires and transaction logs for the viewers and nonviewers makes clear that there are significant differences in the abilities of the two groups to understand and use proper NOTIS commands and to formu- late search strategies. The following sec- tion examines the differences in NOTIS use between viewers and nonviewers. An examination of demographic differ- ences between the two groups follows. Viewers versus Nonviewers Viewers demonstrated greater NOTIS searching ability in three areas: the use and understanding of basic NOTIS com- mands; subject searching (both keyword and Library of Congress subject head- ing); and the ability to locate specific items. Basic Searching. The smallest amount of difference between the viewers and nonviewers is apparent in the most basic types of searching: searching by author and title. While some differences were evident, these two concepts seemed the easiest for the nonviewers to grasp. Viewers exhibited a better under- standing of and performance ability in NOTIS author searching. When asked which of five commands was the correct one to use when searching by author for John Steinbeck's Travels with Charley, 85% of the viewers chose a=Steinbeck John as their only answer, while 79% of the nonviewers did so. When exercises with multiple responses are included, 93% of the viewers chose a=Steinbeck John, compared to 81% of the non- viewers. While the general understand- ing of title searching was similar between the two groups, those who viewed the presentation were much more conversant with its intricacies. While nonviewers performed only slightly more poorly than viewers in a straightforward author search question, a more complicated question showed that they have less understanding of what the a= command locates and when it is appropriate for use. When asked to locate books about Shakespeare, 48% of the nonviewers said a=Shakespeare Wil- liam would do this, while only 31% of viewers did so. At-test reveals a p value of .02 for these responses, meaning that there is a 2% probability a difference of this size in the mean score for this ques- tion between viewers and nonviewers would occur by chance.12 For 38% of the nonviewers a=Shakespeare William was their only answer to the subject search question, compared to 24% of the viewers. The difference between viewers and nonviewers in title searching was less related to their understanding of the t= command than their understanding of one technicality of it. When asked how they could locate The Sun Also Rises by Ernest Hemingway, 99% of the viewers and 94% of the nonviewers chose t= as a response. On a question asking how the respondent could find a book for which the respondent is given two possible titles and told that the author is either Cohen or Kohen, 97% of both viewers and nonviewers selected t= as one of their answers. Despite these similar re- sults, the transaction logs revealed a nonviewer problem with title searching that the questionnaires did not. Twenty- Projected Computerized Presentation 311 nine percent of the nonviewers at- tempted title searches with a title begin- ning a, an, or the, which is not allowed by NOTIS and was covered in the presenta- tion. Only 12% of the viewers did so. While the general understanding of title searching was similar between the two groups, those who viewed the presenta- tion were much more conversant with its intricacies. Subject Searching. Subject searching, the second area of difference, involves keyword searching and Library of Con- gress subject heading searching. Effec- tive keyword searching includes the ability to use truncation and Boolean oper- a tors. All of these skills were covered in the presentation and the exercise. Viewers demonstrated a clearer un- derstanding of what a subject heading does and when it is appropriate to search using subject headings. On the question asking students how they would locate books about Shakespeare, 53% of the viewers chose s=Shakespeare William as one of their answers, while only 36% of the nonviewers did so. At-test reveals that the responses to this question have a p value of .02. Viewers exhibited their better grasp of when subject heading searching was ap- propriate in several additional instances. When asked to locate books about movies, 34% of the viewers used s=, while 20% of the nonviewers did so. On a question in which subject heading searching was not an effective way to find the desired book (the students were told an author's last name was Cohen or Kohen and two var- iant titles), only 10% of the viewers tried s=. Nineteen percent of the nonviewers did so. Viewers were also more aware of in- stances where keyword was an appro- priate search strategy. When asked how they could use NOTIS to locate The Sun Also Rises by Ernest Hemingway, 59% of the viewers said they could use key- word, compared to 45% of the non- viewers. At-test reveals this question's results have a p value of .OS. All told, 35% of the viewers chose all three viable op- tions (t=, a=, k=) while not choosing s=, while only 25% of the non viewers did so. ' I 312 College &: Research Libraries Viewers also demonstrated a better grasp of the intricacies of keyword searching. When asked how they actu- ally used NOTIS to locate a book about which they knew that the author was Cohen or Kohen and that it had one of two titles, 46% of the viewers reported using a Boolean keyword search, com- pared to 23% of the nonviewers. The transaction logs reflect the increased use of Boolean searches by viewers. Fifty- three percent of the viewers employed correct Boolean keyword searches while doing the exercise (outside of the ques- tions asking about the Boolean AND and OR), compared to 39% of the non- viewers. Viewers were also more aware of instances where keyword was an appropriate search strategy. The students were tested on trunca- tion, an aspect of keyword searching, by a question that asked what "mason$" would retrieve. The correct response was all of the multiple-choice options offered. Fifty-four percent of the viewers responded correctly, compared to 33% of the non viewers. At-test reveals these re- sponses have a p value of .01. Two questions tested respondents' knowledge of the Boolean AND and OR. The questions asked what the search re- quests "k=cat and dog" and "k=cat or dog" would retrieve. Sixty-nine percent of the viewers gave the correct answer to the AND question, while only 36% of the nonviewers did so. Interestingly, the OR question was an instance where the non- viewers performed better than the viewers. Sixty-eight percent of the viewers said that OR would locate any items having either cat or dog in it, while 81% of the non viewers gave this correct response. When the AND /OR responses are cross-tabula ted for the viewers and nonviewers, the results show that 60% of the viewers still selected the correct an- swer for both questions, while 33% of the nonviewers did so. The viewers who selected incorrect answers did not fol- July 1992 low. a pattern in which incorrect answer they chose. The nonviewers had a pre- dilection for OR. Twenty-six percent selected the correct OR answer as their response to both questions. The transaction logs revealed an addi- tional way in which viewers had a better grasp of what is and is not an appro- priate way to search for items on subject. Thirty percent of the nonviewers at- tempted to locate items on a subject by using a= followed by a subject, exclusive of the Shakespeare question. Eighteen percent of the viewers did this. Locating Specific Items. Given the viewers' better grasp of basic NOTIS commands and subject searching, they unsurprisingly proved more adept at lo- cating specific items on NOTIS than the nonviewers. A simple question iden- tified a book by author (S. Hawking) and subject (superstrings) and asked stu- dents to indicate which of four call num- bers was the correct one for the desired item. Ninety-four percent of the viewers were able to do so, while only 80% of the nonviewers were successful. A t-test at- taches a p value of .01 to this question's responses. A similar question asked students whether or not the library has an anthro- pology film on the Yanomamo Indians. This was a trickier question, since the library has such a film but it cannot be located using s= Yanomamo. The Library of Congress subject heading is Yanoama Indians. This item could only be located using keyword. Sixty-nine percent of the viewers said the library had the film, while only 44% of the nonviewers re- sponded affirmatively. A t-test shows these results have a p value of .001. Not only does their better grasp of subject searching allow viewers to search for items on specific topics more effectively, it also enables them to locate specific items more successfully. Demographic Differences. The demo- graphic questions on the students' ques- tionnaires were designed with one purpose in mind: to identify differences between the viewing and nonviewing groups and to see if these differences might have affected the study's results. While there were two noticeable differ- ences between the viewers and non- viewers, these demographic differences appear to have had only a slight relation- ship to NOTIS searching skills. The questionnaire asked about the fol- lowing demographic variables: class; number of semesters at A&M; experience in using library catalogs, indexes, and al?- stracts; frequency of use of library cata- logs, indexes, and abstracts; credit hours of computer courses; frequency of com- puter use; and previous NOTIS use. Table 1 shows the differences between viewers and nonviewers in these areas. Two major differences between the groups are apparent. First, more viewers were upperclassmen. Even though En- glish 104 is a freshman-level class, only 44% of the viewers were freshmen, com- pared to 70% of the nonviewers. There were more sophomore, junior, and senior viewers than nonviewers. Not surprising given this, the viewers had been at A&M longer_ than the nonviewers. Twenty- seven percent of the viewers attended A&M for five or more semesters, com- pared to 7% of the nonviewers. The second major difference was in computer experience. Sixty-eight per- cent of the non viewers had not taken any computer classes, compared to 42% of the viewers. Viewers used computers more frequently than nonviewers, 21% noting they used them daily. Only 9% of the nonviewers used computers this frequently. While viewers claimed to have used NOTIS more frequently than non- viewers, the difference here was not as marked as that of class and computer experience. Viewers and nonviewers claimed similar . experience with and frequency of use of library catalogs, in- dexes, and abstracts. The students' seniority may have played a part in how they answered some of the questions about NOTIS. When their answers to questions men- tioned earlier are examined in light of this factor, as opposed to whether or not they viewed the presentation, discrepan- cies were apparent in two instances (see table 2). On the question asking respon- Projected Computerized Presentation 313 dents to indicate the correct call number of a book about superstrings by S. Hawk- ing, all but one of the incorrect responses came from a freshman. All but two of the incorrect answers, not surprisingly, were from students who had been at A&M two or fewer semesters. On the Yanomamo film question, freshmen and sopho- mores performed similarly, while seniors and especially juniors performed better. This pattern is not as strong when examined by number of semesters at A&M. It should be noted that several demographic categories (i.e., juniors, seniors, and those at A&M one or three semesters) do not have as large a popu- lation as other categories. Students' computer experiences reveal a similar pattern (see table 3). Ninety-five percent of students with any computer hours located the correct call number for the superstrings book, compared to 79% of students who had not taken computer courses. Students who never use com- puters located the correct call number 67% of the time, while those who use them once a month or more found the correct call number in from 86% to 100% of the attempts. This pattern does not hold true for the Yanomamo call number question, where students who never use computers actually performed better than any other group except those who use computers daily. It appears that the high number of upperclass viewers and more ex- perienced computer users (two distinct groups, as a higher class does not neces- sarily mean more computer use in this sample) may have slightly skewed the viewer/nonviewer results in these two questions. The varying and sometimes small numbers of respondents in these demographic categories make broader generalizations or a stronger statement difficult. REACTION TO NOTIS All students were asked how confi- dent they felt searching NOTIS in general; searching by author, title, Li- brary of Congress subject heading, and keyword; and how well they felt they used NOTIS. They were asked to re- 314 College & Research Libraries July 1992 TABLEt DEMOGRAPHIC DIFFERENCES %of %of Viewers Nonviewers Class Freshman 44 70 Sophomore 31 23 Junior 16 6 Senior 9 1 Semesters at A&M 1 7 6 2 43 62 3 6 4 4 18 20 5+ 27 7 Experience in Catalog Use Not experienced 12 6 29 30 43 51 13 12 Very experienced 3 1 Frequency of Catalog Use Never 13 7 1 x semester 38 39 1 x month 29 39 1 xweek 16 7 1+ x week 3 7 Computer Hours 0 42 68 3-6 55 30 9-12 3 15+ 0 0 Frequency of Computer Use Never 10 7 1 x month 28 48 1 xweek 29 23 2-6xweek 12 13 Daily 21 9 Prior NOTIS Use Never 21 26 1-2 X 24 29 3-7x 31 25 8-18 X 10 13 19+x 15 7 Projected Computerized Presentation 315 TABLE2 STUDENT SENIORITY Superstrings Correct Yanomamo Correct Seniority No. % No. % Freshman 61 78 40 51 Sophomore 37 100 20 54 Junior 14 93 12 80 Senior 7 100 5 71 One semester 8 89 5 56 Two semesters 57 79 38 53 Three semesters 6 86 6 86 Four semesters 26 100 12 46 Five+ semesters 22 96 16 70 TABLE3 COMPUTER EXPERIENCE Superstrings Correct Yanomamo Correct Computer Experience No. 0 computer hours 59 3-6 computer hours 56 9-12 computer hours 3 15+ computer hours N/A Never use computers 8 Use computers 1x month 45 Use computers 1x week 31 Use computers 2-6x week 17 Use computers daily 18 spond on a scale of one to five, one being "not confident" and "not well" and five being "very confident" and "very well." The responses, overall and broken down by viewing status, appear in table 4. Several observations can be made from the responses to these questions. First, viewers felt more confident using NOTIS and felt they used NOTIS better than nonviewers. In all instances but one, viewers had a higher average re- sponse. This is especially apparent in keyword confidence, where there is a .41 difference between viewers (average re- sponse 4.12) and nonviewers (average response 3.71 ). The greatest difference in the distribution of ratings by viewers and nonviewers is in the "very confident" and "very well" categories. Viewers were much more likely to respond at the high % No. % 79 38 51 97 35 60 100 3 100 N/A N/A N/A 67 8 67 87 24 46 86 21 58 100 10 59 90 14 70 end of the scale, while nonviewers tended to settle in the middle. The most positive responses went to author and title searching confidence, these being the only two categories to average above four with both viewers and non viewers. Over half of all viewers said they were very confident when title searching (52%) and author searching (50%). The figures for nonviewers are 48% for title and 39% for author. Viewers and nonviewers apparently felt identical in regard to Library of Con- gress subject heading searching confi- dence, giving that question the same average response. This question had the lowest average response, with the sec- ond lowest being how well the respon- dent felt he or she used NOTIS. Only 3% of the viewers and 6% of the non viewers 316 College & Research Libraries July 1992 TABLE4 PERCEIVED NOTIS USE AND REACTION TO PRESENTATION Question All Viewers Nonviewers 23. Overall searching (easy I difficult) 3.86 4.03 3.72 24. How well feel used NOTIS (not/very) 3.59 3.67 3.51 25. Confident overall (not/very) 3.78 3.88 3.68 26. Confident author (not/very) 4.26 4.41 4.12 27. Confident title (not/very) 4.39 4.41 4.36 28. Confident LCSH (not/very) 3.26 3.26 3.26 29. Confident keyword (not/very) 3.91 4.12 3.71 Note: All questions are on a 1-5 scale, with 1 beina the most negative and 5 being the most positive. The ends of the scales are indicate in parentheses. said they felt very confident when searching by subject heading. Interestingly, respondents' admitted confidence levels do not correlate to their NOTIS-searching abilities. On the question asking students which of five choices were Library of Congress subject headings, none of the 8% who answered correctly said they felt "very confident" searching by subject heading. None of those who said they were "very confi- dent" searching by subject heading an- swered correctly. On a question testing knowledge of truncation used in key- word searching, 36% of those choosing the correct answer were "very confi- dent" about keyword searching, while 55% of those choosing an incorrect an- swer were "very confident'' about key- word searching. All respondents were asked what made NOTIS easy and hard to use. While not all students answered all the open- ended questions, among those who did several trends emerged. The factor cited most often as making NOTIS easy to use was its on-screen in- structions. This was mentioned by 29% of the viewers and 30% of the non viewers, so even those with prior NOTIS instruc- tion found these helpful. The most men- tioned difficulty using NOTIS was knowing what terms to use when search- ing for items on a subject. Many of these comments specifically mentioned the Li- brary of Congress subject headings. These comments echo the students' low confidence in subject heading searching. RESPONSE TO THE PRESENTATION Those participants who viewed the presentation were asked for their re- sponses to it. On a scale of 1 (not helpful) to 5 (very helpful), they gave it a 3.76. When asked how helpful the presenta- tion was in regards to forming their search strategies, viewers rated it a 3.38. In addition to these ratings, open-ended questions asked what the students liked best and least about the presentation, what would make it more helpful, and what, if anything, in the presentation was not explained in an understandable manner. Comments elicited by these questions pointed in several directions. Of those who responded, 71% said the presenta- tion was understandable (52% of the total viewers). The most popular re- sponse to "what would make the presen- tation more helpful" was hands-on experience. This was selected by 41% of those who answered the question (13% of total viewers). Twenty-nine percent of those who responded said they thought the presentation was informative, and 31% complained that they already knew a lot of the information presented. DISCUSSION The most encouraging result of this study is that the viewers of the presenta- tion clearly demonstrated a better un- derstanding of how to search NOTIS and were more successful in their searches. The library instruction did make a differ- ence. When librarians are at the end of a week full of seemingly unending and unappreciated instructional sessions, it is reassuring and encouraging to know that our attempts are benefiting the li- brary's users. By highlighting the differences be- tween the viewers and nonviewers, this study points out the areas in which in- struction is most needed. The greatest areas of difference, as noted above, are subject heading and keyword searching. This is not surprising, as these are the more complicated aspects of online cat- alog use, but it reaffirms that this is where we should focus our instructional energies. In addition, this study shows areas that continue to be problems despite the instruction provided. Boolean search- ing, although employed to some effect by the viewers, is one area in clear need of additional explanation. Users' ability to perform truncation would also benefit from further instruction. Another major error among the viewers was the distinc- tion between a= and s= when an author's name was involved. These were an even greater problem among non- viewers. These problem areas suggest a need for more instruction in critical thinking as opposed to purely pro- cedural instruction. The type of instruction employed-a projected computerized presentation- points to another benefit of this study. This type of presentation can be effective in teaching online library catalog use. While no type of instruction can replace individualized or interactive instruction, staffing and budgeting realities make such teaching an impossibility. A computerized presentation such as this one can be pro- duced once and then used repeatedly to reach a large number of students in an effective manner. It could be projected large-screen (as in this study) in teaching classrooms or in the library or made avail- able on disk for viewing in campus com- puter laboratories. It also suggests that similar instructional tools that can be pro- duced once and viewed repeatedly, such as videotape, would be helpful. Two advan- tages of a computerized presentation, however, are that it can produce exact rep- Projected Computerized Presentation 317 licas of OPAC screens, and the creator can manipulate those screens to focus viewer attention as needed. While constructing a presentation such as this is certainly more work than produc- ing the most common instructional tool, a handout, the purposes and benefits of the two instructional methods are quite different. A computerized presentation such as this one is intended to introduce patrons to the online catalog before they need to use it. It is designed to communi- cate not just commands and procedures but to show the catalog in action. A pri- mary goal of any instructional session is to let potential patrons know what they need to use in the library, how to use the necessary tool(s), and to instill in them a positive attitude toward the libnuy and its resources. When the subject of the in- structional session is the online library cata- log, these goals are better accomplished when the patron can view the catalog, thus becoming familiar with what is, to many, a strange and confusing resource. Furthermore, patron use of printed instructions occurs only at the patron's discretion. The most useful of online cata- log handouts becomes useless when the patron declines to look at it. In NOTIS, as in many online systems, complete written instructions in online help screens are available to all users. These help screens are accessible at any point during NOTIS use, and users are alerted to them via on- screen instructions. However, the user must ask for them. A presentation such as this one gives the handout and help screen reader a head start, and the non- reader vital information he or she would otherwise be without. The methods employed in this study highlight a relatively new tool-transaction logs-with great potential for library re- search. While much of the data cited in this paper came from library exercises filled out by students, it was supple- mented by the records of the students' actual NOTIS searches. These logs can be used for several purposes, most notably discovering errors or problems that would not be evident from the exercises alone. The nonviewers' problem with beginning title searches with an article is 318 College & Research Libraries a good example of this. Transaction logs are physical reminders of how patrons really search a library catalog: what com- mands are problematic, what skills they lack, what errors they repeatedly make. Transaction logs do not need to be part of a formal study such as this to provide these benefits to a library. July 1992 This study brought valuable insight to the Sterling C. Evans Library. We hope it is useful to other libraries concerned with the value of library instruction and that it encourages similar studies in an effort to add continually to our store- house of knowledge about how and why patrons really use libraries. REFERENCES AND NOTES 1. Betsy Baker and Brian Nielsen, "Educating the Online Catalog User: Experiences and Plans at Northwestern University Library," Research Strategies 1:155-66 (Fall1983). 2. Ibid.; Betsy Baker, "A New Direction for Online Catalog Instruction," Information Technology and Libraries 5:35-41 (Mar. 1986); Brian Nielsen, Betsy Baker, and Beth Sand ore, "Educating the Online Catalog User: A Model for Instructional Development and Evaluation: Final Report to the Council on Library Resources, Inc.," CLR Grant no. 2055. (Jan. 5, 1985); Brian Nielsen and Betsy Baker, "Educating the Online Catalog User: A Model Evaluation Study," Library Trends 35:571-85 (Spring 1987). 3. Brian Nielsen, ''What They Say They Do and What They Do: Assessing Online Catalog Use Instruction through Transaction Monitoring," Information Technology and Libraries 5:28-34 (Mar. 1986). 4. Nielsen and Baker, "Educating the Online Catalog User," p.574. 5. Nielsen, Baker, and Sandore, "Educating the Online Catalog User: Final Report," p.1-102. 6. Ibid., p.3-33. 7. Karen Markey, "Offline and Online User Assistance for Online Catalog Searchers," Online 8:54-66 (May 1984). 8. Mike Berger and Katharina Klemperer, ''The Teaching Function of the Online Library Catalog: An Analysis of Current Catalogs and Prospects for the Future," in National Online Meeting Proceedings-1983, comps. Martha E. Williams and Thomas H. Hogan (Medford, N.J.: Learned Information, 1983), p.39-44. 9. Susan K. Charles, Keith A. Waddle, and Jacqueline B. Hambric, "Using Presentation Software to Train Laserdisk Database Users," Laserdisk Professional1:91-95 (Nov. 1988). 10. Nancy Gusack, "Learning to Use the Melvyl Catalog," DLA Bulletin 7:6-8,24 (1987). 11. Emily J. Batista and Deborah A. Einhorn, "Putting on a Show: Using Computer Graphics to Train End-Users," Online 11:88-92 (May 1987). 12. For information on t-tests and the assumptions underlying their validity, see Bruce J. Chalmer, Understanding Statistics (New York: Dekker, 1987), p.147-53.