College and Research Libraries Research Notes 263 Teaching Search Techniques on the Computerized Catalog and on the Traditional Card Catalog: A Comparative Study Opritsa D. Popa, Deborah A. Metzger, and James A. Singleton Libraries establishing online catalogs often re- tain their card catalogs until complete retroac- tive online conversion can be achieved; conse- quently, bibliographic instruction (Bl) librarians teach search strategies on both sys- tems. This study measures students' grasp of library concepts taught on- and offline and preferences for one system over the other. Results show that students prefer the online catalog. When students are taught search strategies online first, followed by lectures on the card catalog, their test scores increase more than for those taught in the reverse order. Per- formance and preference are similar in Ameri- can and international students. If incorporated in BI programs, these findings will improve teaching effectiveness. As in numerous libraries across the na- tion, at the University of California-Davis Shields Library, the Melvyl online catalog coexists with the traditional card catalog. Both catalogs will be maintained for many years until a complete online conversion occurs. Most students find the online card catalog an attractive library feature. On- line catalog user studies indicate that re- gardless of the degree of knowledge in us- ing the computerized catalog, students overwhelmingly prefer this system to the manual card catalog. 1 Of those who try the online catalog ''almost all change over to use it more often than the card catalog. " 2 Furthermore, for reasons that are not yet apparent, "OPAC (Online Public Access Catalog) users visit the library more fre- quently than do OP AC nonusers. " 3 In contrast, the library literature abounds with recriminations regarding the difficulties in using and teaching the traditional card catalog. It has been called ''complicated,'' ''esoteric,' ' 4 the ''greatest obstacle of library research for all but the most experienced patrons," and has even been branded "3 x 5 and full of holes. " 5 'Because of the problems associated with using the card catalog, bibliographic in- struction (BI) librarians have had difficulty motivating students to learn systematic search techniques. The reason is simple: the majority of students does not demand comprehensiveness or precision from the card catalog. Familiar with its setup and comfortable using it, students believe that they are fairly successful in their search- ing. This belief is reinforced when they find at least some useful references. 6 Any attempt to expand this superficial knowl- edge by teaching tracings, subject head- ings, cross-references, and filing rules is met with a complete lack of interest. At Shields Library the main card catalog is a union catalog for the University of California-Davis main library and its branches. It is divided into author/title and subject catalogs with access points by Opritsa D. Papa is Associate Librarian and Deborah A. Metzger is Associate Librarian at the Shields Library, University of Ollifornia, Davis, Ollifornia 95616. James A. Singleton is Staff Research Associate at the Labora- tory for Energy-Related Health Research, University of Ollifornia, Davis, Ollifornia 95616. This research was supported by a grant from the Librarians Association of the University of Ollifornia-Davis. Irene Hoffman's input during the initial stages of fhe study is gratefully acknowledged. 264 College & Research Libraries personal and corporate "author, title, sub- ject (LC authorized subject headings), and series title. Main entries display full biblio- graphic information with call number and location. Periodicals are accessible not only through the card catalog, but also by using a periodicals list on microfiche or a separate periodicals database, the Califor- nia Academic Libraries List of Serials (CALLS), loaded into the online catalog. The UC Melvyl online catalog has been available since 1981. It primarily contains records for the holdings of the nine Uni- versity of California campus libraries. As of October 1987 the catalog featured ap- proximately 6,500,000 records; of these 787,000 belong to the Davis campus. CALLS was added online in 1984. The Melvyl online catalog functions in parallel with the main card catalog. It comple- ments the latter by being more current: records are entered online months before cards are filed in the main card catalog. Melvyl is available for patron use when- ever the library is open, or it can be ac- cessed at any time by an outside personal computer and modem. Several library ter- minals have printers attached. Points of access parallel those of the card catalog and feature personal and corporate au- thor, title, LC subject headings, and series title. Additionally Melvyl permits access by subtitle, title keywords, and series key- words. Records can be displayed in are- view format (author, fraction of title, and year of publication), an abbreviated for- mat (bibliographic entry, call number, and campus location), or a long format (com- plete bibliographic record). As previously mentioned, periodical titles in complete or . abbreviated form may be searched in the separate periodicals file. The online cata- log is "user-friendly," allowing a menu- · search approach as well as a more flexible, rapid command mode for experienced searchers. Because the card catalog offers full retro- spective coverage but lacks currency, while the Melvyl catalog (as of this date) offers currency without complete retro- spective coverage, at times it is necessary to use both catalogs. This situation has prompted us to test the following hypoth- eses: May 1988 1. Similar search concepts can be taught on the Melvyl online catalog and on the traditional card catalog. Since the online catalog features the same access points as the card catalog- author, corporate author, title, subject, se- ries, periodical titles-it can accommodate search techniques traditionally taught on the card catalog. Furthermore, the "long display'' lists the same elements as the main entry on a catalog card, allowing BI librarians to show and explain online such fields as title/ authorship statement, edi- tion, imprint, collation, series, notes, trac- ings, and call number. The first hypothe- sis is critical to the study as all following hypotheses are dependent upon the dem- onstration that search concepts can be taught by using the online catalog. If the first hypothesis is true, then the potential advantage of teaching on the on- line catalog versus the card catalog needs to be demonstrated: 2. Teaching search concepts online is more effective than teaching the same concepts using the card catalog. Once the online catalog has been estab- lished as an effective bibliographic in- struction tool, the sequence in which the two systems are taught needs to be stud- ied: 3. The sequence in which the two sys- tems are taught plays a role in how well the concepts are understood. The student population of the Univer- sity of California-Davis is made up of in- ternational as well as American students. The fourth hypothesis compares the results of bibliographic instruction for these two groups: 4. Conceptual understanding and pref- erence of one system over the other are similar in American and international stu- dents. We also considered some general ques- tions about both catalogs and their use: After being exposed to both online and traditional card catalog searching, if the students are free to choose, which method will they prefer? Are usage, time spent on a system, and correct answers related? Which concepts are the most difficult to understand? METHODOLOGY AND DESIGN Subjects The subjects were selected from the UC- Davis student body according to the fol- lowing criteria: they had to be incoming students with no formal library training or experience with an online catalog and be willing to participate. American students were contacted at the first meeting of an introductory library credit course while in- ternational students were recruited at the annual introductory library lecture offered during their fall orientation week. Both the American and international students were invited to register for one of two ses- sions teaching library skills. Although 30 American and 30 international students registered for the sessions, only 29 Ameri- can and 22 international students actually attended. As much as we wanted to recruit sub- jects representative of these two student populations, the constraints of using li- brary "illiterate" students limited our choice to those willing to take the library training. Therefore both the American and international students were part of a select rather than random group. Study Design A cross-over design with a baseline (pre- test) and a posttest was used for the study (see figure 1). The pretest sought to deter- mine the initial level of the students' knowledge. After the pretest, the Ameri- can and international students were sepa- rated into two subgroups. The students who registered for the first session were assigned to the subgroup that was taught search techniques in the card catalog and then online, while the students who regis- tered for the second session were taught in the reverse order. Because some stu- dents who had registered failed to attend the sessions, the subgroups had uneven numbers of participants. Each subgroup was taught search strate- gies using either the card catalog (Off sub- group with 13 American and 6 interna- tional students) or Melvyl (On subgroup with 16 American and 16 international stu- dents). This first level of teaching con- sisted of a lecture and practice exercise fol- Research Notes 265 lowed by a test. A second level of teaching followed with the groups "crossing . over'': the On subgroup that had been ex- posed to searching on Melvyl was taught searching in the card catalog while the Off subgroup, which first learned to use the card catalog, was instructed in the use of the online catalog. A second test con- cluded this stage of teaching. A posttest was then given to both On and Off groups allowing students to apply their newly ac- quired search knowledge by using the sys- tem of their choice. Finally, an opinion survey gave students an opportunity to express their views about searching in the card catalog versus searching in the online catalog. Unlike a single group linear design, the cross-over design permits valid inter- group comparisons. 7 1t offers the possibil- ity of comparing test results of On and Off subgroups on the same level of teaching; it permits the analysis of whether teaching online first, followed by offline, facilitates the understanding of concepts. This de- sign also allows a comparison of On and Off subgroups in the preference for one system over another. Methods of Teaching The study was conducted in the first week of the fall quarter before students started using Shields Library. For both the online and card catalog lectures, identical teaching tools and lecture outlines were used. A slide presentation and a discus- sion covered the following topics: • Definition of the catalog • Coverage (types of publications and publication dates) • Access points • Record information (based on main en- try in the card catalog and the long dis- play on Melvyl) • Interpreting and locating material by call number • Filing rules (importance in the card cata- log versus online) The session concluded with -practical searching exercises in the card catalog and on the Melvyl catalog. A test consisting of simple and multiple-choice questions fol- lowed. The tests included comparable 266 College & Research Libraries STUDENTS RECRUITED I card catalog lecture I card catalog test Pretest I First Level Tests Melvyl lecture • Melvyl test card catalog Melvyl lecture lecture -!. Second I card catalog Level Melvyl ·~·~· Post Test Opinion Survey May 1988 QUESTIONS• /oJ onset, Is knowledge of group members comparable? Can same concepts be taught with both methods? tiypothesls ~ Is teaching online more effective than teaching olflne? ~ypothesls 2) Does sequence In which concepts are taught play a role In understanding and retention? ~ypothesis 3) Which concepts are the most cffflcult? Wil students answer post test questions using their favorite catalog? Are usage, time spent, and correct answers related? Are conceptual understanding ani!' preference almDar for American and International students? ~pothesls 4) LEGEND: OFF: Subgroup taught card catalog first ON: Subgroup taught Melvyl (onDne) catalog first • "'!:!'e h~heses are lsted In the same order as they appear In the text FIGURE! Cross-Over Study Flowchart questions relating to the seven key con- cepts (author, corporate author, title, sub- ject, LC subject headings, series title, and periodicals). Lectures and tests for each group were completed within a single four-and-one-half-hour session. The coded test results were statistically ana- lyzed using Mini tab Version 5. STATISTICAL ANALYSIS Descriptive statistical methods included · the calculation of mean, standard devia- tion, and when required by comparison of groups, a 95% confidence interval. At each step, overall test scores for each stu- dent (expressed as a percentage of items answered correctly) as well as key concept scores (percentage of students in a group answering correctly) on author, corporate author, title, series title, subject, LC sub- ject headings, and periodicals were re- corded. At every stage of the analysis, the distri- bution of the data was assessed to deter- mine the appropriate statistical method. If the distribution seemed normal, the t-test or ANOV A was applied. If the distribu- tion was skewed or irregular, nonpara- metric methods that relax the assump- tions to either symmetric distributions (i.e., Mann-Whitney, Wilcoxin) or no re- strictions (sign-procedures) were se- lected. 8 Proportions of students with cor- rect responses were evaluated with 95% confidence intervals, based on the bino- mial distribution. 9 Improvements in pro- portions were also evaluated with the bi- nomial distribution. 10 The pretest (baseline) scores were used: (1) to com- pare library skills at the beginning of the study; (2) to provide a reference level for change in knowledge; and (3) to decrease the bias due to differing amounts of knowledge. RESULTS Pretest Analysis: Determining the amount of library knowledge at the base- line. The initial level of library skills assessed by pretest scores (see figure 2) was similar in American and international students in both the On and Off subgroups. Confi- dence intervals (73% to 79%) for median pretest score based on the sign test all overlapped. Hypothesis 1: Similar search concepts Research Notes 267 can be taught using both the online cata- log and the card catalog. This hypothesis was analyzed by com- paring the results of the pretest and the level-one test. For concepts, the improve- ment from the pretest was expressed as (p2 - P1) = (b - c)/n where n = a + b + c + d; p1 and p2 are the proportion of stu- dents answering correctly on the pretest and first-level test, respectively; a is the number of subjects in the group answer- ing corr~ctly on both the pretest and first- -E 5 ! American Gro~p: Melvyl Questions ~4 : ~~~ :z 0 o 1 2 3 4 5 6 7 8 9 1 o 11 112 113 114 115 116 Mean Median Standard Deviation 7.1 7.5 2.4 I : • median I ( ) • 98% confidence interval for median (CJ.) QUESTIONS Pretest Score [] • 79% CJ. -E 5 !American Grour Card Catalog Questions ~4 : ~3 wl - 2 ! 0 : • median 0 1 I : ~ 0 1 1 CJ 1 1 Q 1 CJ ) 1 1 1 ( ) • 98% confidence interval 0 1 2 3 4 5 6 7 8 9 1 0 1112 13 14 15 16 for median (CJ.) QUESTIONS Pretest Score [] • 73% CJ. Mean Median Standard Deviation 7.7 7.0 3.8 -E 5 ! International Grf~up: Melvyl Questions ~4 : :::J 3 I : L .. CJ .I ([ l~ ..... o:::::nfdence ...... . 0 1 2 3 4 5 6 7 8 9 1 0 11 12 13 14 15 16 for median (CJ.) Pretest Score [] • 79% CJ. Mean Median Standard Deviation 7.4 8.0 3.6 -E 5 !International Groupi Card Catalog Questions ~4 : ~3 : ~2 Q 0: o 1 : ·median ~ 0 1 1 1 1 1 1 ~ 1 1 4) 1 1 1 ( ) • 97% confidence interval 0 1 2 3 4 5 6 7 8 9 1 0 1112 13 14 15 16 for median (CJJ QUESTIONS Pretest Score [] • 78% c.1. Mean Median Standard Deviation 8.2 8.0 2.9 FIGURE2 The Initial Level of Library Skills Assessed by Pretest Scores 268 College & Research Libraries level test; b is the number of subjects in the group improving from the pretest; c is the number of subjects in the group who an- swered correctly on the pretest but not on the first-level test; and d is the number of students who were incorrect in both tests. In the sign test that was used a significant improvement is indicated by improve- ments from wrong on the pretest to cor- rect on the first-level test occurring in ,_ more than 50% of the subjects that changed responses for pretest to first-level test. 11 We found that compared to the pretest, at the first-level test both the American and international students had signifi- cantly higher scores (see table 1), and with an exception for author and title, a better grasp of concepts (see table 2). The two concepts on which no significant improve- ment was observed were the easier ones- author and title. The small number of sub- jects in the international Off subgroup makes the assessment of their changes very difficult. However, when American and international students were com- bined across study groups, all concepts except author showed an improvement (p < .05); there was a small but nonsignifi- cant betterment on author (10%: b = 8, c = 3, p = .11). For other concepts the im- provement ranged from 43 percentage points (subject: b = 23, c = 1, p < .05) to 72 percentage points (LC subject head- ings: b = 37, c = 0, p < .05). The overall first test scores of the On subgroup in both American and international students were uncorrelated with the overall pretest scores (Pearson product-moment correla- tion r = .14 to .35, p > .05). However, in May 1988 the international Off subgroup, the higher the initial score, the higher the level-one test score (r = .80, .05roving (b) 14 10 10 3 24 13 24 13 37 no. declining (c) 0 0 0 0 0 0 0 0 0 no. not changing (a + d) 2 3 6 3 8 6 5 9 14 Title Improvement(%) 32% 23 0 0 15 16 27 0 15 no . improving (b) 6 3 2 1 8 4 9 3 12 no. declining (c) 1 0 2 1 3 1 1 3 4 no. not changing (a + d) 9 10 12 4 21 14 19 16 35 Periodical Improvement (%) 81% 54 81 0 81 37 69 59 64 no. improving (b) 13 7 13 1 26 8 20 14 34 no. declining (c) 0 0 0 1 0 1 0 1 1 no. not changing (a + d) 3 6 3 4 6 10 9 7 16 Subject 43% 50 0 46 37 49 37 43 Improvement (%) 54 no. imf>roving (b) 8 7 8 0 16 7 15 8 23 no . declining (c) 1 0 0 0 1 0 1 0 1 no . not changing (a + d) 7 6 8 6 15 12 13 14 27 Legend : On = Group taught online searching first . Off = Group taught manual card catalog first . No . = Number of students . Im- provement(%) = (p2 -p1) where p1 and p2 are the proportion of students answering correctly on the pretest and first-level test, respec- tively . Explanation: Improvements are statistically significant for LC subject headings, periodical, and subject concepts in all groups except in_ternational Off (p < .05). Statistical significance was evaluated by comparing proportions ri-c to .5 using the binomial distribution (s1gn test) . 270 College & Research Libraries May 1988 TABLE 3 SEQUENCE EFFECT OF TEACHING American International Levell: mean of correct answer Level 2: mean of correct answer Mean of sum On 16 Students 75.5 83.3 Off 13 Students 79.3 73.0 On 16 Students 74.7 81.8 Off 6 Students 65.7 76.8 ± standard deviation Difference (On- Off) 158.8 ± 11.3 152.3 ± 16.1 156.5 ± 21.4 142.5 ± 28.6 6.5 14.0 (95% confidence interval based on t-test) Combined differences ( -4.5, 17.6) ( -16.4, 44.4) . 8.5 ( -2.9, 19.9) Explanation: Although the means obtained by students taught online first are higher than the means of students taught on the card catalog first, the differences are not statistically significant. TABLE4 SEQUENCE EFFECT ON CONCEPTS TAUGHT: COMPARISON OF STUDENTS' CORRECT ANSWERS AT THE FIRST AND SECOND TEST* Difference Between Concept % of Students with Correct Answers on Both Tests Overall On and Off Groups (% On-% Off Students American and International Combined) CA -Corporate author PE -Penodical LC -LC Subject headings SE -Series SU-Subject TI -Title AU-Author 51 72 72 74 76 82 92 30 23 23 1 13 5 5 Legend : On = Group taught online first . Off = Group taught card catalog first. *Percentages of total number of students in the group answering correctly on both tests . groups and therefore displayed the largest variability. With such a small sample size, one or two people may have a large impact on the group's summary statistics. GENERAL OBSERVATIONS CONCERNING THE CATALOGS AND THEIR USE Students' Favored Method Out of 51 stude~ts, only 3 (all in the in- ternational On subgroup) preferred the card catalog; these 3 also answered most of the posttest questions by using the card catalog. Thus for all groups usage was re- lated to preference with all but 3 students preferring and using the Melvyl catalog. Time versus Co"ect Answer Regardless of concept and group (Amer- ican or international), most questions were reported as answered within 5 min- utes (93% of posttest questions). Out of the 51 subjects, those taking 6 or more minutes to answer a question ranged from one (author and corporate author con- cepts) to 12 (subject concept). For 5 of the 7 concepts the percent of correct answers was higher for those taking 0-5 minutes to answer than for those taking 6 minutes or longer, i.e., LC subject headings (87% ver- sus 50%, p > .05), title (86% versus 50%, p > .05), subject (62% versus 42%, p > .05), corporate author (86% versus 0%, p > .05), and series (87% versus 75%, p > .05). This suggests that the longer it took to answer a question, the less likely the an- swer was correct, although the observed differences may have arisen by chance. Difficulty of Concepts Within and between groups (On and American 1nn ON OFF ON OFF no. 16 13 16 6 Author Pretest 94 85 75 83 First Test 94 100 100 67 Second Test 100 100 94 100 Post· Test 94 100 100 100 Library of Pretest 0 15 13 33 Congress First Test 88 92 75 83 Subject Second Test 100 61 100 83 Headings Post-Test 88 61 94 67 Title Pretest 56 77 81 83 First Test 88100 81 83 Second Test 100 77100100 Post-Test 81 77 94 83 Periodical Pretest 13 31 19 17 First Test 94 85 100 17 Second Test 81 92 88100 Post-Test 88 85 100 83 Subject Pretest 38 31 38 67 First Test 81 85 88 67 Second Test 100 92 94 83 Post-Test 75 62 50 17 Corporate Pretest Author First Test 56 77 94 50 Second Test 94 54 75 83 Post· Test 94 n 81 83 Series Pretest First Test 81 100 81 67 Second Test 94 77 94100 Post-Test 88100 88 50 Am. - American students lnt'l • International students ON - group taught online searching first ~%: : ~~g.~::?~~~al card catalog first FIGURE3 Research Notes 271 AI Subjects AI AI Overall ON OFF Am. lnfl 32 19 29 22 51 84 84 90 n 84 97 89 97 91 94 97 100 100 95 98 97 100 97 100 98 6 21 7 18 12 81 89 90 n 84 100 68 83 95 88 91 63 76 86 80 69 79 66 82 73 84 95 93 82 88 100 84 90 100 94 88 79 79 91 84 16 26 21 18 20 97 63 90 77 84 84 95 86 91 88 94 84 86 95 90 38 42 34 45 39 84 79 83 82 82 97 89 97 91 94 63 47 69 41 57 75 68 66 82 73 84 63 76 77 76 88 79 86 82 84 81 89 90 77 84 94 84 86 95 90 88 84 93 77 84 Key Concept Scores of Different Tests: Percentage of Group Answering Correctly by Concept Off) the concepts were ranked according to the percent of overall correct answers in decreasing order. The most difficult pre- test concepts were LC subject headings, periodicals, and subjects (figure 3). On the combined first- and second-level of teach- ing, corporate author ranked number one (table 4). Table 4 also suggests that in the first- and second-level tests the On group performed better than the Off group for the 3 most difficult concepts. For interna- tional students, subjects remained the most difficult concept on the posttest, fol- lowed in descending order of difficulty by series, corporate author, LC subject head- ings, title, and periodical. For American students, on the posttest subject was the most difficult concept as well, followed by LC subject headings, title, corporate au- thor, periodical, and series (figure 3). Opinion Survey Analysis Since the results of the opinion survey did not show any evidence of sequence or nationality effects, results on survey ques- tions were pooled. All but 3 of 48 respond- ing students felt that Melvyl was easier to use. "82°/o of all students considered Melvyl a 'lot of fun' or 'some fun' to use.'' Asked about the degree of ''fun'' in us- ing the system, 82% of all students consid- ered Melvyl a "lot of fun" or "some fun" to use. No group differences were evident (81% of the American On subgroup, 77% of the American Off subuoup, 80% of the international On subgroup, and 100% of the international Off subgroup). In con- trast the card catalog was found "OK" to ''boring'' by 98% of all students (100% of the American On, American Off, and in- ternational Off subgroup; 93% interna- tional On subgroup). The majority (51%) of the responding 272 College & Research Libraries students felt that they were II somewhat'' in control when using Melvyl, and 55% had the same response for the card cata- log. This pattern was the same for all groups except for students in the Ameri- can On subgroup. Fifty percent of these students felt in "total control" on Melvyl. Overall 18% felt unsure when using Melvyl, while 15% felt unsure when using the card catalog. Finally, we probed the degree of satis- faction with each system. Overall, 59% of the responding students were satisfied with Melvyl compared with only 21% sat- isfied with the card catalog (62% were par- tially satisfied with the results found using the card catalog). The majority of the American On subgroup (73%), American Off subgroup (62%), and international On subgroup (53%), were very satisfied with Melvyl. When using the card catalog, the majority was only "somewhat satisfied" with its use (64% of the American On sub- group, 75% of the American Off sub- group, 47% of the international On sub- group, and 67% of the international Off subgroup). 11 An important finding of this study is that similar search concepts can be taught equally well on the online cat- alog and on the traditional card cata- log." DISCUSSION An important finding of this study is that similar search concepts can be taught equally well on the online catalog and on the traditional card catalog (see hypothe- sis 1, tables 1 and 2). Since published user studies stress repeatedly the fascination of students with the computerized card cata- log, bibliographic instruction librarians can capitalize on this interest and teach search strategies online first, followed by a review of the same concepts in the card catalog. Due to small sample sizes, the study falls short of concluding that teaching li- May 1988 brary skills on the online catalog is more effective than teaching on the traditional card catalog (see hypothesis 2, table 1, and figure 3). By looking at the differences be- tween On and Off scores, it appears that there is no advantage to On versus Off teaching. One explanation could be that the Melvyl test was more difficult because the students struggled with two new elements-learning the search techniques and the search concepts. Conceivably this additional degree of difficulty could lower the scores. However, if online teaching is indeed more effective than offline teach- ing, it would result in higher scores. Con- sequently, these two factors would cancel each other, resulting in no observable On versus Off differences. Such a possibility although not tested formally is supported by the overwhelming satisfaction with Melvyl training. The third hypothesis addressed the im- portance of teaching the systems in a se- quence that leads to optimum results in understanding of search concepts. Obser- vation convinced us that students showed increased interest and more active partici- pation during the lecture dealing with on- line searching, despite the fact that both card catalog and Melvyl lectures used slide presentations to avoid a teaching ad- vantage created by the natural curiosity of students about an interactive system. Teaching concepts on the online catalog followed by teaching concepts on the card catalog leads to better understanding of search concepts than instruction in the re- verse order. For American students there was a 6.5% advantage of being taught on- line first. For international students, this advantage was even higher, 14.0%. The combined advantage was 8.5% (table 3). The scores for concepts also suggest a se- quence effect although this was a trend rather than a statistically significant con- clusion. This sequence effect appears to be more evident in the most difficult con- . cepts (table 4). Regardless of nationality the most diffi- cult concepts to grasp were subject and LC subject headings (figure 3). Librarians have long known that author and title searching are immediately understood while LC subject headings are harde.r to grasp. Our data reinforces these findings through statistical observation. The overwhelming use of the online cat- . alog during the posttest represents an im- plicit vote for this method of searching. The online catalog was viewed as easier to use and ''more fun'' than the card catalog. Surprisingly, after a relatively short initia- tion to online searching, students felt "somewhat" in control on Melvyl (51%). They had a similar (55%) response for con- trol of the card catalog, a method presum- ably used throughout their high school years. Finally, most students were ''satis- fied" with the results regardless of method. Melvyl received more "very sat- isfied" comments than the card catalog. Due to the rigorous admission criteria at the UC-Davis, international students have relatively few language difficulties. Their library knowledge is comparable to that of their American counterparts. Dur- ing the study, conceptual understanding, sequence effect, and preference for the on- line system were similar in the American and international subgroups (hypothesis 4). The only difference was the use and preference for the card catalog by the in- ternational On subgroup. This might be due to cultural differences, previous fa- miliarity with the card catalog, weariness of computers, or simply a "copy cat" ef- fect while being tested. The small group size ·and the lack of ran- dom selection in the study groups were the major limitations of the study. Small groups created problems of low power and large variability that did not permit conclusive data in hypotheses 2 and 3. It Research Notes 273 would take approximately thirty-six stu- dents per group (two to three times this study's group size) to have an 80% chance of detecting (reaching statistical signifi- cance) a true difference of ten percentage points from zero for a sequence effect.u Unfortunately for these two hypotheses we could not secure the necessary number of volunteers required by this poststudy analysis to reach a level of statistical signif- icance. Our conclusions are based on the use of a select rather than random group of students. Thus, more research is needed to determine whether these con- clusions can be applied to students with li- brary skills and precollege background that is different from that of our study group. Despite these limitations, several con- clusions are suggested by the data: 1. The online catalog is an excellent bib- liographic instruction tool that can accom- modate search concepts originally taught only for the card catalog. 2. Explaining search strategies with the use of the online catalog, followed by a re- view of concepts in the card catalog, is the suggested sequence for teaching search concepts. 3. Bibliographic instruction librarians ought to devote special time and attention to the concepts of subject and LC subject headings as methods of accessing the cata- log regardless of the type of catalog being taught. 4. The online system is well liked and thus students will use it more often and more effectively than the traditional card catalog. REFERENCES 1. David F. Bishop, "The CRL OPAC Study: Analysis of ARL User Response," Information Technol- ogy and Libraries 2, no.3:315-21 (1983); Pauline A. Cochrane and Karen Markey, "Catalog Use Studies Since the Introduction of Interactive Catalogs: Impact on Design for Subject Access,'' Li- brary and Information Science Research 5, no.4:337-63 (1983); Mary Neoel Gouke and Sue Pease, "Ti- tle Searches in an Online Catalog and a Card Catalog: A Comparative Study of Patron Success in Two Libraries," The Journal of Academic Librarianship 8, no.3:137-243 (1982); Karen Markey, "Thus Spake the OPAC User," Information Technology and Libraries 2, no.4:381-87 (1983). 2. Sue Pease and M. N. Gouke, "Patterns of Use in an Online Catalog and Card Catalog," College & Research Libraries 43:279-91 (1982). 3. P. A. Cochrane, "Catalog Use Studies," p.353. 4. Evan Ira Farber, "Catalog Dependency," Library Journal109, no.3:325-28 (1984) . 274 College & Research Libraries May 1988 5. Mary George and Sharon Hogan, "Teaching Tough Stuff," Research Strategies 1, no.3:136-38 (1983). 6. Evan Ira Farber, "Catalog Dependency," p.326. 7. M. Hills and P. Armitage, ''The Two-Period Cross-Over Clinical Trial,'' British Journal of Clinical Phannacology 8:7-20 (1979); A. Huitson, "A Review of Cross-Over Trials," The Statistician 31, no.1:71-80 (1983). 8. B. F. Ryan, B. L. Joiner, and T. A. Ryan, Minitab Handbook (Boston: Duxbury Press, 1985); E. L. Lehmann and H. J. D' Abrera, Nonparametrics: Statistical Methods Based on Ranks (San Francisco: Holden-Day Inc., 1975), p .55-120, 269. 9. C. J. Clopper and E. S. Pearson, "Th.e Use of Confidence or Fiducial Limits illustrated in the Case of the Binomial," Biometrika 26:404-13 (1934). 10. J. L. Fliess, Statistical Methods for Rates and Proportions (New York: John Wiley, 1973), p.72-77. 11. Ibid. 12. Hills and Armitage, "The Two-Period Cross-Over." APPENDIX A: CROSS-OVER ANALYSIS Considering the unequal group sizes, unequal variances, and skewed distributions (table 1), we did not use a cross-over analysis of variance. We expressed the observed score as the sum of the four ef- fects studied including random variation: score (stg) = u + M + L + S + e (stg), where: u = Overall mean effect common to an scores M = Melvyl effect, i.e., the advantage ofMelvyl teaching and test; this is a composite of actual teaching mode effects and test difficulty differences. They are not separable with the de- sign used. This effect is zero for card catalog test scores. L = Learning improvement effect due to second level of teaching; it is zero for a first-level test. S = Sequence effect, the advantage of.having the online first instead of the offline teaching. This can be interpreted as an interaction between teaching mode and level or as carryover or residual effects from the level. It is zero in the first level tests and in the offline tests. e(stg) = Random deviation of the score from the sum of the above effects due to chance and indi- vidual variability. It is assumed that these average to zero within each teaching mode (t) and group (g) combination. They are unique for each student(s}, and assumed indepen- dent of other students scores. Since the e(stg) average to zero for each (t,g), we can express the true mean score as: u + M(if M test) + L (if second test) + S (if On group second test). The true mean score for each (t,g) combination is shown below. Thus the difference of sums across teaching level (Sum) represents the sequence effect. If the difference in mean sums deviates significantly from zero, we may conclude that a sequence effect exists. Note that this is independent of teaching mode, test difficulty, and level of teaching effects. Level On Off 1 u+M u 2 u+L+S u+M+L Sum 2u + M + L + S 2u + M + L Sum (On)-Sum (Off)= S Where: On = group taught on Melvyl first. Off = group taught on the card catalog first. THE RESOURCE AUTHORITIES. rleueaeD4ae ID.fonaa,ioa oa Rller U pro4uw ua4 HniCII ollecte4: D Newspaper Indexes D Television News Transcripts and Indexes D Journals in Microform D Research Collections in Microform 12 Lunar Drive/Drawer AS Woodbridge, CT 06525 Toll-free: 1-800-REACH·RP TWX: 710-465~5 FAX: 203-397 ·3893 Over 200 years of factual reporting ... dozens of acclaimed international newspapers like The Times, The Washington Post, Le Monde, Der Spiegel, Asahi Shimbun, The Age, Financial Times, Jewish Chronicle .. . all of them available on microfilm from Research Publications. Imagine the research value of The Times alone: every issue since 1785 -reporting events like the French Revolution, the first flight by the Wright Brothers, and the cultural revolution in China. Scholars, researchers, and the inquisitive public at large have invaluable access to every word. This is probably the most inexhaustible, high-quality collection of inter- national newspapers ever preserved in microform- spanning hundreds of years of news publishing, on most of the world's continents, and with editorial content that's limitless in subject matter and interpretation. For further information, utilize the form below. Or to order current subscriptions and backfiles, call Research Publications todey at 1-800-REACH-RP ( 1-800-732-2477). From Connecticut, Alaska and Canada, call collect 203-397-2600. International Newspapers 0 Please send me your brochures. 0 Have a representative call me. Name Inst!tut!on Address City Phons T!Ue State Zip Code For further information, or to place your order directly, call1-800-REACH-RP (1-800-732-2477). · CRL D U.S. and Intsrna- tion&! Patent Documentation D Patent Search and Awareness Services D Reference Books 1-800-BEACH-BP