College and Research Libraries DAVID N. KING AND JOHN C. ORY Effects of Library Instruction on Student Research: A Case Study The proliferation of library-instruction programs in academic libraries has reemphasized the importance of and need for the evaluation of program effectiveness. To date, evaluation efforts have not addressed the problem of determining the behavioral effects of instruction on student use of the li- brary. An evaluation methodology comparing student use of resources, ser- vices, libraries, and catalogs was developed as a means of evaluating the instruction program of the Undergraduate Library at the University of Illi- nois, Urbana-Champaign. The findings indicated significant differences be- tween instructed and uninstructed students in their library research efforts. ALTHOUGH MANY INSTRUCTION LIBRARIANS might disagree on matters of content and presentation, most would agree upon two ultimate goals that function as the raison d'etre of library user-education programs. The first goal is to improve students' ability to use library resources and services effec- tively to meet their information needs. 1 The second goal, often unstated in formal docu- mentation, is to instill realistic attitudes and expectations concerning the library and its accessibility. 2 Given adequate program sup- port, it is within the context of these two ultimate goals that almost every decision concerning content and form of presentation of user-education programs is made. 3 Evaluating the extent to which an instruc- tion program achieves these goals has been a particularly difficult problem for librar- ians, but in the early 1970s several signif- icant steps were taken in providing a David N. King is coordinator for user educa- tion, Houston Academy of Medicine, Texas Medical Center Library . At the time of this study, he was visiting research associate, University of Illinois Undergraduate Library. john C . Ory is coordina- tor of examination services, Division of Measure- ment and Research, Offi~e of Instructional Resources, University of Illinois, Urbana- Champaign. foundation for effective evaluation. The work of Thomas Kirk, John Lubans, Jr., and the ACRL Task Force on Bibliographic In- struction, among others, was indicative of the effort to identify meaningful objectives and methods of evaluation for instruction programs. 4 The terminology adopted at that time reflected the research in other fields. "Behavioral" objectives detailed the desir- able outcome of instruction concerning what students should know how to do in order to meet their information needs. Evaluations based on behavioral objectives, it was hoped, would measure the extent to which · students' knowledge of emphasized methods and resources had improved. 5 Other efforts involved the study of students' opinions, attitudes, and perceptions, both as a means of assessing instructional needs for students and as a method for evaluating the "affec- tive" or "attitudinal" changes brought about by instruction. 6 • 7 The methodologies employed for evalua- tion illustrate the influence of these de- velopments. Most of the evaluation tech- niques in use today were designed to deter- mine students' knowledge, skills, and opin- ions. A brief look at the importance and limitations of these approaches to evaluation provides background for the present study. I 31 32 I C allege & Research Libraries • ] anuary 1981 DETERMINING KNOWLEDGE AND SKILLS Testing has a long history in the evalua- tion of library-instruction programs. 8 Achievement testing by means of standard- ized or locally developed tests, often en- hanced by pre- and posttest comparisons, serves as a means of assessing students' knowledge of the cognitive or objective con- tent of instruction. 9 The assessment of stu- dents' library skills by means of workbook assignments and library exercises is repre- sentative of performance testing as an eval- uative technique. 1o Serious limitations exist concerning the value of paper-and-pencil examinations of any type. The most important limitation of tests is their "artificiality as devices for ascertaining a user's ability to negotiate the complex bibliographic structure of a li- brary." 11 Questions have arisen concerning the link, if any, between students' ability to perform well on such tests and their ability to use effective information-gathering tech- niques in the library. 12 Finally, testing offers no indication of how instruction influ- ences patterns of library use in unsuper- vised situations. DETERMINING OPINIONS The most prevalent of the methodologies currently in use for evaluation of instruction programs is the questionnaire designed to . elicit the opinions and perceptions of stu- dents and/or faculty concerning the value of instruction, the quality of presentation, the relevance of content, and attitudes toward the library. These studies may be relatively sophisticated, such as the attempts of John Lubans, Jr., to identify user needs and satisfaction. 13 More commonly seen are short questionnaires of the "Did-you-like- it?" variety. Questionnaires often request another sort of opinion as well. Students are commonly asked to assess their own abilities and the extent to which they think they have learned from instruction. 14 Evaluation of instruction programs by means of ques- tionnaires soliciting opinions, attitudes, and self-assessment might be conveniently termed perceptual evaluation. The importance of perceptual evaluation must not be minimized. Instilling positive attitudes toward the library and building confidence in library skills are major objec- tives for most instruction programs. If it is true, as James Benson has asserted, that the affective message of instruction is as critical as its cognitive content, perceptual evalua- tion is a necessary indicator of the conse- quences of any instruction program. 15 Unfortunately, the lack of objective observational data and the self-assessment nature of perceptual evaluation present in- terpretational difficulties when program eval- uation is based solely upon this method. 16 There may be a great deal of difference be- tween students' perceptions of their own abilities and their actual capabilities. There may be substantial difference between what a student has learned through library in- struction, and what that student may think has been learned. A similar gulf may exist between opinions expressed concerning the value of instructional content and the extent to which the content is of real use to the student. 17 DETERMINING USE Testing and perceptual evaluation provide a wealth of information concerning students' knowledge, skills, opinions, and attitudes- all of which is necessary for the develop- ment of an effective library-instruction program. 18 But critics have not hesitated to point out that the value of library instruc- tion can be proved only by demonstrating that instruction actually makes a difference in the way students use the library. 19 Given the number of studies of the use of libraries and materials in the recent past, it is remarkable that studies of student use of library services and resources as an evalua- tion methodology for instruction programs have not appeared in the literature. 2° Cer- tainly the design possibilities are numerous, ranging from the study of particular re- sources and services to study of the re- search strategy employed. A variety of tech- niques are available, including interviews, unobtrusive obser~ation, ~nd question- naires. The primary concern in the design of an evaluation methodology based on library use is that students not be influenced or directed in their choice of materials by the evaluation. The study should attempt to dis- cover how students actually use the library in order to meet their information needs, not how well they can follow library assign- ments or perform tasks under the critical eye of the evaluator. Only in that way might the results indicate what portion of the instruction content students adopt for use in an uncontrolled situation. Faced with the need for an evaluation of its library instruction program, an evalua- tion methodology was designed to yield data concerning the effects of instruction on stu- dent use of the library. Following is a re- port of the conduct and findings of that study. CASE STUDY21 Undergraduate Library Instruction Program Students in sixty-five rhetoric classes were among those who received Research Skills Instruction from one of a staff of four librarians and six half-time graduate assis- tants in the University of Illinois Under- graduate Library. Each class was scheduled to attend an instruction session in the Undergraduate Library at a time during which students were selecting topics for their required term papers, but before they had begun their library research. The Re- search Skills Instruction sessions, which .reached more than 1,300 freshmen during the spring 1979 semester and averaged seventy minutes in duration, were designed with two major objectives in mind. The first objective was to introduce stu- dents to the variety of information sources available to them for research. Complete re- liance upon monographs and popular periodical literature was discouraged and the use of more scholarly periodical litera- ture emphasized, along with media sources and government publications when appropriate. As a facet of this objective, be- cause of the vast holdings of the library, students were encouraged to be highly selective in their choice of materials. The instruction session detailed a coher- ent research methodology by which a sys- tematic manual search of the literature could be accomplished. One notable depar- ture from the traditional approach to teaching search strategy was that the role of bibliographies was de-emphasized. The uni- versal types of research tools and proper Effects of Library Instruction I 33 use of catalogs were discussed, with exam- . pies of many of the more useful sources and sample searches included. Individual titles were not emphasized. Several means of eval- uating the authority, appropriateness, and quality of materials located during the search were also discussed. The second major objective of instruction was to successfully acclimate students to the large, complex library system and to instill a sense of confidence in them. Typical prob- lems encountered by researchers, especially in regard to the location and retrieval of materials, were included in the discussion, along with solutions. The use of the Library Computer System (LCS), the library's on- line circulation system, was emphasized as an easy means of eliminating many of the common sources of frustration. Students from twenty-six of the sixty-five rhetoric classes that received Research Skills Instruction also attended a Work Ses- sion. Each of the 506 students received a list of the research tools identified as most appropriate for his or her research topic. The class period was spent in the Under- graduate Library, where students began their research under the guidance of a li- brarian. Instruction by a Rhetoric Teaching Assistant During the period the Undergraduate Li- brary instruction program was in progress, librarians learned that a teaching assistant in the Rhetoric Department was instructing two classes in library research methods. Since the objective of the teaching assistant in providing this instruction was very simi- lar to the major objectives of the library's instruction program, the authors arranged to include these students in the evaluation. Several differences in the approach taken by the teaching assistant should b'e noted, however. First, the teaching assistant offered a total of three class periods (240 minutes) of li- brary-related instruction. Students receiving instruction from the library staff were offered no more than two class periods. Second, the approach taken by the teaching assistant centered upon the use of specific sources, rather than upon universal types of tools as in the Research Skills In- 34 I College & Research Libraries • ] anuary 1981 struction program. Each student was as- signed an index title or reference source upon which the student was to report in class. Many of these students obtained assistance from Undergraduate Library reference per- sonnel in the use(s) of their assigned items. Finally, the teaching assistant did not in- clude a discussion of the Library Computer System. (LCS serves all the functions of the main card catalog, serial record, and shelf- list except that subject searching and full bibliographic record are not yet available. As a result, location and retrieval of items from any of the thirty-five-plus libraries scattered across campus and the main li- brary are greatly simplified by this on-line circulation system. Public terminals are available, and the search procedures easily learned.) Research Skills Instruction stressed the value of LCS and basic search commands. EVALUATION METHODOLOGY In an effort to assure that the content of instruction and the instructional techniques employed were meeting student needs, an evaluation was planned for the 1978/1979 academic year. The goal of the evaluation was to assess the value of library instruction as conducted for students enrolled in rheto- ric writing courses. Objectives for the eval- uation required that it should: 1. indicate the impact of instruction on student term-paper research behavior; 2. be objectively valid and reliable; 3. be ethically within the province of li- brarianship; 4. be cost and time efficient; and, 5. be easy to implement and capable of repetition for comparison of results. The goals and objectives dictated the ne- cessity of a behavioral approach to evalua- tion. The final instrument incorporated a perceptual evaluation section to enhance the thoroughness of the results. The criteria governing the study required that the in- formation-gathering techniques of students not be influenced by the evaluation meth- odology. Instrumentation Since the Undergraduate Library instruc- tion program was based upon behavioral objectives, the evaluation was designed to reveal the manner and extent of student use of the library system and its resources during term-paper research, and to reveal their perception of the library and of their ability to use the library effectively. A ques- tionnaire was designed to determine the effects of instruction on the following: 1. Use of reference sources , including encyclopedias, bibliographies , and, of greatest emphasis, periodical indexes and abstracts ; 2. Use of information sources, including monographs , periodical articles (which re- ceived greatest emphasis) , government pub- lications , and microform and media mate- rials; · 3. Catalog use, including the Under- graduate Library card catalog, the main card catalog in the main library , the serial record in the main library, Library of Con- gress Subject Headings, and the Library Computer System; 4. Use of libraries and services, including the Undergraduate Library reference ser- vice , the main library reference service, the term-paper counseling service, departmen- tal libraries, and the Undergraduate Library media center; and, 5. Student perceptions of the value of li- brary instruction , their opinions about the library, and confidence in their ability to use the libraries effectively. Data Collection During the last two weeks of the spring 1979 semester, when student term papers were due, the evaluation questionnaire was administered unannounced to three groups of freshman students enrolled in rhetoric courses. The three groups included : three randomly selected rhetoric classes that re- ceived no library-related instruction (46 students); two classes that received instruc- tion concerning library research from the rhetoric teaching assistant (28 students); and six randomly selected classes that received Research Skills Instruction, four of which also received a work session, from Under- graduate Library staff (106 students). Since term-paper assignments might have had an effect on student selection of sources and patterns of library use, a question form was sent to each of the rhetoric instructors whose classes were included in the evalua- tion. This question form was intended to determine : 1. the nature and requirements for term- paper assignments; 2. the importance of the quality of sources selected by students for term-paper research ; and, 3. for the classes that received instruction from the teaching assistant and the classes that had received no instruction , comments about the content of any library-related dis- cussions in the classes and opinions about the Undergraduate Library instruction pro- gram ; and for classes that had received in- struction from the Undergraduate Library, opinions of the quality and content of the instruction sessions. Instructor responses indicated a consistent attempt to encourage student 1,1se of a varie- ty of sources, though requirements rarely dictated the types of materials preferred. No difference was reported concerning the number of sources required for student pa- pers, but there were assignment differences that might have affected student responses to a few of the questionnaire items . These possible effects on student research be- havior are noted in the results. Data Analysis Mean responses to each questionnaire item were computed for the three groups of students: students who received instruction from the Undergraduate Library, students who received instruction from their rhetoric teaching assistant, and students who re- ceived no instruction. One-way analyses of variance statistics (ANOVA) were computed on student item responses to identify statis- Effects of Library Instruction I 35 tically significant (p < 0. 05) differences among group mean responses. REPORT OF RESULTS Use of Libraries and Resources Group response means and standard de- viations for each of the questionnaire items are presented along with ANOVA results in tables 1 through 4 . Significant (p < 0. 05) differences were observed among the three groups of students. In order to determine student use of ref- erence materials, students were asked to indicate how many encyclopedia titles , bib- liography titles, periodical index titles , and other reference titles were used in research- ing their papers (table 1). There was no significant difference between groups in the use of encyclopedias. Students who re- ceived no instruction reported greater mean use of bibliographies than did either of the instructed groups. Instructed students re- ported using an average of almost twice the number of periodical indexes than did unin- structed students. Since Research Skills In- struction placed so much emphasis on the use of indexes other than Readers' Guide to Periodical Literature , this finding was thought to be an indication of the effec- tiveness of instructional efforts. Li- brary-instructed students exhibited the greatest use of periodical index titles , with 74.5 percent reporting use of two titles or more . Of the students who received instruc- tion from their teaching assistant, 67.8 per- cent reported using more than one index ti- tle. But only 39.1 percent of the uninstruct- ed students responded that more than one title was used. Unsurprisingly , it was also found that the number of periodical articles used for documenting papers increased pro- TABLE 1 STUDENT USE OF REFERENCE SOURC ES : NUMBER OF TITLES USED* Library Instruction (L.I. ) Reference Sources Mean Use Biblio~raphies 0.91 Encyc opedias 0.33 Periodical indexes 2.42 Other 0.17 *Response scale was 0, 1, 2, 3, and 3 + . tp < 0.01. S. D. l.ll 0.63 1.32 0.61 Teaching Assistant Instruction (T.A.) Mean Use S.D . 0.96 1.35 0.25 0.52 2.46 1.23 0.43 0.84 No Instruction (N.O.) Significance Mean Use S. D. ANOVA Results 1.56 1.57 t N . O . > L.l. and T . A . 0.41 0.96 None 1.37 1.20 tL.I. and T.A. > N.O . 0.20 0.69 None 36 I College & Research Libraries • January 1981 portionately to the number of index- titles used for each of the three groups. (Pearson product moment correlation coefficients in- cluded: L.l. r = .32; T.A. r = .43; N.O. r = .52.) As a means of determining student use of information sources, students were asked to indicate how many books, articles, media and microform sources, and other informa- tion sources they used in researching their papers (table 2). Both the library-instructed students and the students who received in- struction from their teaching assistant re- ported greater mean use of periodical arti- cles than did uninstructed students. It was found that as mean use of articles increased, mean use of monographs decreased for both instructed groups (L.I. r = - .46; T.A. r = - .30). Students who received instruc- tion from their teaching assistant and stu- dents who received no instruction reported greater mean use of monographic literature than library-instructed students. This differ- ence may have been a result of the substan- tial emphasis placed upon the use of a . vari- ety of sources and selectivity in choice of materials by library instructors. Students who received instruction exhibited greater mean use of microform and media materials than did uninstructed students. Whereas 31.1 percent of the students who received instruction from the library and 39.3 per- cent of those who received instruction from their teaching assistant used microform and media materials, only 4.3 percent of the ~n­ instructed students reported using these materials. There was little difference in the reported use of government documents among the three groups. The difficulty of locating documents within the library sys- tem at that time was probably partially re- sponsible for the low use of these materials. To determine student use of catalogs, stu- dents were asked how many times they used the Undergraduate Library card cata- log, the main card catalog in the main li- brary, the serial record, the Library Com- puter System, and Library of Congress Subject Headings (table 3). A majority of TABLE 2 STUDENT USE OF INFORMATION SOURCES : NUMBER OF ITEMS USED* Library Instruction (L.I. ) Information Sources Mean Use Books 3.50 Periodical articles 3.09 Government publications 0.63 Microform and media materials 0.81 Other 0.07 *Response scale was 0, I, 2, 3, 4, and 4+. tp < 0.01. S.D. 1.80 1.92 1.11 1.48 0.32 Teaching Assistant Instruction (T.A. ) Mean Use S.D. 4.43 1.32 3.39 1.64 0.54 0.88 1.14 1.80 0.04 0.19 TABLE 3 No Instruction (N.O.) Mean Use S.D. 4.20 1.34 2.11 1.68 0.59 1.19 Significance ANOVA Results tT.A. and N.O. > L.l. tL.I. and T.A. > N.O. None 0.04 0.21 tL.I. and T.A. > N.O. 0.13 0. 75 None STUDENT USE OF CATALOGS : NUMBER OF TIMES USED* Catalogs Undergraduate Library card catalog Main card catalog Serial Record Library of Congress Subject Headings Library Computer System (LCS) Library Instruction (L.l.) Mean Use S.D. 2.67 1.34 1.47 1.44 0.68 1.11 0.19 0.59 1.30 1.52 •Response scale was 0, 1, 2, 3, and 3+ . tp < 0.01. :tp < 0.05. Teaching Assistant Instruction (T.A.) Mean Use S.D. 2.96 1.32 1.68 1.47 0.68 1.31 0.43 0.92 0.50 1.20 No Instruction (N.O .) Mean Use S.D. 2.61 1.50 1.83 1.69 1.28 1.62 0.13 0.50 0.59 1.09 Significance ANOVA Results None None tN.O. > L.l. and T.A. None *L.I. > T.A. and N.O. students from each group reported using the Undergraduate Library card catalog at least once (L.I. = 92.5 percent; T.A. = 92.9 percent; N.O. = 87 percent). Mean use was almost three times per student. The main card catalog in the main library was used slightly less often; about three- quarters of the students from each group re- ported using this catalog, with mean use to- taling less than twice per student in each group. The card catalog that proved an ex- ception in use patterns was the serial rec- ord, located in the main card catalog area of the main library. Students who received no instruction reported using the serial record almost twice the number of times per stu- dent as did instructed students. Considering the low use of periodical articles and peri- odical indexes exhibited by uninstructed students, this may be an indication of their lack of retrieval skills. Students who re- ceived instruction from the library staff demonstrated strong use of the Library Computer System. Mean use of LCS for this group was more than twice that re- ported by the other students, reflecting the impact of Research Skills Instruction. There was no significant difference in reported use of Library of Congress Subject Headings among groups. To determine student use of libraries and services, students were asked how many times they used the Undergraduate Library reference service, the term-paper counsel- ing service, the main library reference ser- vice, departmental libraries, and the media center (table 4). The responses of students in the three groups reported similar use of reference services. Mean use of the Under- Effects of Library Instruction I 31 graduate Library reference service was less than once per student. Only one-half of the students or less in each of the three groups responded that they asked for assistance at any of these service points. Among the unin- structed students, this result may coincide with their reported feelings of discomfort using the libraries, as indicated below. In- structed students may not have felt the need to use the services. Nonetheless, this low use was puzzling in the case of the Undergraduate Library reference service, which answered nearly 130,000 questions that year. 22 There was no significant difference in re- ported use of departmental libraries. The majority of students in each of the groups reported using a departmental library other than the Undergraduate Library at least once (N.O. = 76.1 percent; T.A. = 64.3 percent; L.l. = 62.3 percent). Mean use of the departmental libraries was greater than once per student. Students who received instruction from their teaching assistant re- ported greater mean use of the Under- graduate Library media center than did stu- dents from either of the other two groups. This result was probably due to the instruc- tor's requirement that each student select a term paper topic from the New York Times published on the date of the student's birth. That newspaper is available on microfilm in the media center as well as in the newspa- per library. Almost one-half of the students who received instruction from their teaching assistant and one-quarter of the li- brary-instructed students reported using the media center, compared to only one-tenth of the uninstructed students. TABLE 4 STUDENT USE OF LIBRARIES AND SERVICES : NUMBER OF TIMES USED* Library Instruction (L.I. ) Libraries and Services Mean Use Undergraduate Library reference service 0.89 Term-paper counseling service 0.45 Main library reference service 0.81 Departmental libraries 1.42 Media center 0.47 *Response scale was 0, 1, 2, 3, and 3 + . tp < 0.01. S. D. 1.10 0.68 1.30 1.45 1.01 Teaching Assistant Instruction No Instruction (T.A.) (N.O.) Significance Mean Use S.D . Mean Use S.D . ANOVA Results 0.96 1.27 0.54 0.86 None 0.54 1.11 0.54 0.75 None 0.75 0.97 0.83 1.22 None 1.54 1.48 1.89 1.51 None 1.18 1.49 0.13 0.45 tT.A. > L.l. and N.O. 38 I College & Research Libraries • January 1981 Student Perceptual Responses Student responses to opinion and self- evaluation items are presented along with ANOVA results in table 5. Again, it is re- vealing to consider items for which there was no variation, as well as those for which significant differences were apparent. The responses of students from all three groups indicated strong agreement that knowing how to use the library is impor- tant, and that a class or two devoted to using the libraries would be helpful to most new students. The majority of students ex- pressed the desire for more instruction (N.O. = 80.4 percent; T.A. = 71.4 per- cent, L.l. = 66 percent). Students who received instruction from the Undergraduate Library and those who received instruction from their teaching assistant exhibited a greater mean response concerning confidence in their ability to lo- cate materials in the Undergraduate Li- brary (L.I. = 88.6 percent; T.A. = 85.7 percent; N.O. = 67.4 percent). All groups reported somewhat less confidence in the ability to locate materials in the main li- brary, with no difference in responses be- tween groups. It was found, however, that TABLE 5 Statement 1. Everyone needs to know how to use the library. 2. I know how to lo- cate most of the materials I might need in the Under- graduate Library. 3. I know how to lo- cate most of the materials I might need in the main library. 4. A class or two de- voted to using the libraries would help most new stu- dents. 5. I feel comfortable asking for assis- tance in the Under- graduate Library. 6 . I feel comfortable asking for assis- tance in the main library. 7. Using the libraries is a frustrating ex- perience. 8. I wish more in- struction were available in how to do library research. 9. I feel confident us- ing the university libraries. 10. The university li- braries are difficult to use. STUDENT OPINIONS AND SELF-EVALUATION RESPONSES* Library Instruction (L.I. ) Mean S.D . 1.34 0.53 1.86 0.65 2.53 0.89 1.70 0.75 1.59 0.61 1.98 0.84 2.72 0.87 2.30 0.83 1.91 0.70 2.85 0.82 Teaching Assistant Instruction (T.A .) Mean S. D . 1.25 0.65 1.86 0.65 2.57 0.74 1.86 0.80 1. 75 0.75 2.18 0.82 2.26 0.94 2.11 0.85 2.00 0.61 2.79 0.74 No Instruction (N.O .) Significance ANOVA Results Mean S. D . 1.35 0.48 None 2.20 0. 78 tL.I. and T.A. < N.O. 2.72 0.91 None 1. 72 0.66 None 1.96 0.93 tL.I. < N.O. 2.18 0.91 None 2.33 1.01 tL.I. > T.A. and N.O. 2.02 0.61 None 2.22 0. 76 tL.I. < N.O. 2.52 0.81 None *Response scale: 1 = strongly agree; 2 = agree; 3 = disagree; 4 = strongly disagree. tp < 0.05. confidence in the ability to locate materials in the main library increased as confidence in the ability to locate materials in the Undergraduate Library rose (L.I. r = .48; T.A. r = . 71; N.O. r = .34). Interestingly, c'onfidence in the ability to locate materials in the main library increased in proportion to reported mean use of the main card cata- log for library-instructed students, but de- creased for the other two groups (L.I. r = .48; T.A. r = - .47; N.O. r = - .47). Over- all, students who received instruction from library staff demonstrated a higher mean re- sponse concerning their confidence using the university libraries than did unin- structed students. A higher percentage of li- brary-instructed students and students who received instruction from their teaching assistant reported feeling confident of their ability to use the university libraries than did uninstructed students (L.I. = 83.9 percent; T.A. = 82.2 percent; N.O. = 63.1 percent). Students who received instruction from the Undergraduate Library reported a great- er mean response concerning their comfort asking for assistance in the Undergraduate Library than did students who received no instruction. A higher percentage of the li- brary-instructed students reported feeling comfortable asking for assistance in the Undergraduate Library (L.I. = 93.4 percent; N.O. = 76.1 percent). Overall, confidence in the ability to use university li- braries increased in proportion to the re- ported feeling of comfort asking for assis- tance in the Undergraduate Library for li- brary-instructed students (L.I. r = .44). Finally, students who received no in- struction and students who received instruc- tion from their teaching assistant reported feeling that "using the libraries is a frustrat- ing experience" more often than did stu- dents who received instruction from the li- brary staff. The percentage of library-in- structed students reporting a feeling of frus- tration using the libraries was much lower than for the other two groups (L. I. = 35.8 percent; T.A. = 60.7 percent; N.O. = 58.7 percent). This finding was particularly re- vealing since there was no significant differ- ence in the responses of students in the three groups concerning how difficult it is to use the libraries. Effects of Library Instruction I 39 Discussion The objectives of the instruction program of the Undergraduate Library at the Uni- versity of Illinois, Urbana-Champaign, were primarily concerned with the task of intro- ducing students to those basic research methods and materials that would enable them to use a variety of sources in their re- search, evaluate materials for authority and appropriateness, and exploit the complex li- brary system confidently and competently. The findings of the evaluation are best ex- pressed within the context of these objec- tives, comparing the library-use behavior of students involved in library research. Students who received instruction from the Undergraduate Library used a greater number of periodical index and abstract ti- tles, a greater number of periodical articles, and a greater number of microform and media sources per student than did unin- structed students. Conversely, the li- brary-instructed students demonstrated less reliance upon books than did students who received no instruction. Thus, it might be concluded that students who received in- struction from the Undergraduate Library were successfully introduced to the variety of resources available to them for research and were induced to use them. The only evidence found concerning the expressed need for selectivity in the choice of mate- rials was the fact that library-instructed stu- dents relied less heavily upon books in a li- brary of such huge volume and sought out the less familiar and less accessible periodi- cal, microform, and media materials. This is perhaps supported by the finding that no significant variations were reported in the use of the main card catalog, the Under- graduate Library card catalog., library ser- vices, or departmental libraries. Students who received instruction from the rhetoric teaching assistant compared favorably with library-instructed students in the variety of sources used. The evidence for selectivity in the choice of materials among these students is perhaps less · compelling, however, since these students reported using as many books as did unin- structed students. All three groups of students appear to have used campus libraries, card catalogs (except for the serial record), and library 40 I College & Research Libraries • january 1981 services to a similar extent. On the whole, they seem to have preferred the resources and services of the Undergraduate Library and the departmental libraries over those of the main library, perhaps because these li- braries have open stacks, study facilities, and contain the bulk of current periodical and monographic literature. Library-in- structed students exhibited a significantly greater use of the Library Computer Sys- tem than students in the other two groups. Since neither of the other groups received LCS instruction, this result can probably be attributed directly to the Undergraduate Li- brary instruction effort. Students themselves were not remiss in recognizing the need for instruction. Stu- dents from all three groups expressed the nearly unanimous opinion that to use the li- brary effectively is important, and most felt that instruction in library skills would be of value to new students. A majority of stu- dents in each of the groups expressed an in- terest in receiving more instruction. Students who received instruction from the Undergraduate Library demonstrated confidence in their ability to use the librar- ies, especially the Undergraduate Library, to a greater extent than did uninstructed students. The library-instructed students also reported feeling more comfortable asking for assistance in the Undergraduate Library. Reported feelings of frustration using the libraries were lower for students who received instruction from the Under- graduate Library than in either of the other two groups. This occurred in spite of the fact that just as many of these students re- ported feeling that the libraries are difficult to use as did students who received instruc- tion from their teaching assistant and those who received no instruction. Overall, the evaluation appears to have indicated that the instruction program of the Undergraduate Library was successful in achieving its objectives. Students who re- ceived instruction from the Undergraduate Library used a greater variety of sources, seem to have exhibited a degree of selectiv- ity in their choice of materials, and appear to have attained a sense of confidence and competence in their ability to use the li- brary effectively. CONCLUSION Library user-education programs have long needed a method by which the impact of instruction on student use of the library could be determined. Evaluations based on library use offer an objective, data-based impression of the overall success or failure of an instruction program by identifying portions that have achieved or failed to achieve the desired results, and by indicat- ing any undesirable consequences of the in- struction effort. When used in conjunction with perceptual evaluation methods, such evaluations provide a means of gaining in- sight into the motivational and practical aspects of student behavior as manifested in uncontrolled library situations. The evaluation described in this paper is but one step in the development of an adequate methodology for evaluations based on library use. A notable shortcoming of the evaluation design employed here was its in- ability to reveal the order in which mate- rials were used-an extremely important concern for programs that attempt to teach search strategy. Evaluations based on library use cannot reveal all the causes of success or failure in library instruction. Achievement and per- formance testing along with perceptual eval- uation remain important tools for monitor- ing the cognitive and affective results of in- struction on a day-to-day basis. Yet to be found is a methodology that will express the role student motivation and maturation play in the development of effective library skills. Evaluations based on library use can, however, present a picture of the impact of instruction within the context of a program's behavioral objectives. REFERENCES 1. See, for example, ACRL Bibliographic In- struction Task Force, "Toward Guidelines for Bibliographic Instruction in Academic Librar- ies," College & Research Libraries News 36:137-39, 169--70 (May 1975). 2. See, for example, James Benson, "Biblio- graphic Education: A Radical Assessment," in Cerise Oberman-Soroka, ed., Proceedings from the 2nd Southeastern Conference on Approaches to Bibliographic Instruction, College of Charleston, March 22-23, 1979 (Charleston, S.C.: College of Charleston Associates, 1980). 3. An overview of goals and objectives for a variety of user-education purposes is pro- vided by Nancy Fjallbrant and Malcolm Stevenson, User Education in Libraries (Lon- don: Clive Bingley, 1978). 4. Two reviews of the efforts of that period should be noted: Thomas Kirk, "Biblio- graphic Instruction-A Review of the Re- search," in Richard J. Beeler, ed. , E valuat- ing Library Instruction (Ann Arbor, Mich.: Pierian Pr., 1975), p.1-29; and Arthur P. Young, "Research on Library-User Educa- tion: A Review Essay," in John Lubans, Jr., ed., Educating the Library User (New York: Bowker, 1974), p.1-15. 5. Sara Lou Whildin, "Planning a Library In- struction Program Based on Measurable Objectives," in Hannelore B. Rader, ed., Academic Library Instruction: Objectives, Programs, and Faculty Involvement (Ann Arbor, Mich.: Pierian Pr., 1975), p. B-21. 6. John Lubans, Jr., "Library-Use Instruction Needs from the Library Users'/Nonusers' Point of View: A Survey Report," in his Edu- cating the Library User (New York: Bowker, 1974), p.401-9. 7. John Lubans, Jr., "Evaluating Library-User Education Programs," in his Educating the Library User (New York: Bowker, 1974), p.232-53. 8. Masse Bloomfield, "Testing for Library-Use Competence," in John Lubans, Jr., ed., Edu- cating the Library User (New York: Bowker, 1974), p.221-31. 9. One of the more commonly used and adapted standardized tests has been Ethel M. Feagley and others, A Library Orientation Test for College Freshmen (New York: Teachers Col- lege Pr., Columbia Univ., 1955). 10. Several methods of performance testing are discussed in Patricia B. Knapp, The Monteith College Library Experiment (New York: Scarecrow, 1966). Refinements and variations are still in use. See, for example, Thomas Kirk, "Evaluation of Library Orientation and Instruction Programs: A Taxonomy," in Mary Bolner, ed., Planning and Developing a Li- Effects of Library Instruction I 41 brary Orientation Program (Ann Arbor, Mich.: Pierian Pr., 1975), p.41-51. 11. Young, "Research on Library-User Educa- tion," p.7. 12. Susan Burton, "Objective Tests as an Evalua- tion Tool: Problems in Construction and Use," in Hannelore B. Rader, ed., Library Instruction in the Seventies: State of the Art (Ann Arbor, Mich.: Pierian Pr., 1977), p. ~ 103. 13. Lubans, "Library-Use Instruction Needs." 14. See, e.g., John Lubans, Jr., "Evaluation Attempts of Library Use Instruction Pro- grams at the University of Colorado Librar- ies," in Richard J. Beeler, ed., Evaluating Library Instruction (Ann Arbor, Mich.: Pier- ian Pr., 1975), p.67-81; and John Lubans, Jr., "Final Report . . . to the Council on Li- brary Resources and the National Endow- ment for the Humanities ... (on the) Pro- gram to Improve and Increase Student and Faculty Involvement in Library Use," mimeographed (Boulder, Colo.: University of Colorado Libraries, 1978). 15. Benson, "Bibliographic Education." 16. See also the comments of Richard R. John- son, "Library Instruction: The Mythology of Evaluation," in Richard J. Beeler, ed., Eval- uating Library Instruction (Ann Arbor, Mich.: Pierian Pr., 1975), p.31-41. 17. Benson, "Bibliographic Education." 18. Combinations of the evaluation techniques described above, along with subjective observations of the programs, have been termed illuminative evaluation and applied to the Travelling Workshops program in Eng- land. See Colin Harris, "Illuminative Evalua- tion of User Education Programmes," Aslib Proceedings 29, no.l0:348-62 (1977). 19. Benson, "Bibliographic Education." 20. Both Kirk ("Bibliographic Instruction") and Johnson ("Library Instruction") urged the de- velopment of user study methodologies for instruction-program evaluation. 21. More detailed information concerning this study, including copies of the questionnaire employed, is available from author David King. 22. UGLi Newsletter, no.15, 1980 (published by the Undergraduate Librarians Discussion Group, Association of College and Research Libraries).