College and Research Libraries The Comparable Effects of Term Paper Counseling and Group Instruction Sessions Patricia Morris Donegan, Ralph E. Domas, and John R~ Deosdade One hundred fifty-six community college students enrolled in an introductory management course participated in a bibliographic instruction experiment to determine whether individual- ized term paper counseling sessions conveyed information for immediate recall better than group instruction sessions. The two instructional methods were designed to accomplish identi- cal learning objectives concerning basic library search strategy. Prior to the experiment, a mul- tiple choice test instrument was designed and pretested. Immediately after instruction, the students were given a twenty-one-item multiple choice test on the content of the instruction sessions. Analysis of test scores revealed no significant differences between the term paper counseling group and the lecture instruction group. n trying to teach college stu- dents how to use libraries effec- tively and efficiently, academic librarians have used a variety of instructional methods. Depending upon the resources at hand and the nature and learning needs of the students, librarians have employed teaching methods ranging from traditional group lecture tours to in- dividualized instruction. In recent years, the college student population, especially at community colleges, has become more heterogeneous with wide variations in ability, knowledge, and library skills. This heterogeneity implies that, to achieve maximum learning effectiveness, librari- ans should consider using primarily indi- vidualized instruction methods rather than group instruction. Among the possible individualized li- brary instruction methods, there is one technique that is tailored exactly to meet the student's learning need. Term paper counseling (TPC), sometimes called re- search consultations, term paper assis- tance, or tutorials, has successfully com- bined elements of pure reference work and traditional library instruction to pro- duce a new perspective on academic li- brary service. . However, this method requires contin- ual large investments of instructional time and effort from librarians. If term paper counseling is an optimally effective in- struction method, it may be worth the cost involved. Knowing this requires that we answer one question: is term paper coun- seling a more effective technique for teaching library skills than traditional group instruction sessions? CHARACTERISTICS OF TERM PAPER COUNSELING Term paper counseling provides a stu- dent the opportunity to meet individually with a librarian to learn a search strategy for identifying library resources suitable for a research topic. After the student Patricia Morris Donegan is Bibliographic Instruction Librarian, Ralph E. Domas is Public Services Coordina- tor, and John R. Deosdade is Circulation/Reserve Coordinator at San Antonio College Library, San Antonio, Texas 78284. 195 196 College & Research Libraries schedules an appointment, the librarian investigates the various sources of infor- mation available in the library's collection and prepares a pathfinder for the stu- dent's topic. A typical pathfinder will: (1) suggest specific reference books such as encyclopedias, handbooks, and bibliogra- phies; (2) list subject headings to be searched in the library catalog; and (3) identify relevant periodical indexes and suggest subject headings to be searched in these indexes. At appointment time, the student meets with the librarian, who uses the pathfinder as the basis for teach- ing an effective search strategy for the student-specified topic. After this hands- on instructional session, the student leaves with a copy of the pathfinder and is prepared to begin gathering information sources for a term paper. From the student's perspective, this ap- proach may be a valuable safety net sup- plying needed library instruction when it is not available through organized class- room experiences. Because term paper counseling sessions offer not only imme- diacy but also student-specified rele- vancy, this learner-centered technique has the potential to address readily di- verse learning abilities and styles. In addition to serving the student, term paper counseling may also be beneficial to libraries that provide this service. These benefits stem primarily from the genera- tion of the custom-designed pathfinders. Some of these benefits are that: • Librarians will improve their knowl- edge of the library's collection and thus be able to provide more in-depth refer- ence and information service; • Areas of weakness in the library's col- lection will be identified and materials may be ordered to strengthen those ar- eas; • The collection of pathfinders may be made available for independent use by students; and • There will be a reduction in the amount of labor and time devoted to prepara- tion of the pathfinders because topics are repeated and pathfinders need only be revised to add/ delete sources and change call numbers/locations as neces- sary. March 1989 An overall beneficial aspect of the term paper counseling program is that students and faculty become more aware of the li- brarian's value as an individual who can provide assistance and instruction in the use of the library and in identification and selection of materials relevant to a particu- lar information need. Furthermore, the anxiety level of the student towards use of the library may be reduced by the one-to- one interaction between the librarian and the student. PERTINENT LITERATURE Library instruction, bibliographic in- struction, and user education are synony- mous terms that refer to librarians teach- ing people how to use library resources. In a college setting, this type of instruction strives to support the library-related learn- ing needs of the students as they investi- gate information resources to complete course assignments. Depending upon the available resources and the differing na- ture and needs of the users, library in- struction programs use a variety of in- structional strategies such as lectures, tours, audiovisual presentations, pro- grammed instruction, computer-assisted instruction, individualized counseling sessions, credit courses, and point-of-use instruction. By collecting surveys from 830 libraries that contributed to the national LOEX li- brary instruction clearinghouse and by noting information requests made by an additional 800-plus libraries, Carolyn Kirkendall has gathered data on the vari- ety of instructional methods that have been employed in recent years for user ed- ucation programs. 1 She has noted that ''although each program is tailored to the needs of the individual institution, prefer- ences for particular instruction methods swell and wane as often as their effective- ness varies.''2 Comparing data from 1973 and 1979, Kirkendall has shown that cer- tain group instruction methods, such as tours, have declined in popularity while some individualized instruction methods, such as computer-assisted instruction and self-paced/programmed workbook/ exer- cises, have become more widely used. Mignon Adams has noted that in- creased use of individualized instruction methods for user education has come about because "librarians have found [large group instruction] to be unsatisfac- tory; students tend to be unmotivated, they forget important skills by the time they need them, or, in any given group of students, the level of library sophistica- tion varies widely.' '3 Within the last eight years, several de- scriptions of term paper counseling pro- grams have appeared in the professional journal literature. Phyllis Hughes and Ar- thur Flandreau described Berea College's freshman tutorial research instruction program and warned that there was no hard evidence to prove that its individual- ized approach was more effective than other instructional methods. 4 Tim Scho- bert discussed term paper counseling as provided at the University of Ottawa while Jamie Coniglio summarized Iowa State University's Term Paper Advisory Service. 5' 6 Kathleen Bergen and Barbara MacAdam presented a description of the University of Michigan Undergraduate Li- brary's TAP (term paper assistance pro- gram) and produced a study of motivating factors and user characteristics.7 Although most of these programs have employed attitudinal evaluations show- ing that students believe these services are valuable and successful, only Berea Col- lge' s program has collected data on cogni- tive change resulting from term paper counseling. Results of Berea College's lim- ited use of pretest and posttest evalua- tions showed that tutorial conferences produced a significant gain in students' understanding of library search skills. Thus, it is apparent that although many academic libraries offer term paper con- sultations to their students as a method of user education, ''there has been limited analysis of the nature of research consulta- tions for undergraduates and precisely why or if they work. " 8 DESCRIPTION OF SAMPLE The 156 participants in this study were students at San Antonio College in San Antonio, Texas, enrolled in Management 1302, the introductory practicum of the management program. San Antonio Col- Comparable Effects 197 lege is a publicly supported two-year com- munity college with an enrollment of over 22,000 students. Approximately two- thirds of the college's enrollment is in the liberal arts and sciences, while one-third is in occupational-technical programs. The college has an open admissions policy. The management program is an occupational-technical program designed to develop the fundamental skills, knowl- edge, attitudes, and experience necessary for men and women to function in decision-making positions. Combining academic instruction with practical, on- the-job training, the management pro- gram is available in both the day and eve- ning divisions. Management 1302 combines internship training and application of principles and skills under the supervision of a student's employer and a member of the manage- ment faculty. Weekly one-hour seminars, held in conjunction with on-the-job train- ing, stress job-related skills and indepen- dent research. INSTRUCTION GOALS A major course requirement of Manage- ment 1302 is the production of a research paper. This learning activity is designed to instruct the student in the techniques of researching and writing a ten-page re- search paper on a management topic. Each semester, the Management 1302 faculty has requested that the library fac- ulty present lectures on basic search strat- egy so that the students will be prepared to gather information sources for the re- search paper. Although the term ''search strategy" is defined in several ways, the teaching of this systematic approach to in- formation gathering has become popular in recent years. 9 Susan Deese has noted that ''there has been a shift from a skills orientation to a research orientation to li- brary use.'' 10 A primary goal of search strategy instruction is to teach a process of information retrieval rather than the con- tent of particular information sources. If this goal is achieved, students acquire not only general library skills but also the po- tential to produce high-quality research papers. Although the librarians established lee- 198 College & Research Libraries ture content and expected students to learn certain information, specific goals for the management library instruction sessions had not been drafted prior to this study. Recognizing that test items must measure the degree to which the student has accomplished designated learning ob- jectives, the authors developed a formal list of student learning goals. The goals specified that the learner should be able to identify the techniques required to carry out a basic library search strategy. The rec- ommended search strategy process has three steps: (1) find background informa- tion using reference materials, such as en- cyclopedias; (2) find books by using the card catalog; and (3) find periodical arti- cles by using appropriate indexes. A list of the library instruction goals is provided in appendix A . INSTRUMENT DEVELOPMENT With definite instruction goals in hand, the authors began a search for a commer- cially published test. Although Adams identified the limited number of currently available standardized library tests, none of these instruments evaluated a student's progress toward all of the instruction goals of this study .11 Noting that Larry Hardesty, Nicholas Lovrich, and James Mannon had also made this observation in a similar experiment, the authors devel- oped an original instrument to measure students' immediate recall of library search strategy information. 12 A major concern when using an original instrument is to determine the degree to which the test measures what it is in- tended to measure. Therefore, an impor- tant step in this study was to create, test, evaluate, and select test items that would constitute collectively an optimal self- developed instrument. The authors selected a multiple-choice format because it would permit grading ease and computerized data analysis. Rather than create unique items to test a student's recognition of the parts of a cata- log card, the authors decided to follow the pattern of four pertinent multiple-choice questions from the "Library Skills Test" published by the Illinois Association of March 1989 College and Research Libraries. 13 Working independently and using Norman Grunlund's rules for constructing multiple-choice items, two of the authors separately created one test question for each remaining instruction goal and thus produced thirty-six potential questions. 14 Together the authors refined the phrasing of these original items and eliminated two questions that contained information which might help students answer a pre- ceding question. The thirty-four edited test items were then divided in half. Each half contained a question relating to each instruction goal. The four questions modeled after items from the illinois Library Skills Test were not added to these seventeen-item skills tests because those questions were con- sidered valid. The authors then gave the two tests and a copy of the instruction goals to a faculty member from the management depart- ment and a librarian colleague and asked them to review the test items for verbal clarity and goal relevancy. These review- ers indicated that no significant changes were required. The authors incorporated the reviewers' few phrasing recommen- dations during a final edit prior to the in- strument test. During spring 1986, the authors tested the instruments on six classes of students enrolled in Management 1302. Immedi- ately before taking the test, these 105 stu- dents listened to a librarian give a lecture covering the designated library instruc- tion goals. To promote performance moti- vation, the librarian informed these stu- dents, prior to this instruction, that their test scores would be worth 5 percent of their final course grade. To allow equal testing of both instrument versions (C and D), two classes took Test C, two classes took Test D, and the remaining class was split so that half took Test C and half took Test D. The authors further tested the instru- ments on two classes of freshman orienta- tion students during the summer of 1986. These seventy-one students did not re- ceive library instruction immediately be- fore the test nor did they have a grade to motivate them. Table 1 presents mean score data for both test versions administered to the management and orientation classes. Construct validity, in which test perfor- mance is used to infer the possession of certain traits or qualities, was confirmed by this data because no significant score difference appeared between versions (C and D) for either the management or ori- entation groups. The data, however, did indicate a difference in knowledge be- tween the management and orientation groups. This variance in mean scores is consistent with the assumption that the instruction received by the management group should have resulted in a mean score difference between the management and orientation groups. Rather than use either test version C or D as originally presented, the authors compared the mean score for each test item from version C and D and used face validity to select the most discriminating test items from each version. This resulted in eleven items being selected from ver- sion C and six items from version D. These seventeen items were then combined with the four items from the Library Skills Test to arrive at the final instrument for the ex- periment. A copy of the test is given in ap- pendix B. PROCEDURES During the fall 1986 semester, 156 stu- dents were enrolled in ten sections of Management 1302. Six of the sections met on campus, while four sections met off campus at major San Antonio corpora- Comparable Effects 199 tions. These ten sections were randomly divided into three groups such that four sections received group instruction ses- sions (GIS), three received term paper counseling (TPC), and three received no instruction and functioned as the control group. There was a comparable number of students in the three groups. Each author worked with at least one section of each group so that there would be an equal distribution of the teaching load. The course syllabus distributed to all the students at the first class meeting stated that a library skills test would be ad- ministered and that scores on the test would be worth 5 percent of each stu- dent's final test grade. To increase the in- fluence of this grade motivation factor, the students were reminded of this fact prior to each instruction/test session. Sections assigned to group instruction met in their classrooms during a regularly scheduled class period. At the beginning of these sessions, the librarian stated the purpose of instruction, distributed a handout that summarized the lecture, and reminded the students that the test would be administered immediately after the lec- ture. Each lecture lasted approximately forty minutes and included overhead transparencies to emphasize key con- cepts. Students were allowed twenty-five minutes to complete the test after the lec- ture. It was necessary for a librarian to visit briefly with each TPC section because TPC requires the librarian to prepare in ad- vance both a pathfinder and an individu- alized counseling session for each stu- TABLE 1 Oass Management classes Orientation classes Totals COMPARISON OF TEST VERSION MEAN SCORES {X)* FOR MANAGEMENT AND ORIENTATION CLASSES Test Version Test Version C Test Version D x = 13.75 x = 14.41 N =51 N = 56 x = 10.14 x = 11.29 N =35 N =34 x = 12.28 x = 13.23 N =86 N =90 *Scores refer to responses on a seventeen-item skills test. Totals x = 14.09 N = 107 x = 10.71 N = 69 N = 176 200 College & Research Libraries dent. During these visits, the librarian described term paper counseling and its relationship to the management research paper. After informing the students that they would take the test immediately after their individual TPC sessions, the librar- ian distributed TPC application forms and explained the appointment scheduling procedures. Students were required to schedule an appointment at least four days prior to the start of the designated two-week TPC period. Permitting ap- pointments during both day and evening hours, the TPC schedule offered ninety- four potential one-hour time slots for the convenience of the forty-five students. Students in the TPC sections came to the library for their counseling sessions at their individually arranged dates and times. To provide instruction on basic li- brary search strategy, the librarian used a pathfinder that was tailored to the stu- dent's topic as well as pertinent examples from reference books, the card catalog, and periodical indexes. Depending on in- dividual learning needs, these sessions varied from thirty to fifty minutes in length with an average .session lasting thirty-five minutes. The twenty-five- minute test period concluded each term paper counseling session. March 1989 A librarian visited the control group sec- tions during a regular class session and announced that the management faculty had invited the librarians to administer the test. The test was then given without any library instruction. Immediately after the test, the librarian informed the stu- dents that they had participated in an ex- periment and that they would have the opportunity to retake the test during the next class period after instruction. This procedure allowed control group data col- lection without adversely affecting the students' course grades. ANALYSIS OF RESULTS Table 2 illustrates the mean score and standard deviation for the three treatment groups. The one-way analysis of variance (ANOV A), illustrated in table 3, yielded a significant F (F = 23.85, df = 2/153, p < .01) indicating that a difference existed across the three treatment groups. Tukey' s HSD (Honestly Significant Dif- ference) Test was used to determine where significant differences existed be- tween pairs of means. The results demon- strated that a significant difference existed between term paper counseling and the control group (q = 8.585, df = 153, k = 3, p < .01) as well as between group in- TABLE2 COMPARISON OF FINAL TEST SCORES FOR GROUP INSTRUCTION SESSIONS, TERM PAPER COUNSELING, AND CONTROL GROUP CLASSES Groul' Instruction Term Control Sessions Paper Counseling Group N= Mean Score* Standard Deviation *Scores refer to responses on a twenty-one-item skills test. TABLE 3 63 45 16.68 17.02 2.82 2.63 ANOVA SUMMARY TABLE Source Treatments Error Totals ss df 398.35 2 1,277.88 153 1,676.23 155 48 13.38 3.19 MS F 199.18 23.85 8.35 p < .01 struction sessions and the control group (q = 8.397, df = 153, k = 3, p < .01). However, no difference was found be- tween term paper counseling and group instruction sessions (q = 2.563, p > .05). OBSERVATIONS A statistical comparison of term paper counseling and group instruction test results did not indicate that either method was superior. Term paper counseling pro- duced an 81 percent mean score and group instruction yielded a mean score of 79 percent, while the control group's mean score was 63 percent. These findings show that term paper counseling and group instruction are com- parably effective techniques for teaching basic library search strategy. However, term paper counseling is clearly a more labor-intensive method. During this ex- periment, the librarians spent an average of thirty-three minutes to prepare each of the thirty-one pathfinders. On the other hand, 31 percent of the forty-five individ- ualized sessions required no preparation time because fourteen students chose top- ics previously selected by other students in the TPC group. In contrast to the nu- merous hours spent on TPC pathfinder preparation, only a couple of hours were required for the librarian to develop the group session lecture presentation. Given that the instructed subjects in ~his experi- Comparable Effects 201 ment performed comparably well on the test, it appears that group instruction was the less-expensive way to achieve the des- ignated learning goals. A potential concern of this study was the limitations of the self-developed test. Although care was taken to determine the instrument's validity, time and resource constraints prohibited extensive testing efforts using more subjects from a variety of undergraduate backgrounds. Further testing of the self-developed instrument might reveal a clearer picture of its validity and reliability. Term paper counseling is a relatively new method for teaching people how to use library resources. This study is one of the first attempts to evaluate the instruc- tional effectiveness of this method. This basic experiment focused on a comparable assessment of term paper counseling's ability to affect cognitive change. No at- tempts were made to determine how TPC influenced students' attitudes or how well students could actually execute basic search strategy after TPC instruction. These are topics for further study. Addi- tional studies are also necessary to explore the impact of potential TPC benefits, such as librarians' increased familiarity with their collections, collection development in weak subject areas, and the creation of a pathfinder collection available to indepen- dent library users. REFERENCES 1. Carolyn A. Kirkendall, "Library Use Education: Current Practices and Trends," Library Trends 29:29-37 (Summer 1980). 2. Ibid., p.30. 3. Mignon Adams, "Individualized Approach to Learning Library Skills," Library Trends 29:83 (Summer1980). . 4. Phyllis Hughes and Arthur Flandreau, "Tutorial Library Instruction: The Freshman Program at Berea College," Journal of Academic Librarianship 6:91-94 (May 1980). 5. Tim Schobert, "Term-Paper Counseling: Individualized Bibliographiclnstruction," RQ 22:146-51 (Winter 1982). 6. Jamie Wright Coniglio, "Bibliographic Counseling: The Term Paper Advisory Service," Show-Me Libraries 36:79-82 (Oct./Nov. 1984). 7. Kathleen Bergen and Barbara MacAdam, "One-To-One: Term Paper Assistance Programs," RQ 24:333-40 (Spring 1985). 8. Ibid., p.333. 202 College & Research Libraries March 1989 9. Sharon J. Rogers, "Research Strategies: Bibliographic Instruction for Undergraduates," Library Trends 29:69-70 (Summer 1980). · 10. Susan Annora Deese, "Integrating Library Use Education ·with Study Skills Instruction," in Teaching Librarians To Teach: On-The-Job Training for Bibliographic Instruction Librarians, ed. Alice S. Oark and Kay F. Jones (Metuchen, N.J.: Scarecrow, 1986), p.206. 11. MignonS. Adams, "Data Gathering Instruments," in Evaluating Bibliographic Instruction: A Hand- book, ed. Bibliographic Instruction Section, Association of College and Research Libraries (Chi- cago: American Library Assn.), p.61-78. 12. Larry Hardesty, Nicholas P. Lovrich, Jr., and James Mannon, "Evaluating Library Use Instruc- tion,'' College and Research Libraries 40:309-17 Guly 1979). 13. Illinois Association of College and Research Libraries, Library Skills Test (Bensenville, m.: Scholas- tic Testing Service, 1980). 14. Norman E. Gronlund, Constructing Achievement Tests, 2d ed. (Englewood Oiffs, N.J.: Prentice- Hall, 1977), p.39-54. APPENDIX A: MANAGEMENT LIBRARY INSTRUCTION GOALS 1. The student will identify the three steps of a basic library search strategy. 2. As an initial step in the search strategy, the student will choose an encyclopedia to find background information. A. Using his chosen topic, the student will begin his search in the encyclopedia by consulting the index. (1) Using the index, the student will: (a) identify subject headings which describe the topic; (b) identify subject heading subdivisions which may give ideas for narrowing the topic; (c) identify cross-references which may lead to related information on the topic; (d) identify page numbers for information on or related to the topic. B. The student will recognize that one function of an encyclopedia article is to give ideas for nar- rowing the topic. C. The student will recognize that bibliographies following encyclopedia articles will identify addi- tional information sources on a topic. 3. The student will recognize that the card catalog should be used to identify books in the library. A. Given the need to identify books about a topic, the student will choose to search in the subject catalog. B. Given the need to determine if the library owns a particular book title or books by a particular author, the student will choose to search in the author and title catalog. C. The student will recognize that the call number printed on the catalog card should be used to locate a book on the shelves. D. Given a sample catalog card, the student will identify selected elements on the card. 4. The student will recognize that indexes should be used to identify periodical articles for current information on a topic. A. The student will choose appropriate periodical indexes for the topic. B. Using the index, the student will identify subject headings which describe the topic. C. Given a sample index entry, the student will identify the information necessary to prepare a working bibliography card. D. The student will recognize that the periodicals holdings list should be used to identify periodi- cals owned by the library. E. Given a sample page from the periodicals holdings list, the student will determine if the library owns a particular periodical volume. APPENDIX 8: LIBRARY INSTRUCTION TEST Mark on the answer sheet the letter of the response which best completes each statement. 1. What statement best summarizes the steps of a basic library search strategy? Comparable Effects 203 a. identifying subject headings, subject heading subdivisions, and related terms. b. finding background information, books, and periodical articles. c. locating the card catalog, book shelves, and periodicals holdings list. d. selecting dictionaries, almanacs, and periodical indexes. 2. Background information needed for a management research paper can generally be found in: a. encyclopedias. b. dictionaries. c. almanacs. d. directories. 3. The best way to begin using an encyclopedia for information on a topic is to: a. look for an article. b . consult the index. c. look in the table of contents. d. read the introduction. 4. One function of an encyclopedia index is to: a. locate a list of definitions for difficult or technical words concerned with a topic. b. identify a list of additional books and periodical articles on a topic. c. identify page numbers in the encyclopedia for information on or related to a topic. d. present illustrative material (e.g., illustrations, maps, tables) concerned with a topic. For items 5 to 8, use this excerpt from the index to the Encyclopedia of Professional Management. Jencks, Christoper, 331 Jethro's advice to Moses, 78 Joan of Arc, 610 Job analysis, 178, 179, 580-581 and EEOC, 581 job description, 580 job specifications, 580 position guides, 349 (See also Job evaluation) Job analysis sheet, 179 Job assignments, leadership and, 619 Job