michel.p65 What Do They Really Think? 317 What Do They Really Think? Assessing Student and Faculty Perspectives of a Web-based Tutorial to Library Research Stephanie Michel Over the past thirty years, libraries have increasingly used forms of com­ puter-assisted instruction (CAI) in place of librarians for basic instruc­ tion tasks. This study evaluates student and faculty perceptions of the Highlander Guide, a Web-based tutorial to library research. Overall, stu­ dents (particularly those required to use it) and faculty reported positive views of the guide. Correlations drawn between student confidence in using the Web or conducting library research revealed that confident students reacted more favorably toward the Highlander Guide than av­ erage students did. In contrast to previous studies, the results of this study indicated that students and faculty were not strongly in favor of using the tutorial to replace traditional library instruction. ince development of the mod­ ern computer in the 1950s, so­ ciety has sought ways to use it in place of human labor. As early as the 1970s, although the micro­ computer had not yet been developed, this statement held true for libraries, par­ ticularly in the area of bibliographic in­ struction.1 At a time when libraries were faced with budget cuts and reduced staff­ ing, combined with growing demands for instruction, librarians began to search for alternative methods of providing in­ struction.2 In the midst of this situation, librarians recognized the potential for computers to provide interactive, indi­ vidualized instruction without the inter­ mediation of a librarian, and thus com­ puter-assisted instruction (CAI) was born.3 Definition of CAI In 1971, Alan B. Salisbury defined com­ puter-assisted instruction as “a man–ma­ chine interaction in which the teaching function is accomplished by a computer system without intervention by a human instructor. Both training material and in­ structional logic are stored in computer memory.”4 Salisbury’s definition remains the authoritative definition for CAI today. In 1985, Patricia D. Arnott and Deborah E. Richards elaborated on Salisbury’s defi­ nition by including a description of the functions performed by the program as well as some of its advantages: Stephanie Michel is the Humanities Reference Librarian at the University of Oregon; e-mail: smichel@oregon.uoregon.edu. This article is based on work the author did as a Reference/Instruction Li­ brarian at Radford University. 317 mailto:smichel@oregon.uoregon.edu 318 College & Research Libraries July 2001 The term computer-assisted instruc­ tion, or CAI, may apply to all in­ structional uses of the computer. More specifically, CAI means using the computer as an instructional tool: to present new information to the user, test a user ’s knowledge of the information previously pre­ sented, or allow her or him to dis­ cover new concepts. No instructor need be present, since the user in­ teracts directly with the computer. All necessary directions are in­ cluded within the lesson. The user is allowed to control the process by pressing keys to advance or review, all at his or her own pace.5 A variety of different programs have been used to create CAI tutorials, includ­ ing: PLATO (Programmed Logic for Au­ tomatic Teaching Operation), the first pro­ gramming package used to create CAI pro­ grams; HyperCard (for the Macintosh); E- mail; the World Wide Web; and other pro- gram-authoring software for both IBM and Macintosh computers.6–9 History of CAI Although the 1970s are recognized as the beginning of “operational” CAI programs, the theoretical framework that paved the way for CAI was established in the 1950s. At this time, the behavioral psychologist B. F. Skinner supported a novel instruc­ tional approach called “programmed in­ struction,” which was essentially the pre­ decessor of CAI. Skinner theorized that “the real focus in education should be on consistent, immediate, positive reinforce­ ment for appropriate behavior and for the attainment of delineated educational ob­ jectives.”10 He emphasized a “careful and sequential arrangement of teaching mate­ rials so that the ‘learning experience will be presented at a size or rate that the stu­ dent can handle, and so that prerequisite skills will have been mastered before more complex tasks are attempted.’ “11 Using this behaviorist approach, several CAI tutorials were established as early as the 1960s. The PLATO system was used at the University of Illinois to teach a library skills course in the 1960s, which is still cited as among the most significant and success­ ful ventures into CAI.12 The PLATO pro­ gram also was used in the biology library at the University of Illinois in 1975 to teach students to use reference and bibliography collections and at the University of Dela­ ware to teach basic library skills to fresh­ men English students.13, 14 At both univer­ sities, the PLATO tutorials ran on a main­ frame computer that was accessible from terminals across campus.15 By the early 1980s, the necessary technological advancements had occurred that would make way for improved and enduring CAI. From 1972 to 1975, computer-assisted instruction was used at the University of Denver to teach traditional library skills and online searching techniques.16 This program has been described as a “well­ established watershed” that had signifi­ cant influence on later CAI programs. Al­ though the program had ceased to exist by 1982, its founders concluded that CAI, “whether used for direct public access to information or for the more traditional teaching skills, is the wave of the future.”17 Like the University of Denver pro­ gram, by the end of the 1970s, many of the early CAI programs had ceased to exist. Citing reasons such as cost, inflex­ ibility of time-sharing on the institution’s mainframe, and lack of training or com­ puter expertise on the part of librarians, most of the first CAI programs were even­ tually abandoned.18 Although the first innovative programs were no longer in existence, they left their mark on the in­ struction world. CAI had been proven by pioneers in the field to be an effective form of providing instruction and needed only a boost in computer technology to resurge.19 By the early 1980s, the neces­ sary technological advancements had oc­ curred that would make way for im­ proved and enduring CAI. In the 1980s, the personal computer (PC) was developed: it was smaller, less expen­ http:resurge.19 http:abandoned.18 http:techniques.16 http:campus.15 http:students.13 What Do They Really Think? 319 sive, easier to program, and well suited to library use. At the same time, librarians were becoming more knowledgeable about automated systems and beginning to assume responsibility for managing their own systems, which reduced the cost of computer operation. As a result of these advances, libraries now had direct control over access and cost for the programs, clearing the way for a resurgence of com­ puter-assisted instruction in libraries.20 Why CAI? Computer-assisted instruction arose out of a situation in libraries wherein the re­ sources were no longer sufficient to meet the needs of a growing and increasingly demanding population. In 1995, budget cuts helped to spur the creation of the Gateway to Information program at Ohio State Libraries to fill a need created by shortened library hours and a reduction in bibliographic instruction staff.21 This situation is not unique to Ohio State: due to a national trend in budget reduction in libraries across the country, many librar­ ies are struggling to provide the same level or additional services with fewer resources.22 As a result, many libraries have sought out alternative forms of in­ struction, such as CAI.23 Similarly, the CAI phenomenon re­ sulted from libraries’ inability to keep up with the tremendous demand for instruc­ tion. Several authors have cited an inabil­ ity to keep up with consistently increas­ ing demands on library instruction staff as a motivating factor in the creation of their CAI tutorials.24 This is particularly true as libraries continue to incorporate a growing number of electronic products into their collections. However, each new resource brings new demands on the in­ structional librarian to teach the use of a growing number of interfaces in addition to his or her usual instruction load.25 Thus, “[a]s the demand for advanced instruc­ tion increases, librarians search for opti­ mal ways to minimize the time devoted to teaching basic library skills.”26 One means of accomplishing this is through the use of CAI tutorials. Advantages of CAI Computer-assisted instruction offers nu­ merous advantages over all other formats of instruction. Evan Ira Farber outlined several inherent advantages to using com­ puters, rather than humans, for repetitive forms of instruction: “A computer has infinite patience, no time constraints, does not take coffee breaks or fails to show up on weekends, and it can adapt to indi­ vidual needs and requests.”27 Moreover, CAI offers a more individu­ alized approach that allows each student to work at his or her own pace.28 It is con­ sistent (providing each student the same information), flexible, and designed to meet the needs of students with various skill levels, and enables students to repeat or skip sections according to their own needs.29–31 Subsequently, it allows for “equalization in the levels of achievement” so that even if some students take longer than others to complete the program, ev­ ery student should end up with a “roughly equivalent knowledge of the topic.”32 Because CAI does not require the di­ rect intervention of a librarian to initiate, it has the potential to reach a greater num­ ber of students per semester.33 Mean­ while, it can be time-saving for librarians, freeing them to spend less time doing basic instruction and to use it elsewhere (for example, to help students with indi­ vidual questions) and avoid burnout.34 Further, CAI allows librarians to provide a greater range of instruction, from basic skills to complex topics; and may use a variety of approaches, including humor, to convey the message.35 Finally, it also may alleviate routine questions.36 CAI programs are readily available, so students can use them according to their own schedule, whenever they are moti­ vated to learn.37 CAI also allows for greater interactivity between students and the computer, and is useful for providing hands-on simulations of online searching techniques.38 For foreign students, a CAI program may seem more approachable than a reference desk.39 As a result, CAI may enable libraries to reach segments of the user population who neither sign up http:techniques.38 http:learn.37 http:questions.36 http:message.35 http:burnout.34 http:semester.33 http:tutorials.24 http:resources.22 http:staff.21 http:libraries.20 320 College & Research Libraries July 2001 for classes nor have the opportunity to participate in course-related instruction.40 Moreover, CAI methods have been proposed as a means of reaching a tradi­ tionally underserved population of li­ brary users—remote users. Janice Simmons-Welburn stated that “[w]e must acknowledge that many of our users would prefer not to leave their worksta­ tions, offices, labs, dorm rooms, or homes to do a certain amount of information seeking.”41 Providing instruction to con­ tinuing education, distance education, and nontraditional students has continu­ ally posed a challenge to librarians. How­ ever, by using CAI programs, it is pos­ sible to deliver instruction to remote sites and to offer instruction during evening and weekend hours so as to more effec­ tively reach these students at their con­ venience and point-of-need.42 Moreover, CAI programs are easy to update, particularly in Web format. They can offer automatic data collection so that right and wrong answers can be collected and the feedback immediately provided to users.43, 44 In addition, CAI may assist in the development of computer literacy by reducing the anxiety of working with microcomputers.45 Further, CAI may ap­ peal to the current generation of students who already may be familiar with com­ puters and help boost the library’s im­ age.46 Finally, students often show a more positive attitude toward CAI techniques.47 CAI programs on the Web offer sev­ eral unique advantages. First, the Web is continuously available to anyone, any­ where, with a computer (Macintosh or IBM), a Web browser, and a network con­ nection, enabling libraries to “extend ser­ vices beyond the reference desk and the classroom and reach an audience not lim­ ited by physical proximity.”48 It also al­ lows the CAI designer to incorporate multimedia materials such as images, sound, or video into the tutorial to enliven the content. In addition, Internet access and Web browsers are free, cheap, or al­ ready available to most members of an academic community.49 Hypertext docu­ ments are easy to create and update, and links may be easily provided to point us­ ers toward relevant outside material.50 Finally, the associative nature of the Web allows students to create “individual pathways to problem solving” by inter­ acting directly with the computer to be­ come independent learners.51 Thus, the Web may foster critical thinking and of­ fer a more challenging and varied learn­ ing experience for the student.52 Disadvantages of CAI Although computer-assisted instruction techniques offer numerous advantages over other formats of library instruction, these methods are not infallible. CAI pro­ grams have been criticized for several rea­ sons. First, they are expensive, requiring a significant investment in costly software and hardware in addition to the cost of the librarian or programmer’s time.53 Second, they require substantial preparation time, generally one hundred to two hundred hours for each hour of CAI.54 Further, CAI, particularly in non-Web formats, is limited to the number of terminals or computers available with access to that program.55 In addition, CAI may reduce personal contact between student and librarian, which lessens the opportunity to receive feedback, ask questions, or develop a rela­ tionship.56 This method of instruction also may exclude students who are unfamiliar or uncomfortable with using computers, as well as anyone who requires special at­ tention.57 Furthermore, most people read 20 to 30 percent slower and less accurately on a computer screen than from a printed page, which reduces the program’s effec­ tiveness. Prolonged reading from a com­ puter monitor also may result in eye­ strain.58 And depending on the format, CAI may not be easily accessible for review. One instructor warns his students: “[y]ou can’t mark this program with a yellow highlighter or study it over lunch.”59 How­ ever, the ability to print out information may help alleviate this problem. Use of the Web for CAI can pose some unique problems. The Web is often slow, particularly during daytime hours when the program is most likely to be accessed. http:tention.57 http:tionship.56 http:program.55 http:student.52 http:learners.51 http:material.50 http:community.49 http:techniques.47 http:microcomputers.45 http:users.43 http:point-of-need.42 http:instruction.40 What Do They Really Think? 321 Although the Web has the capability to include graphics, sound, and video clips, not all users have the necessary hardware or plug-ins to access these files. Lastly, it is much easier to leave a Web tutorial sim­ ply by pressing one button and thus end­ ing the instruction, whereas other forms of CAI might require more thought and deliberate action to exit the program.60 CAI versus Traditional Instruction Several studies have been conducted to compare the effectiveness of computer-as­ sisted instruction to more traditional in­ structional methods. In 1989, Central Mis­ souri State University evaluated the effectiveness of a library tour written in the Utah PILOT (Programmed, Inquiry, Learn­ ing, or Teaching) language, as compared to a traditional library tour.61 At Western Michigan University, a Hypercard CAI program was compared to traditional workbook instruction.62 Librarians at UCLA’s Louise M. Darling Biomedical Li­ brary compared a lecture-type presentation to a CAI module for instruction to three hundred undergraduate biology stu­ dents.63 Lastly, at the University of Albany, librarians compared a Web-based interac­ tive tutorial to traditional in-class instruc­ tion, which involved a combination of lec­ ture and hands-on activities.64 In every study, CAI was found to be as effective or more effective than the more traditional forms of instruction.65, 66 Further, students preferred the CAI method of instruction, noting that it was easy to use, interactive, and self-paced.67 In every case, based on the results of the survey, the universities decided to replace traditional instruction with CAI. Description of the Environment Radford University is a state-sponsored university of about 10,000 undergraduate and graduate students located in south­ western Virginia. In 1997, the library staff developed a brief library tutorial outlin­ ing basic library services and resources. However, in 1998, the instruction librar­ ians perceived a need to expand on this preliminary tutorial in order to include more information on how to conduct re­ search, including finding, evaluating, and citing information. Two instruction librar­ ians and the technology manager pre­ pared a grant proposal and were awarded a Radford University grant that provided the necessary software and release time to work on the project. During the summer of 1998, the pre­ liminary tutorial was replaced with an ex­ panded and enhanced tutorial modeled after James Madison University’s tutorial entitled Go for the Gold. The tutorial con­ tains seven sections: Orientation to McConnell Library, Develop a Research Strategy, Finding Information Resources, Searching Electronic Databases, Using Internet Resources, Evaluating Informa­ tion Sources, and Giving Credit: Citing Sources. The developers intended to have a quiz at the end of each section to test the user’s knowledge of the subject mat­ ter; due to time constraints, at this time, only two quizzes are available. The graph­ ics used in the tutorial reflect a Scottish theme, and the tutorial is named the Highlander Guide (after the Radford University mascot, the Highlander). In the planning stages, the Highlander Guide was intended for use by any un­ dergraduate student, graduate student, or faculty member, although it was espe­ cially designed for use with University 100 (an introduction to the university course taken by most freshmen), English 102 (a basic freshman writing course), and for off-campus students at the university’s two extended campus locations. The tu­ torial can be found online at . Methodology In spring of 1999, the instruction librarians wished to evaluate student and faculty perceptions of the Highlander Guide. A student questionnaire containing thirty- one questions was devised, based on ex­ amples of survey forms printed in Evalu­ ating Library Instruction: Sample Questions, Forms, and Strategies for Practical Use.68 The survey included questions about the student’s computer experience, comfort http:self-paced.67 http:instruction.65 http:activities.64 http:dents.63 http:instruction.62 http:program.60 322 College & Research Libraries July 2001 level using computers, prior library use, prior library instruction, and comfort level using the library to find information. In addition, students were asked how often they had used the Highlander Guide and were presented with twelve questions us­ ing a Likert scale, ranging from “strongly agree” to “strongly disagree,” which asked them to rate the Highlander Guide based on organization, ease of use, assistance in their research, and overall experience. Fi­ nally, they were given the opportunity to answer three short-answer questions about the most and least helpful aspects of the Highlander Guide and how it could be improved. After development, the sur­ vey was pretested on students who worked in the library’s computer lab for accuracy and clarity. A separate faculty questionnaire also was devised, based on the student ques­ tionnaire, and examples taken from Evalu­ ating Library Instruction. Faculty were asked about prior experience with library instruction, perceptions of their students’ research skills, prior use of the Highlander Guide, and attitudes toward the guide. Like the questions in the student question­ naire, questions in the faculty question­ naire were based on a Likert scale, with responses ranging from “strongly agree” to “strongly disagree.” Faculty also were given four short-answer questions that asked them to state the most and least help­ ful aspects of the guide, how it could be improved to assist in classroom instruc­ tion, and how it could be modified to be more relevant to their needs. The audience for this study was com­ posed of English 102 students and faculty. During library instruction sessions earlier that semester, many sections of English 102 had been introduced to the High­ lander Guide as a source for further in­ formation about library research. Further, one section of English 102 had been re­ quired to use part of the Highlander Guide as an assignment in their class. The instruction librarians identified English 102 courses where it could be positively determined that the Highlander Guide had been presented during library in­ struction, and surveys were sent to those faculty members in mid-March. A total of six faculty members, representing twelve sections of English 102, received surveys. Results A total of 145 surveys were returned—141 student surveys and four faculty surveys. The preliminary questions asking for in­ formation about student computer usage produced some interesting results. Of the students surveyed, 78.7 percent reported owning a PC and 97.9 percent reported using a computer daily or weekly in the past six months. However, only 46 percent of those students stated their confidence in using a computer to be very good or excellent. The primary purposes for using a computer were word processing (88.6%), surfing the Web (88.6%), or class assign­ ments (not including word processing) (75.9%). Although 85.8 percent of students reported using the Web daily or weekly, only 46.8 percent rated their confidence in using it as very good or excellent. Further, students were not frequent li­ brary users. Only 31.2 percent reported using the library daily or weekly, com­ pared to 66.7 percent who used it occa­ sionally. Overwhelmingly, students re­ ported using the library to obtain infor­ mation for research and/or class assign­ ments (93.6%), followed by studying for courses (54.6%), and using reserve mate­ rials (44.7%). The majority of students (83.6%) had asked at least one question at the reference desk, and most students were not confident of their ability to find a list of resources on a topic of their choice in the library. Only 7.88 percent rated their ability to find information as very good or excellent, whereas the majority (53.1%) rated their ability as average. On questions related to their use of the Highlander Guide, forty-seven students (33.3%) reported having used it three or more times and thirty-four (24.1%) had never used it. Of those students who had used it, thirty-two (22.6%) had used it for thirty minutes or more and fifty-nine (41.8%) had used it for ten to thirty min­ utes. What Do They Really Think? 323 As table 1 shows, student reactions to all experience with the Highlander Guide the Highlander Guide were mostly posi- as successful, and sixty-seven (56.3%) tive. The majority of students surveyed agreed or strongly agreed that they would agreed or strongly agreed that the guide recommend it to other students. was easy to use, was clearly organized, and In the short-answer questions, stu­ assisted them in finding books or maga- dents varied in their opinions of the most zines in the library. Just over half (56.1%) helpful aspect of the Highlander Guide. the students reported learning valuable in- Many students found the description of formation from the guide, and 50.4 per- searching for books or periodicals most cent preferred using it to attending a for- helpful; others appreciated the informa­ mal library instruction session. Seventy- tion on interlibrary loan (ILL). Several stu­ four students (62.2%) described their over- dents remarked that the guide was easy to use and enjoyed the ability to use it from their own computer. One student cited “the quickness and simplicity of using it,” and another stated that “the Highlander Guide provides a fair amount of information to a student without be­ ing consumed by too much information.” For the least helpful aspect, some stu­ dents listed “confusing” or “frustrating,” or the inclusion of unnecessary informa­ tion. One student said the tutorial pro­ vided “too much information not exactly relating to topic.” In answer to the ques­ tion What would make the Highlander Guide more relevant to your needs? stu­ dents requested that it be made clearer or less confusing. One student said that there were too many ways to get to too many things; another wanted more updating. Two students wished it would “do my work for me.” Faculty responses to the Highlander Guide were overwhelmingly positive. In the “Perceptions of Students’ Skills” sec­ tion of the questionnaire, three out of four faculty members strongly agreed with the statement that “incoming freshmen do not have the necessary skills to use an academic library.” All four faculty mem­ bers strongly agreed that it is important for students to know how to use a library. Three faculty members had used the Highlander Guide at least once, with two of them using it for between ten and thirty minutes and one using it for between thirty and sixty minutes. Faculty also demonstrated positive opinions of the Highlander Guide, as seen in table 2. All the faculty members agreed or strongly agreed that it was easy to use � < 324 College & Research Libraries July 2001 and clearly organized. Three faculty mem- percent for the other students surveyed. bers felt that it provided a valuable learn- For total time spent using the guide, 46.7 ing experience for their students; one was percent of students in this section had undecided. Only one faculty member pre- used it for thirty or more minutes, com­ ferred students to use the Highlander pared to 22.6 percent overall. Guide to a formal instruction session; the Students in this section of English 102, other three disagreed. In assessing their who had had more frequent exposure to overall experience, two faculty members the Highlander Guide, gave it much were undecided and two agreed or higher results, as seen in tables 3 and 4. Of strongly agreed that their experience was these students, 80 percent agreed or successful. Similarly, two faculty members strongly agreed that it was easy to use, were undecided and one agreed that their compared to 63.6 percent overall (table 4). students’ overall experience was suc­ cessful. After using the Highlander Guide, two faculty members agreed that students seemed more knowl­ edgeable about how to use the li­ brary. All four faculty members agreed or strongly agreed that they would recommend the guide to other faculty members. For the short-answer questions, faculty reported the most helpful as­ pect of the Highlander Guide to be the maps of the library, keyword searches, citation information, and information on evaluating sources. Responses to the least helpful aspect included too much use of jargon and the quizzes that have not yet been posted. One faculty member sug­ gested adding a sample student pa­ per to the guide to model documen­ tation. Other comments included “thanks for creating and evaluating this great tool” and a faculty mem­ ber who had never used the High­ lander Guide stated “now that I am aware of the Highlander Guide, I plan to make use of it in future classes I teach at RU, especially English 102.” Fifteen surveys were received from the English 102 class that had been required to use the Highlander Guide for an assignment. The re­ sponses from this “required” group provided an interesting contrast to students in the other sections of En­ glish 102. In this class, 53.3 percent of students had used the High­ lander Guide three or more times, E- �. � N 0" ' 0 < . : . " � 0 � " I . compared to the overall total of 33.3 What Do They Really Think? 325 TABLE 3 Perceptions of Students Required to Use the Highlander Guide (n = 15) Attitudes Toward the Strongly Agree Undecided Disagree Highlander Guide Agree Easy to use Organization of information was clear Helped to find books and magazines in the library I learned valuable information Prefer to formal instruction session Overall experience was successful Would recommend to other students 4 3 3 1 1 2 3 8 11 7 10 6 11 8 3 1 4 4 7 2 4 1 1 *The option of "strongly disagree" received no responses and has been omitted from this table. With regard to clarity of information, 93.3 percent of students in this section agreed or strongly agreed that the guide was clear, compared to 61.7 percent overall. More­ over, the guide helped 66.7 percent of these students to find books and magazines in the library, compared to 59.2 percent over­ all. Further, 73.3 percent of students re­ ported they learned valuable information from it, compared to 56.2 percent overall. In rating overall experience with the High­ lander Guide, 86.7 percent of students in this section agreed or strongly agreed that it was successful, compared to 62.2 percent overall; and 73.3 percent of these students agreed or strongly agreed that they would recommend it to other students, compared to 56.3 percent overall. The only question to which student responses from this sec­ tion mirrored the overall responses was whether students preferred using the Highlander Guide to attending a formal library instruction session: 46.7 percent of students in the “required” section agreed or strongly agreed with this statement, compared to an overall response of 50.4 percent. Student responses to short-answer questions in this in this section demon­ strated that they found the information on ILL, finding periodicals, and “the in­ structional ideas about research” most helpful. The least helpful aspect was the “thick” writing. Suggestions for making the guide more useful to students in­ cluded “continue current progress” and “make it more clear.” In addition, correlations were drawn between students’ perceptions of the Highlander Guide and their confidence in using the Web, or in their self-perceived ability to find resources on a topic of their choice in the library, as shown in table 5. Overall, the fifty-five students who re­ ported their confidence in using the Web For almost every response, students in the “required” group had a significantly more favorable reaction to the Highlander Guide than the other students did. as very good/excellent gave the guide higher ratings. Sixty-nine percent of these students agreed or strongly agreed that the guide was easy to use, compared to an overall score of 63.6 percent; 63.6 per­ cent of these students agreed or strongly agreed that it helped them find books and magazines, compared to 59.2 percent overall; and 54.5 percent preferred it to library instruction, compared to 50.4 per­ cent overall. Lastly, 69 percent of the stu­ dents in this category agreed or strongly agreed that their overall experience was successful, compared to 62.2 percent over­ all. By comparison, of the sixty-five stu­ dents who rated their Web confidence as poor/fair or average, 61.5 percent agreed or strongly agreed that the Highlander 326 College & Research Libraries July 2001 U e N e e e e o o e e o e e e e �oe N oeN e e � e N e o e e � e o o e e e o NeN o e e e � e oe N 75.7 percent agreed or strongly agreed that it was easy to use, com­ pared to 63.6 percent overall; 64.9 percent said it helped them find books or magazines, compared to 59.2 percent overall; and 59.5 per­ cent preferred using it to tradi­ tional instruction, compared to 50.4 percent overall. In terms of overall experience, 78.4 percent of these students agreed or strongly agreed that their overall experience was successful, compared to 62.2 per­ cent of all students surveyed. By comparison, of the eighty-four stu­ dents who rated their ability to find resources in the library as poor/fair or average, 58.3 percent agreed or strongly agreed the Highlander Guide was easy to use; 56 percent reported that it helped them find books or magazines; 46.4 percent preferred it to traditional instruc­ tion; and 54.8 percent reported their overall experience as success­ ful. Discussion Perhaps the most surprising find­ ing of this study was the underwhelming majority of stu­ dents who preferred to use the Highlander Guide rather than at­ tend a traditional library instruc­ tion session. In the literature, the results usually showed students definitely preferring an online tu­ torial or other self-paced format to traditional library instruction.69, 70 However, this study showed only 50.4 percent of students and 25 per- Guide was easy to use; 55.4 percent said cent of faculty preferring or strongly pre- it helped them find books or magazines; ferring the Highlander Guide to traditional 49.2 percent preferred it to library instruc­ instruction. Both students and faculty tion; and 55.4 percent reported an overall seemed to be ambivalent and not strongly successful experience with it. in favor of this substitution. Based on their self-reported ability to For faculty, the answer may lie in the find resources on a topic of their choice in Highlander Guide’s content. The instruc­ the library, students who reported them­ tion librarians usually tailor an instruc­ selves as very good/excellent also gave the tion session to the content of the course, Highlander Guide higher ratings, as seen which ensures that the sources and search in table 6. Of these thirty-seven students, examples demonstrated are relevant to http:instruction.69 e What Do They Really Think? 327 and it was assumed that they would be receptive to learning new information us­ ing this medium. In the author’s experi­ ence teaching traditional library instruc­ tion sessions, students are not always open to receiving information in a traditional instruction forum. Evidently, they either enjoyed the traditional methods more than was readily apparent during the instruc­ tion session or were not accustomed to the format of the online tutorial. Perhaps they were uncomfortable with learning that particular content from the computer or the Highlander Guide’s current organiza­ tion and wording were not conducive to their learning. Or, possibly, like faculty, students prefer the more specialized, in­ dividualized nature of an instruction ses­ sion organized and presented to their in­ dividual class, rather than the more gen­ eral nature of an online tutorial. Some of the correlations discussed in this article also provided an interesting insight into student perspectives of the Highlander Guide. In particular, students who had been required to use the guide had a significantly more positive reaction to it than did the total of all the English 102 sections surveyed. Students in the “re­ quired” group used the guide more fre­ quently and for a longer period of time than the other English 102 students did and thus had a greater period of time in which to form their opinions. Moreover, it is possible that because their instructor required them to use the Highlander Guide, these students were influenced by the faculty endorsement and thus per­ ceived the guide to be more important or useful to their research. For almost every response, students in the “required” group the students in that course. However, the had a significantly more favorable reaction Highlander Guide is devised to be used to the Highlander Guide than the other by anyone, so its content is more general students did. The only exception was the and may or may not meet a particular question about preferring it to traditional course’s objectives. library instruction. In this case, students However, it is not as easy to explain the in the “required” group responded less students’ response. It was hypothesized positively than the other students did, but that students would prefer the more inter- the difference was not significant. That active, self-paced nature of the online tu- answer is particularly telling because these torial. For the most part, undergraduates students had more opportunity to inter- are familiar with computers and the Web, act with the Highlander Guide, but only M0 M M M M M 0 M M 0 M0 M M 328 College & Research Libraries July 2001 N o NN N N NN N N N using the Highlander Guide, suggesting that unfamiliarity with Web format or with using a computer did not influence their opinion of the tutorial. Conversely, stu­ dents who did not report themselves as confident users of the Web answered slightly less positively than average on all points of using the Highlander Guide. It could be inferred that part of these stu­ dents’ reactions to the Highlander Guide could be related to lack of familiarity with a Web browser or conventions of using a computer and navigating on the Internet. Further, the correlation between stu­ dent confidence in using the library to find information and their opinions of the Highlander Guide produced interesting results. Students who felt confident in using the library responded significantly more positively to all Highlander Guide questions than average, whereas students who indicated a lower level of confidence in using the library were less positive about their Highlander Guide experience. This suggests that students already famil­ iar with the material presented in the Highlander Guide may have found the guide easier to use because the topics may have been a review, rather than new in­ formation. This also may reinforce some student comments that the Highlander Guide included too much jargon, which might be familiar to students accustomed to library research, but less evident to those just learning how to use the library. The results of the faculty survey re­ flected expectations that faculty on the whole would be positive about using the Highlander Guide. The most remarkable result was the response to the question about use of the Highlander Guide as slightly less than half preferred the tuto- opposed to traditional library instruction, rial to traditional library instruction. where the faculty were strongly against The other correlations demonstrate how this substitution. As mentioned previ­ a student’s prior experience could influ- ously, this may be due to the more per­ ence his or her opinion of the Highlander sonalized, specialized nature of course- Guide. The students’ statement of confi- related instruction sessions. It also is pos­ dence in using the Web may have influ- sible that this response stems from fac­ enced their opinions of the guide. Students ulty unfamiliarity with using computers who reported a strong degree of confi- or the Web, which was not measured on dence in using the Web responded more the faculty survey, or hesitance in adopt- positively than average in all aspects of ing new methods of instruction. What Do They Really Think? 329 It should be noted that several circum­ stances may have influenced the results of this survey. First, not all students an­ swered every question. For each question on the student’s perception of the High­ lander Guide, at least twenty students did not respond. Of those students who did respond, many who reported in their comments that they had not used the Highlander Guide answered “unde­ cided” to every question, which influ­ enced the results. Further, some students misinterpreted the Likert scale used on the questionnaire. For the questions about the Highlander Guide, students were in­ structed to circle a number between 1 and 5, with 1 meaning “strongly agree” and 5 “strongly disagree.” Some students who consistently circled numbers representing disagree or strongly disagree listed posi­ tive comments about the Highlander Guide, leading to the belief that these stu­ dents had intended to agree or strongly agree with these statements. Finally, some students did not seem to know what they were evaluating. Many commented that they either had not used the Highlander Guide or did not know what it was. Others rated it, but their com­ ments reflected that they believed they also were rating periodical databases, the library catalog, or the library’s Web site. Conclusions and Suggestions for Future Research There are several methods by which this survey could be improved in its next it­ eration. First, it might be sent out earlier in the semester, closer to the time of the initial instruction. This survey was sent out and completed near the end of the semester, when some of the content of that instruction session may have been forgot­ ten. Delivering the survey earlier in the semester would poll students while the instruction was fresh in their minds and perhaps would help to remind them that the tutorial was available for their use. The results of this survey also suggest that this online tutorial should be pro­ moted more actively. Even though the Highlander Guide had been mentioned during an instruction session, many stu­ dents were unaware of its existence or purpose. This suggests that more atten­ tion should be given to the guide during the instruction session to help students remember that it is available. Further, the Highlander Guide should be mentioned more prominently on the library’s Web page so that students can more easily find this resource. Since this survey was con­ ducted, the library has completed a ma­ jor revision to its Web page. Previously, the link on the library’s Web page to the tutorial read only “Highlander Guide,” which many students did not recognize as an online tutorial. In the revision, the link reads “Highlander Guide to the Li­ brary” and a pop-up menu in the center of the screen provides the following de­ scription: “An online tutorial to library research including: finding information, searching databases, evaluating re­ sources, citing sources, and more …” The survey indicates that students are mostly positive about using the High­ lander Guide, although they are ambiva­ lent about using it in place of traditional library instruction. In particular, students required to use the tutorial as a part of their course were very favorable, suggest­ ing that it was a positive addition to this course. In addition, students who were very confident in their knowledge of the Web or in using library resources also expressed very positive views of the Highlander Guide, suggesting that it may be especially suited to these populations. Based on the results of this survey, fur­ ther revisions should be made to the High­ lander Guide. Student comments suggest that the wording or navigation used in the tutorial may be confusing; perhaps these issues can be resolved in future revisions. Further, during the next revision, it would be a good idea to conduct usability testing of the Highlander Guide with a group of students so as to get immediate feedback about their perceptions and concerns. A follow-up survey of the Highlander Guide will likely be conducted in the next few years to see how perceptions and at­ titudes toward it have changed over time. 330 College & Research Libraries July 2001 Moreover, a search of the literature re­ vealed that there are relatively few cur­ rent articles assessing the effectiveness or perceptions of current Web-based online tutorials, which suggests that this is an­ other area in need of further research. Fi­ nally, more research is needed to evalu­ ate students’ preference for online tutori­ als as compared to traditional instruction because the results of this survey seem to differ from results previously reported in the literature. Ultimately, this survey suggests that an online tutorial can be an effective supplement to, and possibly a replace­ ment for, traditional library instruction. When designed well and adequately promoted in instruction sessions and through the library’s Web page, an online tutorial can assist students or fac­ ulty at any time of day or night, at their own pace, to focus on learning a specific skill or developing an overall knowledge of library research. Notes 1. Mitsuko Williams and Elisabeth B. Davis, “Computer-Assisted Instruction: An Overview,” in Theories of Bibliographic Education: Designs for Teaching, ed. Cerise Oberman and Katina Strauch (New York: R. R. Bowker, 1982), 171–91. 2. Neosha Mackey, Charlotte Dugan, Willa Garrett, and C. Lynne Freeman, “Teaching with HyperCard in Place of a Textbook,” Computers in Libraries 12 (Oct. 1992): 22–26; Fred Roecker, “Successful Research Using the Gateway to Information: Meeting the Challenge of User Inde­ pendence,” in The Upside of Downsizing: Using Library Instruction to Cope, ed. Cheryl LaGuardia, Stella Bentley, and Janet Martorana (New York: Neal-Schuman, 1995), 191–202. 3. Judith M. Pask, “Computer-Assisted Instruction for Basic Library Skills,” Library Software Review 7 (Jan./Feb. 1988): 6–11. 4. Alan B. Salisbury, “An Overview of CAI,” Educational Technology 11 (Oct. 1971): 48. 5. Patricia D. Arnott and Deborah E. Richards, “Using the IBM Personal Computer for Li­ brary Instruction,” Reference Services Review 13 (spring 1985): 69–72. 6. Williams and Davis, “Evaluation of PLATO Library Instructional Lessons,” Journal of Aca­ demic Librarianship 5 (Mar. 1979): 14–19. 7. Gwynneth T. Heaton, Sophia Kazuba, and Dan D’Agostino, “A HyperCard Program to Assist Users at a Self-help Periodicals Information Center,” Technical Services Quarterly 11, no. 2 (1993): 55–70; Eric T. Rumsey, “HyperCard for Bibliographic Instruction: Teaching IBM Com­ puter Use, Plus Much More,” Computers in Libraries 12 (June 1992): 43–45. 8. Ann Jensen and Julie Sih, “Using E-mail and the Internet to Teach Users at their Desk­ tops,” Online 19 (Sept./Oct. 1995): 82–86. 9. Carol Anne Germain, Trudi E. Jacobson, and Sue A. Kaczor, “A Comparison of the Effec­ tiveness of Presentation Formats for Instruction: Teaching First-year Students,” College & Re­ search Libraries 61 (Jan. 2000): 65–72. 10. Williams and Davis, “Computer-Assisted Instruction,” 173. 11. Ibid. 12. Ibid. 13. Williams and Davis, “Evaluation of PLATO.” 14. Arnott and Richards, “Using the IBM Personal Computer for Library Instruction.” 15. Ibid. 16. Pask, “Computer-Assisted Instruction for Basic Library Skills.” 17. Williams and Davis, “Computer-Assisted Instruction,” 180–81. 18. Pask, “Computer-Assisted Instruction for Basic Library Skills.” 19. Williams and Davis, “Computer-Assisted Instruction,” 171–91. 20. Pask, “Computer-Assisted Instruction for Basic Library Skills.” 21. Roecker, “Successful Research Using the Gateway to Information.” 22. Otis Chadley and Jacquelyn Gavryck, “Bibliographic Instruction Trends in Research Li­ braries, Research Strategies 7 (summer 1989): 106–13. 23. Dorothy F. Davis, “A Comparison of Bibliographic Instruction Methods on CD-ROM Da­ tabases,” Research Strategies 11 (summer 1993): 156–63; Laura Manzari and Ellen McCartney, “Li­ brarian Attitudes toward the Use of a Self-paced BI Program,” Research Strategies 12 (winter 1994): 33–44; Marjorie Melick, “Changing to a Hypertext-based Library Instruction Program Us­ ing Inexpensive Software for IBM Compatibles,” Computers in Libraries 14 (Nov./Dec. 1994): 35– 38; Marsha Miller, “Self-paced Tours in the Electronic Library,” Research Strategies 13 (fall 1995): 219–34; Pask, “Computer-Assisted Instruction for Basic Library Skills”; Patricia Fravel Vander What Do They Really Think? 331 Meer and Galen E. Rike, “Multimedia: Meeting the Demand for User Education with a Self- instruction Tutorial,” Research Strategies 14 (summer 1996): 145–58. 24. Lana S. Dixon, Marie Garrett, Rita Hoyt Smith, and Alan Wallace, “Building Library Skills: Computer-Assisted Instruction for Undergraduates,” Research Strategies 13 (fall 1995): 196–208; Germain, Jacobson, and Kaczor, “Comparison of the Effectiveness of Presentation Formats for Instruction”; Bruce A. Leach, “Computer-based CD-ROM Tutorials—Providing Effective On- demand Instruction,” CD-ROM Professional 6 (July 1993): 113–17; Mackey, Dugan, Garrett, and Freeman, “Teaching with HyperCard in Place of a Textbook.” 25. Linda Creanor, Helen Durndell, and Carol Primrose, “Library and Study Skills Using Hypertext: The TILT Experience,” New Review of Hypermedia and Multimedia 2 (1996): 121–47; Vander Meer and Rike, “Multimedia.” 26. Vander Meer and Rike, “Multimedia,” 146. 27. Evan Ira Farber, “Plus Ça Change…,” Library Trends 44 (fall 1995): 435. 28. Pask, “Computer-Assisted Instruction for Basic Library Skills.” 29. Joan Kaplowitz, and Janice Contini, “Computer-Assisted Instruction: Is It an Option for Bibliographic Instruction in Large Undergraduate Survey Classes?” College & Research Libraries 59 (Jan. 1998): 19–27; Pask, “Computer-Assisted Instruction for Basic Library Skills”; Harold B. Shill, “Bibliographic Instruction: Planning for the Electronic Environment,” College & Research Libraries 48 (Sept. 1987): 433–53. 30. Creanor, Durndell, and Primrose, “Library and Study Skills Using Hypertext”; Pask, “Com­ puter-Assisted Instruction for Basic Library Skills”; Ann Turner, “Computer-Assisted Instruction in Academic Libraries,” Journal of Academic Librarianship 15 (Jan. 1990): 352–54. 31. Andrew Cox, “Hypermedia Library Guides for Academic Libraries on the World Wide Web,” Program 24 (Jan. 1996): 39–50; Creanor, Durndell, and Primrose, “Library and Study Skills Using Hypertext”; Gregg Richardson, “Computer-Assisted Library Instruction? Consider Your Resources, Commitment, and Needs,” Research Strategies 12 (winter 1994): 45–55. 32. Donna E. Bourne, “Computer-Assisted Instruction, Learning Theory, and Hypermedia: An Associative Linkage,” Research Strategies 8 (fall 1990): 164. 33. Richardson, “Computer-Assisted Library Instruction?” 34. Creanor, Durndell, and Primrose, “Library and Study Skills Using Hypertext”; Pask, “Com­ puter-Assisted Instruction for Basic Library Skills.” 35. Creanor, Durndell, and Primrose, “Library and Study Skills Using Hypertext.” 36. Cox, “Hypermedia Library Guides for Academic Libraries on the World Wide Web”; Heaton, Kazuba, and D’Agostino, “A HyperCard Program to Assist Users at a Self-help Periodi­ cals Information Center.” 37. Cox, “Hypermedia Library Guides for Academic Libraries on the World Wide Web”; Creanor, Durndell, and Primrose, “Library and Study Skills Using Hypertext”; Pask, “Computer- Assisted Instruction for Basic Library Skills”; Shill, “Bibliographic Instruction.” 38. Creanor, Durndell, and Primrose, “Library and Study Skills Using Hypertext”; Richardson, “Computer-Assisted Library Instruction?”; Shill, “Bibliographic Instruction.” 39. Cox, “Hypermedia Library Guides for Academic Libraries on the World Wide Web.” 40. Janice Simmons-Welburn, “Alternative Models for Instruction in the Academic Library: Another View of the Upside of Downsizing,” in The Upside of Downsizing: Using Library Instruc­ tion to Cope, ed. Cheryl LaGuardia, Stella Bentley, and Janet Martorana (New York: Neal-Schuman, 1995), 15–23. 41. Ibid., 21. 42. Shill, “Bibliographic Instruction.” 43. Cox, “Hypermedia Library Guides for Academic Libraries on the World Wide Web”; Pask, “Computer-Assisted Instruction for Basic Library Skills.” 44. Creanor, Durndell, and Primrose, “Library and Study Skills Using Hypertext”; Richardson, “Computer-Assisted Library Instruction?” 45. Pask, “Computer-Assisted Instruction for Basic Library Skills.” 46. Cox, “Hypermedia Library Guides for Academic Libraries on the World Wide Web.” 47. Kaplowitz and Contini, “Computer-Assisted Instruction”; Melick, “Changing to a Hypertext-based Library Instruction Program Using Inexpensive Software for IBM Compatibles”; Richardson, “Computer-Assisted Library Instruction?”; Rumsey, “HyperCard for Bibliographic Instruction”; Rama Vishwanatham, Walter R. Wilkins, and Thomas Jevec, “The Internet as a Medium for Online Instruction,” College & Research Libraries 58 (Sept. 1997): 433–44. 48. Michael O. Engle, “Using World Wide Web Software for Reference and Instruction,” Internet Reference Services Quarterly 1, no. 2 (1996): 8. 49. Andrew Cox, “Using the World Wide Web for Library User Education: A Review Article,” Journal of Librarianship and Information Science 29 (Mar. 1997): 39–43. 50. Ibid. 332 College & Research Libraries July 2001 51. Bourne, “Computer-Assisted Instruction, Learning Theory, and Hypermedia,” 168. 52. Ibid. 53. Kaplowitz and Contini, “Computer-Assisted Instruction”; Pask, “Computer-Assisted In­ struction for Basic Library Skills”; Richardson, “Computer-Assisted Library Instruction?” 54. Pask, “Computer-Assisted Instruction for Basic Library Skills.” 55. Kaplowitz and Contini, “Computer-Assisted Instruction”; Pask, “Computer-Assisted In­ struction for Basic Library Skills.” 56. Germain, Jacobson, and Kaczor, “A Comparison of the Effectiveness of Presentation For­ mats for Instruction”; Pask, “Computer-Assisted Instruction for Basic Library Skills”; Richardson, “Computer-Assisted Library Instruction?” 57. Vishwanatham, Wilkins, and Jevec, “The Internet as a Medium for Online Instruction.” 58. Richardson, “Computer-Assisted Library Instruction?” 59. Ibid., 47. 60. Cox, “Using the World Wide Web for Library User Education.” 61. V. Lonnie Lawson, “Using a Computer-Assisted Instruction Program to Replace the Tra­ ditional Library Tour: An Experimental Study,” RQ 29 (fall 1989): 71–79. 62. Vander Meer and Rike, “Multimedia.” 63. Kaplowitz and Contini, “Computer-Assisted Instruction.” 64. Germain, Jacobson, and Kaczor, “A Comparison of the Effectiveness of Presentation For­ mats for Instruction.” 65. Ibid.; Kaplowitz and Contini, “Computer-Assisted Instruction”; Vander Meer and Rike, “Multimedia.” 66. Lawson, “Using a Computer-Assisted Instruction Program to Replace the Traditional Li­ brary Tour.” 67. Kaplowitz and Contini, “Computer-Assisted Instruction.” 68. Research Committee, Library Instruction Round Table, American Library Association. Evaluating Library Instruction: Sample Questions, Forms, and Strategies for Practical Use, ed. Diana D. Shonrock (Chicago: ALA, 1996). 69. Kaplowitz and Contini, “Computer-Assisted Instruction”; Melick, “Changing to a Hypertext- based Library Instruction Program Using Inexpensive Software for IBM Compatibles”; Richardson, “Computer-Assisted Library Instruction?”; Rumsey, “HyperCard for Bibliographic Instruction”; Vishwanatham, Wilkins, and Jevec, “The Internet as a Medium for Online Instruction.” 70. Miller, “Self-paced Tours in the Electronic Library.”