College and Research Libraries Students' Perceived Effectiveness Using the University Library Cheryl Ann McCarthy Many academic libraries are wondering whether they are providing adequate physical and intellectual access to library resources for their students. Before planning new services, academic librarians must first evaluate students' cur- rent information needs, skills, and satisfaction in using library resources. By using a survey in this investigation, students were asked to rate their effective- ness, their satisfaction, and their needs and expectations in the use of library resources at the University of Rhode Island. After collecting 608 surveys, the data were coded, tabulated, and analyzed, using both quantitative statistical analysis and qualitative content analysis. While the majority of students believe that they are effective seekers and users of library resources, 40 percent were not satisfied with their search for information and materials found. When students were asked what they needed to become more effective users, students recommended better organization and availability of materials, more books and journals, more training and classes, more staff and staff assistance, and en- hanced computer facilities. hile demands and costs for li- brary resources and services increase and budgets decrease, how can university libraries en- sure that students have physical and intel- lectual access to resources needed? How can university libraries ensure that stu- dents know how to use library resources effectively, especially online catalogs and CD-ROM databases? How do stu- dents' expectations change with the ad- vent of these new technologies? Furthermore, do students know how to formulate research questions and how to identify, locate, select, and use resources relevant for their research questions? While planning library instruction and services, librarians should first ask stu- dents about their information needs, skills, and satisfaction in using library resources. Therefore, this study assesses the stu- dents' perceived effectiveness, their sat- isfaction, their needs and expectations in the use of library resources at one uni- versity library. THE UNIVERSITY OF RHODE ISLAND LIBRARY: EXPANSION AND EXPECTATION At the time of this study, spring 1993, the University of Rhode Island Library reached a milestone with near comple- tion of an expansion and renovation project, adding 89,000 square feet and acquiring its one-millionth volume. Simultaneously with the rebuilding project, the new integrated online pub- lic access catalog (OPAC) arrived. The new library, aesthetically pleasing with enhanced computer access, faces a new dilemma: how to meet the increasing expectations of its users in this state-of- Cheryl Ann McCarthy is Assistant Professor at the University of Rhode Island Graduate School of Library and Information Studies, Rodman Hall, Kingston, Rhode Island 02881-0815. 221 222 College & Research Libraries the-art-facility. Diminishing budgets from the state of Rhode Island continue to affect both materials collection and staffing. Expensive serials have been eliminated and staff positions have re- mained unfilled. During the two-year construction project, the library staff en- dured five moves of the entire collec- tion while they continued to provide uninterrupted library service. Everything seemed topsy-turvy, even when it was not. Stress on both library staff and users was great. No longer could users determine location of materials from past experi- ence. Each visit to the library required a mental shift. At the time of this study, most of the materials and the OPAC ter- minals were in their permanent home, but some confusion lingered. With the introduction of new CD- ROM databases and new OPAC termi- nals, it became apparent that students expected better and faster access to ma- terials both inside and outside the Uni- versity Library. The University Library's OPAC is part of a consortium of academic libraries in Rhode Island called the Higher Education Library Information Network (HELIN). HELIN consists of the libraries of the University of Rhode Island, Rhode Island College, the Community College of Rhode Island, Providence College, and Roger Williams University. In addition to the refurbished library, University President Robert Carothers proposed a new vision for the University of Rhode Island, redefining the role of student from a passive learner to an ac- tive learner and a collaborator in re- search. Also under consideration is a proposal for a revised undergraduate in- terdisciplinary program using smaller seminar classes. Common sense sug- gests that demands for library materials and services will increase with a new cur- riculum emphasizing research. Whether present and future students have the nec- essary research skills to develop search strategies and to use appropriate library resources remains the question. This study attempts to identify stu- dents' perceived research abilities and their satisfaction in the use of university library resources. Though narrow in May1995 scope and focus, this study identifies strengths and weaknesses in students' research skills. In addition, it may influ- ence planning and future studies on the role of the University Library. Future plans for bibliographic instruction may also benefit from this study. The focus of this study, therefore, is on students' as- sessment of their research strategies and effectiveness, not their assessment of the library's effectiveness. The author recog- nizes, however, that there may be a cor- relation between students' perceptions of their effectiveness and their percep- tions of the library's effectiveness. The investigator also acknowledges that stu- dents' perceptions may not be the reality. However, the investigator assumes that their responses are an honest attempt to assess their abilities. STUDY DESIGN AND METHODOLOGY Study Design and Validity This study resulted from a request by the acting dean of the library for an in- vestigation into students' use of library resources in order to plan future pro- grams and services. According to Doris Schlichter and J. Michael Pemberton, "Planning and evaluation are not inde- pendent processes. Analysis of users' needs and measurement of the effective- ness of programs and services provide the data upon which rationalized future plans of the library must be based." 1 Thus, the objectives of this study were: • to identify students' perceived effec- tiveness in identifying, locating, se- lecting, and using the University Library resources; • to assess students' reasons for satisfac- tion or dissatisfaction in their search for information and resources at the University Library; and • to analyze what students need to im- prove their effectiveness in the use of the University Library resources. To measure these three objectives ade- quately, the investigator designed and tested a questionnaire. After pretesting the survey and consulting with both the acting dean of the library and the acting director of the library, the questionnaire was revised to eliminate ambiguous lan- guage and to provide a more efficient layout. Research investigations show that user studies employing the survey method may not produce useful data for planning unless the design of the survey asks what is intended and the questions are straightforward.2 After revising the questionnaire and testing for validity, the instrument appeared to measure what was intended-users' success, sat- isfaction, and expectations.3 After an analysis of user studies, Douglas L. Zweizig proposed that meas- uring user satisfaction is probably a bet- ter alternative than measuring the benefits of material availability and in- formation obtained.4 Thus, the user sur- vey asked students to evaluate the degree of their effectiveness for each step of the research process: searching (question 5), identifying (question 7), lo- cating and selecting (question 9), and using materials (question 11). Questions 6, 8, and 12 asked students if they were satisfied with each step of their search and why or why not. Question 13 asked students to explain what would help them to become more effective users of the library's resources. (Copies of the user survey and the coding sheets are available from the author.) Data Collection During the last two weeks in April 1993, the principal investigator along with a graduate assistant, distributed 1,800 questionnaires for approximately two hours each day to students entering the library.5 An attempt was made to ask each student entering the library to take a survey and to return it in the question- naire box upon leaving. The time of the visits varied in order to obtain a wide distribution of students. During this two-week period, 681 library users re- turned their surveys in the box. Only one was eliminated for failure to answer any questions. Of the remaining surveys, 608 student responses were coded and tabu- lated, while 72 faculty or others were coded but not tabulated. Thus, the re- turn rate was more than one-third of the total surveys distributed. Using the University Library 223 The data results reflect the population of students who use the library. The con- cerns of nonusers, therefore, were not considered. In April 1993 there were 10,800 registered University Library bor- rowers out of a student population of approximately 14,000, a faculty of 750, and a staff of 2,000. Methodology, Quality of Data, and Reliability While the survey method was used to collect data, both quantitative and qualitative means were employed to analyze the data. A graduate assistant hand coded the quantitative data. The statistical computer program PC/SAS performed frequency counts, percents, and chi-square tests by comparing sets of data to determine significance.6 The principal investigator performed a content analysis on the open-ended questions by creating coding schemes identifying categories of reasons given for satisfaction, dissatisfaction, and for needs. To ensure intercoder reliability, the graduate assistant and the princi- pal investigator reviewed each other's coding. The principal investigator, how- ever, analyzed and interpreted all the results. (Copies of the frequency counts coding sheets are available from the author.) Content analysis was chosen to ana- lyze data because of its advantage in making inferences by objectively and systematically creating categories from specified characteristics of responses.7 A quantitative content analysis enabled the investigator to identify, count, and rank order categories of responses and assess students' common traits or prob- lems with their search strategies. In ad- dition, with a qualitative content analysis the investigator can offer insights into stu- dents' reasons for satisfaction or dissatis- faction, and their expectations to improve effectiveness. The investigator could make valid inferences and draw conclusions from students' responses while moving between the quantitative and the qualitative content analysis, thereby gleaning an insight into the full meaning of the data. 8 224 College & Research Libraries Utility of Study The data results are interpreted as the beginning stage of an ongoing evalu- ation of the University Library's re- sources and services to evaluate how effectively the library meets the informa- tion needs of its community. Upon com- pletion of this investigation, the library staff was given a draft of this study and invited to discuss the implications for future planning and evaluations. Peter Hernon and Charles McClure identify the importance of this type of evaluation research to help maintain an effective and efficient organization: Evaluatioi). should incorporate plan- ning, research, and change. Wanting to make changes is a necessary prereq- uisite for professional development, the meeting of organizational goals May1995 and objectives, and satisfying the in- formation needs .of current and po- tential clientele in a timely and com- prehensive way.9 INTERPRETATION OF DATA User Satisfaction and Dissatisfaction (Questions 6, 8, and 12) Paradoxically, though 88 percent of re- spondents (488 out of 552) indicated suc- cess in identifying materials (question 6) and 82 percent of respondents ( 453 ·out of 552) indicated success in locating and selecting materials (questiom 8), only 60 percent of respondents (332 out of 550), indicated satisfaction with their search for information and materials found (question 12) (see table 1). Why do stu- dents indicate an almost 25 percent decrease in overall satisfaction with TABLE 1 TABLE OF STATUS BY USER SATISFACTION Question 12: "Were You Satisfied with Your Search for Information and the Material(s) Found? " User Satisfaction or Success 0-No 1-Yes Status (Class Year) Not Satisfied Satisfied Row Total Graduate Frequency 49 79 128 Percentage 9 14 23 Row percentage 39 62 Column percentage 22 24 Upper Frequency 111 162 273 Percentage 20 30 50 Row percentage 41 60 Column percentage 51 49 Lower Frequency 58 91 149 Percentage 10.5 16.5 27 Row percentage 39 61 Column percentage 27 27 Column total 218 332 550 Column percentage 40 60 100 Not satisfied Satisfied Statistics for Table 1-Status by Satisfaction or Success Statistic DF Value Probability Chi-square 2 0.249 0.883 Likelihood ratio chi-square 2 0.249 0.883 Phi coefficient 0.021 their search for information and materi- als found? By analyzing the reasons offered by students for both satisfaction and dissat- isfaction in their searches, the investiga- tor gleaned an insight into students' rationales. Although there were a total of 1,273 yes responses for satisfaction in questions 6, 8, and 12, only 151 reasons were given for satisfaction. More than twice as many reasons were given for dissatisfaction, however, with 330 rea- sons offered from a total of 381 no re- sponses for these questions. Thus, 87 percent of students who were dissatis- fied with their search gave reasons for their discontent, while only 12 percent of those who were satisfied gave reasons for their satisfaction. Perhaps, when stu- dents checked yes for satisfaction on these questions, they thought that the reason was obvious: I succeeded at the task. On the other hand, students who checked no wanted to express their rea- Using the University Library 225 sons for dissatisfaction in hope that the library would make changes. The fol- lowing scenario illustrates students' de- sires for action. Two pharmacy students hand delivered their surveys to the in- vestigator and asked whether the library staff would consider their request for more journals. They asked: "Will the li- brary respond to our recommendations? We know how to research, but we need more current pharmacy journals in order to do our research." The investigator combined reasons given for satisfaction in questions 6, 8, and 1.2 and collapsed them into five ma- jor categories with frequency counts, percents, and rank order under "Satis- faction" in table 2. In addition, reasons given for dissatisfaction in questions 6, 8, and 12 were also combined and classified into ten major categories with frequency counts, percents, and rank order under "Dissatisfaction" in table 3. TABLE2 RANK ORDER "SATISFACTION" Reasons Given for Yes to Questions 6, 8, and 12 Frequency Count % 1. Located materials needed 49 32 2. Used computer (HELIN or CD-ROM) 46 30 3. Satisfied with information or resources found 35 23 4. Received help from staff 15 10 5. Completed assignment 6 4 Total 151 99 TABLE3 RANK ORDER "DISSATISFACTION" Reasons Given for No to Questions 6, 8, and 12 Frequency Count % 1. Lack of materials at URI (books or current materials) 71 22 2. Unable to find enough materials (missing or not on shelf) 65 20 3. Lack of journals wanted on topic (not at URI)-ownership 62 19 4. Unable to find journals (not on shelf, not available, missing, or mutilated) 61 18 5. Search process too long, too confusing, or too difficult 24 7 6. Need more reference staff or staff help 11 3 7. Could not identify sources on topic* 11 3 8. Can not find books listed in HELIN (Not on shelf or not at URI) 10 3 9. Library materials are disorganized* 10 3 10. Need faster or better ILL service 5 2 Total 330 100 •Denotes tie with item above. 226 College & Research Libraries The number one reason given for sat- isfaction was simply, "I got what I needed" or "I found what I wanted." Students also indicated contentment with the quality of materials found (rea- son number three). Evidence cited for satisfaction with the quality and quan- tity of materials found include com- ments such as, "diverse information and good quantity," "fairly compre- hensive coverage of information," and materials found "pertained well to my topic." Thus, by combining reasons number one and three, more than half of all responses given for satisfaction were ease in locating materials and contentment with the quality and quantity of materials found. The second highest reason cited for satisfaction is the use of computers to help find information resources. Thirty percent of the written responses for satis- faction praised CD-ROM databases and HELIN. Comments included, "HELIN is GREAT," and "CD-ROM and HELIN, es- pecially [are the] best addition to the li- brary." Satisfaction with computer access seems, paradoxically, to increase students' expectations for more computer services and better access to materials. One stu- dent suggested that although "I found some of what I needed, a lot mentioned in HELIN [was] not available at URI." Another student recommended that the library should add computer "online re- quest for books and Gopher service in [the] library building." Others requested more computer terminals, more CD- ROM databases, and more online serv- ices . Students using CD-ROM databases indicated satisfaction with their serial citations, but dissatisfaction with a lack of journals at URI. Several users of MEDLINE claimed they were dissatis- fied with a lack of medical and science journals to match their citations. One student wrote, "This library didn't have any of the articles/journals listed in computer [database-MEDLINE], had to go to Brown." Another user praised search results using several computer databases: HELIN, MEDLINE, Applied Science & Technology, and ABI, but com- plained, "once [I] got the references, it May1995 was difficult to obtain journals & books in [the URI] library." Moreover, 10 percent of satisfied stu- dents indicated that the reason they suc- ceeded was because of staff assistance. Students commented that they "used staff help" or "asked staff guidance" to get the information or sources they needed. The remaining 4 percent of satisfied students stated that they were pleased merely to complete their assignments. Satisfaction with computer access seems, paradoxically, to increase students' expectations for more computer services and better access to materials. Coincidentally, the major reasons given for dissatisfaction correlate with the major reasons given for satisfaction. While 55 percent of students claimed sat- isfaction with locating materials and with the quality and quantity of materi- als they found, 42 percent of students claimed dissatisfaction because of their inability to find materials and because of the lack of materials available. Journals were cited as the item most frequently missing or not available at URI. Nine- teen percent expressed discontentment with the lack of journals. In addition, 18 percent expressed frustration with their inability to find journals or articles be- cause they were missing, misshelved, mutilated, or not on the shelf. By com- bining these top four reasons, nearly 80 percent of the dissatisfied students were discontent because of a lack of materials or journals and their inability to find materials or journals needed. More- over, what appears to please students most-the ability to find materials and the quality and quantity of materials found-also appears to displease stu- dents most-the inability to find materi- als and the lack of resources needed. Although 10 percent of the satisfied students received help from the staff, 13 percent of the dissatisfied students indi- cated a need for more staff assistance. By combining three categories for dissatis- faction (search process too difficult, need more staff assistance, and could not identify sources) into one explanation, the investigator inferred that 13 percent would benefit from more instruction or staff assistance in their search process. If more staff were available to assist stu- dents or to teach them, then students might be able to clarify their own needs and locate appropriate resources while increasing their satisfaction and skills using the library. Some students blamed the staff for being unresponsive; how- ever, others claimed that the library was understaffed, especially in the evening. One comment illustrates a plea for help, "What we need is more staff members to assist in the [research] process." Although there is not a statistically significant difference in the degree of effectiveness, it appears that graduate students are more confident in their skills to search and use library resources while lower division undergraduates appear less confident in their skills. Other reasons cited for discontent were: cannot find books listed in HELIN (3 percent) and need faster or better in- terlibrary loan (ILL) service (2 percent). While 30 percent of the students ex- pressed satisfaction with the new com- puters, 3 percent claimed frustration with computers because of their inabil- ity to find materials listed in HELIN. "Some materials [are] not on shelf, but should have been, according to HELIN," wrote one student. Students commented that, although they identified citations from CD-ROM databases, the journals either were not available at URI or that they were unable to find them: "I found the information from CD-ROM with no problem, but the journal articles weren't available." One inference seems plausi- ble: once students' identify citations from the computer databases (HELIN or CD-ROM), they expect all of the materi- als (books and journals) listed in the computers to be available at URI, Using the University Library 227 whether the library owns them or not. Moreover, they desire access to these materials either online or via document delivery. They also requested online ac- cess to ILL via the computer terminal. Although no one suggested the virtual library, dormitory delivery, or robotics retrieval, requests were made for more online access to library materials from remote areas. In addition, 2 percent com- plained that library materials are disor- ganized. These responses may relate to the construction moves, temporary shelving, or misshelving. Some students commented that materials, especially journals on the lower level, should be reshelved faster and missing or muti- lated articles should be replaced. User Effectiveness (Questions 5, 7, 9, and 11) A comparison of student status (class year) by user effectiveness in questions 5, 7, 9, and 11, shows that regardless of task or class year, when it comes to seek- ing and using library resources, approxi- mately 65 percent of all students believe that they do it well. Surprisingly, there 'was no significant difference among class year and effectiveness as evidenced by the chi-square test where the value did not exceed the expected ratio for each of these questions. The phi coeffi- cient, which measures the strength of a relationship, approximated the value of zero in each case, thereby demonstrating that there is no association or correlation between class year and students' per- ceived ability to search, identify, locate, select, and use library resources. Before tabulating these results, class years were collapsed into three groups in order to create valid contingency ta- bles where the cells had a value of at least five. Freshmen and sophomores were grouped under the status "lower" representing lower division undergradu- ates. Juniors and seniors were combined under the status "upper" representing upper division undergraduates. Master and Ph.D. candidates were grouped as "graduate" students. Faculty and others were not tabulated. Half of the total respondents were upper division undergraduates, while approximately 228 College & Research Libraries one-quarter were lower division under- graduates and the remaining quarter were graduate students. Thus, there was an adequate representation of each of the class years. The levels of effectiveness were collapsed into three categories: high (4 and 5), fair (3) and low (2 and 1). By comparing the degree of effective- ness, similar patterns emerge for each question. The ratios for each question are surprisingly constant, except for an increase of 20 percent in question 11 (ef- fectiveness in using materials). Students responded to question 5 ("rate your effectiveness in searching and using the resource(s) checked in question 4") with 65 percent rating their May 1995 effectiveness high (good or excellent) compared with 25 percent fair, and 9 percent low (poor or needs training) (see table 4). Students responded to ques- tion 7 ("rate your effectiveness in identifying material(s) for your infor- mation need") with 70 percent rating their effectiveness high, compared with 23 percent fair, and 7 percent low (see table 5). Similarly, when students responded to question 9 ("rate your effectiveness in locating and selecting materials"), 66 percent rated their effec- tiveness high, compared with 24 percent fair, and 10 percent low (see table 6). Thus, in all three questions there ap- pear only slight differences in the de- TABLE4 TABLE OF STATUS BY SEARCH EFFECTIVENESS Question 5: "Overall, How Would You Rate Your Effectiveness in Searching and Using the Resource(s) Checked in Question 4?" User Effectiveness in Searching Low (1 and 2) Fair High (4 and 5) 1 2 3 4 5 Status (Class) Need Help Poor Fair Good Excellent Row Total Graduate Frequency 5 8 19 65 39 136 Percentage 0.82 1.32 3.12 10.69 6.41 22.37 Row percentage 3.68 5.88 13.97 47.79 28.68 Column percentage 19.23 26.67 12.50 23.55 31.45 Upper Frequency 14 12 84 136 61 307 Percentage 2.30 1.97 13.82 22.37 10.03 50.49 Row percentage 4.56 3.91 27.36 44.30 19.87 Column percentage 53.85 40.00 55.26 49.28 49.19 Lower Frequency 7 10 49 75 24 165 Percentage 1.15 1.64 8.06 12.34 3.95 27.14 Row percentage 4.24 6.06 29.70 45.45 14.55 Column percentage 25.92 33.33 32.24 27.17 19.35 Column totals 26 30 152 276 124 608 Percentage 4.28 4.93 25.00 45.39 20.39 100 Combine column totals 1 and 2 (Low) 3 (Fair) 4 and 5 (High) Frequency 56 152 400 608 Percentage 9 25 66 100 Statistics for Table 4-Status by Search Effectiveness Statistic OF Value Probability Chi-square 8 17.894 0.022 Likelihood ratio chi-square 8 18.863 0.016 Phi coefficient 0.172 gree of effectiveness in comparison to class status. Graduate students had the highest percentage of high ratings in questions 5, 7, and 9 while lower division under- graduates had the highest percentage of fair and low ratings. Although there is not a statistically significant difference in the degree of effectiveness, it appears that graduate students are more confi- dent in their skills to search and use library resources while lower division undergraduates appear less confident in their skills. Moreover, with approxi- mately thirty-five percent of all respon- dents identifying their effectiveness as fair or low, a significant number of stu- dents appear to need help. Thus, it seems Using the University Library 229 important to this investigator to identify what students need to succeed so that the library can plan accordingly. Question 11 ("Rate your effectiveness using the material(s) found") had the highest rating with 86 percent of all stu- dents rating their effectiveness high, compared with 11 percent fair and 3 per- cent low (see table 7). Apparently, once students have found materials, they seem confident in their ability to use them effectively. Statistically there was no significant difference among class status. Thus, assessing the responses to question 13 provides insight into what students expect in order to be- come more effective users of library resources. TABLE 5 TABLE OF STATUS BY EFFECTIVENESS IN IDENTIFYING SOURCES Question 7: "How Would You Rate Your Effectiveness in Identifying Material(s) for Your Information Need?" User Effectiveness in Identifying Sources Low (1 and 2) Fair High ( 4 and 5) 1 2 3 4 5 Status (Class Year) Need Help Poor Fair Good Excellent Row Total Graduate Frequency 0 4 23 66 40 133 Percentage 0 0.70 4.02 11.54 6.99 23.25 Row percentage 0 3.01 17.29 49.62 30.08 Column percentage 0 22.22 17.42 22.00 39.22 Upper Frequency 12 9 66 158 44 289 Percentage 2.10 1.57 11.54 27.62 7.69 50.52 Row percentage 4.15 3.11 22.84 54.67 15.22 Column percentage 60.00 50.00 50.00 52.67 43.14 Lower Frequency 8 5 43 76 18 150 Percentage 1.40 0.87 7.52 13.29 3.15 26.22 Row p~rcentage 5.33 3.33 28.67 50.67 12.00 Column percentage 40.00 27.78 32.58 25.33 17.65 Column totals 20 18 132 300 102 572 Percentage 3.50 3.15 23.08 52.45 17.83 100 Combine column totals 1 and 2 (Low) 3 (Fair) 4 and 5 (High) Frequency 38 132 402 572 Percentage 7 23 70 100 Statistics for Table 5-Status by Effectiveness Identifying Sources Statistic DF Value Probability Chi-square 8 26.153 0.001 Likelihood ratio chi-square 8 29.123 0.000 Phi coefficient 0.214 ~--------------------------~------------------------------------------------, 230 College & Research Libraries May1995 TABLE 6 TABLE OF STATUS BY EFFECTIVENESS IN LOCATING & SELECTING SOURCES Question 9: "How Would You Rate Your Effectiveness in Locating and Selecting Material(s)?" User Effectiveness in Locating and Selecting Sources Low (1 and 2) Fair High (4 and 5) 1 2 3 4 5 Status (Class Year) Need Help Poor Fair Good Excellent Row Total Graduate Frequency 0 7 23 68 31 129 Percentage 0 1.23 4.05 11.97 5.46 22.71 Row percentage 0 5.43 17.83 52.71 24.03 Column percentage 0 17.50 17.04 24.55 31.63 Upper Frequency 11 19 64 141 49 284 Percentage 1.94 3.35 11.27 24.82 8.63 50 Row percentage 3.87 6.69 22.54 49.65 17.25 Column percentage 64.71 47.50 47.41 50.90 50.00 Lower Frequency 7 14 48 68 18 155 Percentage 1.24 2.46 8.45 11.97 3.17 27.29 Row percentage 4.52 9.03 30.97 43 .87 11.61 Column percentage 35.29 35.00 35.56 24.55 18.37 Column totals 18 40 135 277 98 568 Percentage 3.17 7.04 23.77 48.77 17.25 100 Combine column totals 1 and 2 (Low) 3 (Fair) 4 and 5 (High) Frequency 58 135 375 568 Percentage 10 24 66 100 ~-··--·-···-----·--··-·······-·-········· ·· ········-······ ·········-········---·----· ·-----·-··-·········-···-····-········-·········-···-···-··--·····-·--·- Statistics for Table 6-Status by Effectiveness Statistic DF Chi-square 10 Likelihood ratio chi-square 10 Phi coefficient Students' Expectations (Question 13) Students' responses to question 13 ("What would help you to become more effective in using the resources of the Uni- versity Library?") were coded into nine categories then identified, counted, and rank ordered by the investigator. Interest- ingly, students tend to recommend improve- ments in library resources, services, and staff, rather than identify areas for their own growth or improvement in skills. They tend to blame the library's resources and staff for their ineffective searches rather than identify their own inade- quacies in using library resources. in Locating and Selecting Sources Value Probability 22.048 0.015 25.616 0.004 0.197 Forty percent of the students believe that they could become more effective users of the library if the library had better organization and availability of materials, and more materials (see rea- sons one and three in table 8). Students seem more concerned with improve- ments in library resources than with im- provements in their own skills. One student claimed, "You are asking the wrong question. I'm O.K., but the library needs to improve." Thirty-five percent of the students claimed that the library needed to pro- vide more training sessions and more staff assistance if students were to be- Using the University Library 231 TABLE 7 TABLE OF STATUS BY EFFECTIVENESS IN USING SOURCES FOUND Question 11: "How Would You Rate Your Effectiveness in Using the Material(s) You Found?" User Effectiveness in Using Sources Found Low (1 and 2) Fair High ( 4 and 5) Status (Class Year) Graduate Frequency Percentage Row percentage Column percentage Upper Frequency Percentage Row percentage Column percentage Lower Frequency Percentage Row percentage Column percentage Column totals Percentage 1 Need Help 2 0.35 1.57 25.00 5 0.88 1.75 62.50 1 0.18 0.65 12.50 8 1.42 2 3 Poor Fair 1 10 0.18 1.77 0.79 7.87 9.09 16.13 5 34 0.88 6.02 1.75 11.93 45.45 54.84 5 18 0.88 3.19 3.27 11.76 45.45 29.03 11 62 1.95 10.97 4 5 Good Excellent Row Total 75 39 127 13.27 6.90 22.48 59.06 30.71 22.26 26.53 164 77 285 29.03 13.63 50.44 57.54 27.02 48.66 52.38 98 31 153 17.35 5.49 27.08 64.05 20.26 29.08 21.09 337 147 565 59.65 26.02 100 Combine column totals Frequency 1 and 2 (Low) 3 (Fair) 4 and 5 (High) 19 62 484 565 Percentage 3 11 86 100 ·······-········-····---· .. ····-······················································································-···-·-··--····-·--···---·-·-··········-·········-···········-·· Statistics for Table 7-Status by Effectiveness Statistic Chi-square Likelihood ratio chi-square Phi coefficient DF 8 8 come more effective users of library re- sources (see reasons two and four on table 8). One respondent said that the li- brary should require a "mandatory refer- ence course for newcomers-our gym facility mandates one-why shouldn't the library?" Some students would like classes tailored to their subject interests while others recommended general orientation sessions scheduled peri- odically so students can learn "every- thing" in the library and how to use it. Several students requested training ses- sions on HELIN and CD-ROM databases to learn search strategies. One student asked: "Why not publicize a schedule in Using Sources Found Value 8.469 8.842 0.122 Probability 0.389 0.356 of training sessions or classes in the Cigar [the student newspaper] so stu- dents will know when the library is offer- ing programs?" The fourth highest request was to in- crease the staff and provide more staff assistance. Twelve percent of the stu- dents stated that the library was either understaffed or the present staff was not accessible enough for students. Some commented that during busy times, es- pecially in the evening, there is never enough help at the reference desk. Others want staff assistance on each level, espe- cially the lower level, to help students locate journals. 232 College & Research Libraries May 1995 TABLES RANK ORDER: EXPECTATIONS TO BECOME MORE EFFECTIVE USERS Question 13: "What Would Help You Become More Effective in Using the Resources of the University Library?" Reasons Given for Question #13 and Additional Comments Frequency Count % 1. Better organization and availability of materials (reshelve journals, replace missing volumes or mutilated journal articles) 111 28 2. More training, tours, classes, or more practice and skill in using library resources 90 23 3. Buy more materials-journals, books, videos 47 12 4. Need more staff and more staff help 46 12 5. Better signage, maps, guidelines, handouts or location (on computer) 45 11 6. Improve computer facilities-more access, printers, better databases and cross-references, including CD-ROMs and periodical holdings online, and combine HELIN and CD-ROM 7. Better and faster ILL or send HELIN materials 8. Better access to materials 9. More and better working copy machines Totals Better signage (i.e., maps, guidelines, handouts, and highlighting locations on the computer terminal) was requested by another 11 percent of the students. In fall 1990 investigators performed an evaluation of signs at the University Li- brary. The investigators concluded that although they did not have enough data to make statistical inferences, signs could be more effective if simple, straightforward, and unambiguous in communicating location, especially for new users trying to find specific items. The investigators acknowledged that new signs were needed at URI and would be included with the construction project. They also recommended an evaluation of signage upon the comple- tion of the construction project.10 When an evaluation of signage is redone, us- ers may show an increased satisfac- tion with the new signs in helping them to locate materials in the larger and better designed facility. With the advent of the computer ter- minals in this newly refurbished facility, students seem to express a desire for better and faster access to materials. Thus, students requested more computer serv- 36 9 9 2 7 2 4 1 395 100 ices in the library. Nine percent of the students indicated that they would like expanded computet facilities in the library. Students seem to expect the computers to both simplify and speed up their search process. After identify- ing books or serial citations, students complained about seeking materials: "I found what I needed in the computer, but now I have to find whether the li- brary has them and where they are lo- cated." Students also complained about having to verify citations in the serial holdings "red book" for call numbers and then having to go to the basement to search for journals. They asked, "Why can't the red book be online?" Some suggestions for improving com- puter facilities reveal both students' so- phistication and their naivete with regard to the technology. Some students asked if HELIN and CD-ROM databases could be merged with one interface to provide access to all holdings on one computer, including serials and the full text of journal articles. Students wanted to know why the CD-ROM databases gave citations for journals that the URI Library did not own. Several students requested more computer terminals on each level as well as printers. One stu- dent admitted that s/he wanted a print- out of citations from HELIN because "I'm to lazy" to write them down. Fur- thermore, students asked for better da- tabase access with cross references and the ability to do online searching on the Internet using the HELIN terminals. Along with the expectation for im- proved computer facilities, students re- quested better and faster ILL, with online ILL access for items from the other academic libraries in HELIN. Some students complained that ILL re- quests take too long and if they need materials, then they have to drive to other libraries in Rhode Island to re- trieve them. Students also complained that they were not notified of the status of ILL materials or of recall books. Moreover, students requested better access to materials and better working copy machines. Some individuals asked for more creature comforts such as couches, less heat, and completion of the construction project. One student com- mented that s/he would become more effective using the library by "never join[ing] the Greek system ... [then] I would have to use my brain more often." Another student claimed "less procrasti- nation" would help. These isolated re- sponses do not appear in the rank order in table 8. However, this investigator ob- served that students who procrastinate until the end of the semester in begin- ning their search for sources become eas- ily frustrated with the library. Students expect to find library resources quickly with the use of computer databases or with the help of the staff. They also want to find the materials they need readily available on the shelf. CONCLUSIONS How effective are students in using university library resources? Based on students' perceived abilities to search and use university library resources, most students believe that they can use library resources effectively. While 86 percent of the total students perceive that they can use materials effectively, Using the University Library 233 only 60 percent, however, are satisfied with their search for information and the materials found. Why were 40 percent of the students dissatisfied with their search process and what do they need to become more productive users? Students expect to find more materials on the shelf when they need them. They also recognize that they need more help in using the library, and thus requested training sessions or classes. They specifi- cally identified the need to learn effec- tive search strategies on HELIN and CD-ROM databases. Students would like more assistance from staff and more accessibility to staff. In addition, they would appreciate better signage to com- municate not only location but also to provide guidance while searching. Stu- dents would like enhancements to com- puters for better access to collections by combining HELIN and CD-ROM data- bases with one interface, including seri- als holdings. One student seemed to say it all when s/he said that in order for students to become more effective us- ers of library resources: "Get the library finished, fully staffed, and immensely funded." The library construction was com- pleted and a ribbon-cutting ceremony was held in September 1993. Now it ap- pears that the other two pieces are needed to fulfill students' expectations in this state-of-the-art-facility: a fully staffed library and an increased budget to improve holdings. Whether the library can maintain its current level of services with a reduced staff and a diminished budget is doubtful. The dilemma of increased ex- pectations for more technology, materials, training, and staff, without an increase in budget, remains a problem. Requests for expanded library instruction and more user services may require a shift in personnel. The current level of individ- ual instruction provided at point-of- need requires more reference staff than is currently available to teach students on a one-to-one basis . Recent studies indicate that in order for academic ref- erence services to work, academic librar- ies need organizational change and rethinking of reference services. 11 234 College & Research Libraries It is apparent that if students are to improve their effectiveness, they need more instruction to become more skilled using library resources. In order for this to occur, the library needs more public services staff and a rethinking of refer- ence services. It is recommended that the library assess its current reference serv- ices and instruction program in light of students' curriculum needs and skills. Furthermore, it is recommended that the library identify what training sessions are needed in order for students to im- prove their skills and adequately com- plete their research assignments. A reassessment of current library staff and services could help identify programming, staffing, and budgeting needs to expand programs in public services. Workshops and training ses- sions on search strategies using HELIN and CD-ROM databases are recom- mended in addition to the current fresh- men orientation sessions for Writing 101. Hands-on workshops for CD-ROM da- tabase searching by subject could help both faculty and students become better acquainted with the new library resources and the technology as well as relieve li- brary anxiety. Whether this can become May1995 a reality at the University of Rhode Is- land is uncertain, unless the public services staff wants to assess current serv- ices and offer expanded programs. Also, the university needs to make a contin- ued commitment to the library budget in order to enhance materials collection and to replace unfilled staff positions. EPILOGUE One year after this study, the budget and staff levels at the University Library remain the same, but future plans are ongoing for evaluating library services. As a result of this study, the investigator and the head of Reference Services are preparing to offer and to assess CD- ROM workshops and hands-on training sessions to evaluate students' search strategies. Both participant evaluation surveys and observation methods will be used to evaluate students' success in searching. This joint investigation is in di- rect response to students' requests in this study for training in CD-ROM database searching. Thus, with another study the library can continue to evaluate, plan, and improve services at the University Library to better serve the information needs of the university community. REFERENCE AND NOTES 1. Doris Schlichter and J. Michael Pemberton, "The Emperor's New Clothes?: Problems of the User Survey as a Planning Tool in Academic Libraries," College & Research Libraries 53 (May 1992): 40. 2. Lowell A. Martin, "User Studies and Library Planning," Library Trends 24 (Jan. 1976 ): 483-96. 3. Ole R. Holsti, Content Analysis for the Social Sciences and Humanities, (Reading, Mass.: Addison-Wesley, 1969), 40. 4. Douglas L. Zweizig, "Measuring Library Use," Drexel Library Quarterly 13 (July 1977): 3-15. 5. Assistance provided by Alison Maxell, graduate assistant at the Graduate School of Library and Information Studies, University of Rhode Island in 1993; currently director of the North Attleboro Public Library, North Attleboro, Massachusetts. 6. Assistance with program for PC/SAS provided by Gale Eaton, associate professor, Graduate School of Library and Information Studies, University of Rhode Island. 7. Holsti, Content Analysis, 14. 8. Ibid., 30. 9. Peter Hernon and Charles R. McClure, Evaluation and Library Decision Making, (Norwood, N.J.: Ablex, 1990), 224. 10. Gale Eaton, Michael Vocino, and Melanie Taylor, "Evaluating Signs in a University Library," Collection Management 16 (1992): 81-101. 11. See David W. Lewis, "Making Academic Reference Services Work," College & Research Libraries 55 (Sept. 1994): 445-56.