Issues in Science and Technology Librarianship | Winter 2002 | |||
DOI:10.5062/F4T43R1M |
URLs in this document have been updated. Links enclosed in {curly brackets} have been changed. If a replacement link was located, the new URL was added and the link is active; if a new site could not be identified, the broken link was removed. |
The University of Washington Libraries has conducted large-scale library surveys on a triennial cycle since 1992. These surveys are sent to all faculty members and a random sample of graduate and undergraduate students. Surveys measure user satisfaction with library services and resources and have also included questions on the reasons faculty and students use (or don't use) libraries, use and application of electronic information, importance of information resources, and their priorities for library services and resources. Survey results comprise a rich lode of information about library use and needs during a period of rapid change in the information environment.The University of Washington Libraries is unique not only in the regularity of its systematic surveying of the user community but also in the large number of surveys that are returned, especially by faculty (1,100 to 1,500 for each survey). The size and representative nature of the respondent pool has enabled us to generalize for the population as a whole and do statistical analysis by academic subject area. One of the more striking findings of this analysis has been the substantial differences shown in library/information needs and use by those in different subject fields. This paper will focus on the differences (and also some similarities) between scientists/engineers and other academic areas in their library use and information needs at a large American research university.
These are just a few questions that many of us are asking about our user communities. We understand that life in the information world has changed dramatically during the past ten years but we're unsure of the impact on our users and what it means for the future. A number of libraries have developed user surveys or employed other methods such as focus groups to gain a better understanding of these changes. While these efforts have provided information at the institutional level, or in specific subject areas, it is rare to find studies that provide sufficient information at both the aggregate and subject area levels which can be used to provide a cohesive picture of library and information use at an American research university. For example, surveys done at the University of California, San Diego (Talbot, Lowell, and Martin 1998), Duke University (Berger and Hines 1994), and the University of Virginia (Hiller and Self 2001) generally did not have a sufficient number of faculty responses to do subject level analysis.
Data from surveys conducted by the University of Washington Libraries provide a detailed view of library and information use and needs both at the institutional level and by broad subject area that can begin to answer some of the questions posed above. Survey instruments and aggregate level results can be found at: {http://www.lib.washington.edu/assessment/}
Because this paper compares survey data from those in different subject areas, it includes results from faculty and graduate student surveys, but not the undergraduate ones. The connection between undergraduates and specific fields of study are not as well delineated.
User surveys can be designed and administered in a number of ways. Surveys can range from broad and comprehensive to those narrowly focused on specific services or activities. When properly designed and administered, user surveys can provide both quantitative and qualitative data directly from the target population. If the sample or survey response is large enough and representative of the surveyed population, data and results can be used to generalize for the population as a whole. This ability to provide statistically valid results from a smaller group makes the user survey a very powerful tool. Surveying the user community on a regular cycle can also provide valuable longitudinal data and the ability to measure change over time.
The user survey also has limitations. User perceptions are recorded, not actual performance. Surveying is often time consuming and expensive. Changes in survey design and group composition may reduce the reliability of longitudinal comparisons. It is difficult to frame questions regarding complex issue and nuances may be lost in a mass survey.
Survey results can and should be used with other measures/user input such as counts, observation, and focus groups to provide a more comprehensive view of library performance and user behavior. The user survey is most useful when its results are combined with other data. It can corroborate apparent trends, support proposed initiatives, or reveal hidden problems. The survey is a valuable tool, but only one in a whole array of data collection possibilities.
The University of Washington Libraries supports the work of the academic community through a decentralized library system and online provision of remote resources and services. The collection totals six million volumes and there are substantial electronic resources available through the campus computing network. The "main" library houses the primary humanities and social sciences collections as well as a separate science library that supports programs in the earth and life sciences as well as general science. There are separate Health Sciences, Business, East Asia, and Undergraduate libraries, as well as several branch libraries for the fine arts and a number of science branch libraries. The latter includes libraries for Chemistry, Engineering, Fisheries-Oceanography, Forest Resources, Mathematics, and Physics-Astronomy.
The survey population included all faculty and a random sample of graduate students. While distributing the survey to all faculty would increase costs, it would also facilitate survey promotion and publicity, obtain sufficient number of responses to do analysis by academic subject areas, and foster positive political outcomes.
Survey questions were similar for faculty and graduate students. Each survey contains a series of 12-18 questions, many with five-point Likert scales (three-point scales were used in 1992). Approximately 25% of the questions change between surveys due to new areas of interest or the responses in the previous survey not providing useful information. However, there were a core group of questions in each survey that dealt with: information sources needed for research, teaching and learning; reasons and frequency of library use; use of electronic resources; instructional needs and effectiveness; library unit use; satisfaction; services availability or satisfaction.
Additional information on UW Libraries survey methodology, administration, and design can be found in Hiller (2001) and Hiller and Self (2001).
Survey results also documented significant variations between groups and academic areas. Differences in academic areas are most pronounced in priorities, use patterns, importance of information resource formats and the impact of new technology on library use. Differences between areas were not evident in overall satisfaction and libraries' importance. This paper will focus primarily on results from the 1998 and 2001 surveys, with some data from 1995 where comparable. Where a significant difference is noted between groups or surveys, statistical analyses using t-tests or ANOVA have been employed.
Survey return rates are shown in Table 1. The number of completed surveys returned by faculty is sufficiently large to perform statistical analysis of results at the school/college level and in some cases by academic department. While graduate students had a slightly higher response rate, the sample size precluded valid analysis at the department or school level. The survey size for graduate students was increased in 2001 in order to obtain a larger number of responses.
Table 1. Surveys distributed and returned 1998 and 2001 | ||||||
---|---|---|---|---|---|---|
Survey | Faculty | Graduate Students | ||||
Year | Sent | Returned | Rate | Sent | Returned | Rate |
2001 | 3720 | 1345 | 36.2% | 1394 | 563 | 40.4% |
1998 | 3750 | 1503 | 40.1% | 1000 | 457 | 45.7% |
Respondents provided information on their primary academic department and these were then grouped into three broad subject areas for analytical purposes (Appendices I and II). As Table 2 shows faculty and graduate student respondents by broad academic areas closely resembled the population as a whole. The minimum number of respondents in each of these broad areas for the 1998 and 2001 surveys was 305 for faculty and 128 for graduate students. While there was some variation within groups, these were generally much lower than between groups.
Table 2. Faculty and graduate student population (P) and respondents (R) by academic area, 1998 and 2001. | ||||||
---|---|---|---|---|---|---|
Academic Area | Faculty 1998 R | Faculty 2001 P | Faculty 2001 R | Grad 1998 R | Grad 2001 P | Grad 2001 R |
Health Sciences | 44.6% | 48.6% | 47.7% | 28.0% | 30.8% | 27.9% |
Humanities-Social Science | 24.4% | 21.0% | 22.7% | 43.8% | 38.8% | 41.0% |
Science-Engineering | 27.1% | 26.2% | 26.3% | 28.2% | 30.4% | 30.9% |
Other* | 3.9% | 3.7% | 3.3% | 0.2% | ||
*Other includes Law, Branch Campus, and where academic area could not be identified. |
In 2001, slightly more than 40% of the faculty reported visiting the library at least weekly compared to 47% in 1998. The decline in weekly visits among graduate students was more dramatic: from nearly 78% in 1998 to less than 60% in 2001. The decrease in physical visits was most pronounced among faculty and graduate students in the health sciences, science, and engineering. It is interesting to note that undergraduate use remained relatively constant in 2001 compared to 1998 with about 67% visiting the Libraries at least weekly. The primary use of library facilities by undergraduates tends to be as a work place rather than to find books or journals.
Table 3 shows the dramatic change in in-library use categories between 1998 and 2001 among faculty and graduate students. These responses are validated by other data such as circulation statistics, in-library material use, number of photocopies made, and decline in reference activity.
While we can see significant differences in physical visits to the library by subject area, there is no difference by academic area in remote use from a campus office by academic area. There continues to be a difference in the frequency of use from home by faculty and graduate students. That gap is narrowing as a larger percentage of scientists and engineers connect from home.
Table 3. Type of Library Use by Group and Academic Area. % of respondents who marked at least weekly | ||||||
---|---|---|---|---|---|---|
Visit in person | Use office computer | Use home computer | ||||
FACULTY | 1998 | 2001 | 1998 | 2001 | 1998 | 2001 |
Health Sciences | 37.9 | 28.1 | 76.2 | 75.7 | 40.5 | 43.4 |
Humanities-Soc Science | 60.7 | 56.4 | 70.2 | 76.7 | 47.1 | 51.5 |
Science-Engineering | 49.3 | 41.8 | 64.7 | 75.4 | 23.6 | 33.9 |
All Faculty | 47.3 | 40.6 | 71.0 | 76.1 | 37.4 | 43.0 |
GRAD | ||||||
Health Sciences | 79.7 | 59.6 | 39.8 | 50.6 | 49.2 | 59.6 |
Humanities-Soc Science | 82.5 | 72.1 | 47.5 | 56.1 | 52.0 | 62.6 |
Science-Engineering | 68.2 | 45.1 | 57.4 | 69.4 | 32.6 | 42.5 |
All Grad Students | 77.7 | 59.6 | 48.1 | 58.5 | 45.7 | 55.2 |
The reasons graduate students gave for visiting libraries in 1998 and 2001 provide useful information related to the decline in physical use (Table 4). Science and engineering students visited most often to find journals and were least likely to use the library as a place to do work. As more information became available through the desktop there was less need to visit the library. Most graduate students in science and engineering have workspace within their departments or labs. That's not the case for graduate and professional students in the health sciences who often lack private workspace and use the large computer lab in the Health Sciences Library to do their work.
Table 4. Type of Use When Visiting the Library % of graduate students who visit the Libraries at least weekly to: | ||||||||
---|---|---|---|---|---|---|---|---|
Look for a journal | Photocopy | Look for book | Place to do work | |||||
GRAD | 1998 | 2001 | 1998 | 2001 | 1998 | 2001 | 1998 | 2001 |
Health Sciences | 57.8 | 42.0 | 57.1 | 40.1 | 16.4 | 20.4 | 55.4 | 46.5 |
Hum-Social Sciences | 55.5 | 47.6 | 42.5 | 29.9 | 65.0 | 48.1 | 59.5 | 36.4 |
Science-Engineering | 64.4 | 32.2 | 35.7 | 13.2 | 39.5 | 24.7 | 30.2 | 11.5 |
All Grad Students | 58.7 | 41.3 | 44.6 | 27.7 | 44.2 | 33.0 | 50.3 | 31.7 |
The reasons for remote use show a somewhat similar pattern with faculty in the health sciences most frequently looking for full text while those in the humanities and social sciences were more likely to search the catalog. Science and engineering faculty were equally likely to search the catalog, a bibliographic database or look for full text.
Table 5. Type of Use When Using the Library Remotely in 2001 % of faculty who connect to the Libraries remotely at least weekly to: | ||||||||
---|---|---|---|---|---|---|---|---|
Search lib catalog | Search Bib Database | Look for full text | Renew/place holds | |||||
Faculty | Grad | Faculty | Grad | Faculty | Grad | Faculty | Grad | |
Health Sciences | 32.4 | 43.3 | 44.3 | 36.9 | 61.5 | 60.5 | 2.3 | 7.6 |
Humanities-Social Sci. | 75.4 | 63.6 | 57.4 | 44.6 | 35.4 | 46.8 | 14.8 | 21.6 |
Science-Engineering | 55.1 | 70.0 | 55.1 | 46.0 | 54.0 | 62.1 | 5.1 | 11.5 |
ALL | 49.2 | 59.7 | 49.9 | 43.0 | 53.5 | 55.4 | 6.4 | 14.5 |
When asked if online library resources and information technology had changed the way they work, scientists and engineers were less likely to visit the library and more likely to find journal citations, keep current, and be a more productive researcher than those in the humanities and social sciences. There were no significant differences in use of interlibrary loan, getting information from non-library sources, consulting librarians, using library services in teaching, or being a more effective instructor.
Table 6. Impact of online information resources and information technology on work. Mean score ranging from 1 (less likely) to 5 (more likely) with 3 equivalent to no change | ||||||
---|---|---|---|---|---|---|
Health Sciences | Humanities-Soc Sci | Science-Engng | ||||
Faculty | Grad | Faculty | Grad | Faculty | Grad | |
Visit library in person | 1.93 | 2.68 | 2.68 | 2.91 | 2.20 | 2.55 |
Find books | 3.06 | 3.48 | 3.71 | 3.84 | 3.56 | 3.98 |
Find journal citations | 3.63 | 4.01 | 3.72 | 3.94 | 3.96 | 4.10 |
Use interlibrary loan/article delivery | 3.02 | 3.12 | 3.35 | 3.61 | 3.22 | 3.60 |
Get information from non-library sources | 3.18 | 3.16 | 3.25 | 3.26 | 3.34 | 3.36 |
Keep current | 3.97 | 4.06 | 3.70 | 3.76 | 4.03 | 4.00 |
Be a more productive researcher | 4.19 | 4.36 | 3.93 | 4.23 | 4.21 | 4.39 |
Be a more effective instructor | 3.94 | 3.74 | 3.86 |
Table 7. Importance of selected resource types by academic area 1998 and 2001. % marking 5 on a scale of 1 (not important) to 5 (very important). | ||||||
---|---|---|---|---|---|---|
Academic group and area | Books | Print journals>1980 | Electronic journals | |||
1998 | 2001 | 1998 | 2001 | 1998 | 2001 | |
Faculty Health Sciences | 31.8 | 25.0 | 84.4 | 75.6 | 41.3 | 64.6 |
Faculty Humanities-Social Sciences | 84.0 | 78.9 | 82.7 | 74.6 | 26.7 | 35.6 |
Faculty Science-Engineering | 58.3 | 49.3 | 86.8 | 75.9 | 35.0 | 58.6 |
ALL FACULTY | 53.7 | 44.9 | 84.8 | 75.2 | 35.9 | 56.6 |
Grad Health Sciences | 31.3 | 30.8 | 88.3 | 73.1 | 52.3 | 70.5 |
Grad Humanities-Social Sciences | 68.0 | 60.4 | 73.5 | 67.0 | 27.5 | 45.2 |
Grad Science-Engineering | 48.1 | 36.4 | 88.4 | 68.8 | 47.3 | 65.9 |
ALL GRAD STUDENTS | 52.5 | 44.6 | 81.7 | 69.3 | 40.0 | 58.8 |
Results also showed a positive correlation between importance of resource types and satisfaction with the library's provision of those formats.
When asked to choose from a list of steps the Libraries could take to deal with insufficient funding for journal price increases, only one option (canceling print journals and subscribing to electronic only) received majority agreement. However, while science, engineering, and health sciences faculty favored that step, faculty in the humanities and social sciences opposed it.
Table 8. Steps Libraries can take to make up for inadequate journals funding. % marking disagree (1 or 2) or agree (4 or 5) on a scale of 1 (strongly disagree) to 5 (strongly agree) | ||||||
---|---|---|---|---|---|---|
FACULTY | Health Sciences | Humanities-Soc Sci | Science-Engng | |||
Agree | Disagree | Agree | Disagree | Agree | Disagree | |
Cancel lesser used titles | 52.0 | 29.3 | 35.7 | 38.2 | 45.6 | 25.8 |
Cancel high cost titles with large price hikes | 34.5 | 33.3 | 46.1 | 28.0 | 45.6 | 25.1 |
Cancel high cost titles, get articles on demand | 43.1 | 32.5 | 51.9 | 25.4 | 45.6 | 29.2 |
Cancel print get electronic only | 63.2 | 24.0 | 32.1 | 47.7 | 59.7 | 25.8 |
Buy fewer books | 33.6 | 33.6 | 6.4 | 82.2 | 14.8 | 62.7 |
License fewer databases | 29.9 | 40.6 | 20.8 | 53.5 | 22.3 | 52.2 |
When asked to choose from a list of actions that they could take to reduce the cost of scholarly communication, the only area of uniform agreement lay in publishing in scholarly society publications. However, science and engineering faculty indicated greater agreement with publishing in lower-cost journals and refusing to serve on the editorial boards of high-cost journals than did faculty in other areas.
Table 9. Steps faculty can take to reduce the cost of scholarly communication % marking agree (4 or 5 ) or disagree (1 or 2) on a scale of 1 (strongly disagree) to 5 (strongly agree) | ||||||
---|---|---|---|---|---|---|
FACULTY | Health Sciences | Humanities-Soc Sci | Science-Engng | |||
Agree | Disagree | Agree | Disagree | Agree | Disagree | |
Publish in lower-cost journals | 22.7 | 46.3 | 41.4 | 38.6 | 51.6 | 25.9 |
Retain copyright | 30.6 | 29.1 | 58.0 | 13.9 | 44.4 | 17.8 |
Publish in alternative peer-reviewed e-journal | 22.9 | 45.8 | 27.0 | 45.2 | 29.3 | 42.0 |
Publish in scholarly society publication | 61.5 | 11.3 | 72.6 | 7.6 | 79.0 | 6.7 |
Don't serve on editorial boards of high cost titles | 25.9 | 39.0 | 38.3 | 35.3 | 44.2 | 26.8 |
Table 10. Top priorities by academic area 1998 and 2001. % identifying as top priority | |||||||
---|---|---|---|---|---|---|---|
Academic group and area | Deliver full-text to desktop | Maintain print collection quality | Preserve library materials | E-access to older journals | |||
1998 | 2001 | 1998 | 2001 | 1998 | 2001 | 2001 | |
Faculty Health Sciences | 71.5 | 86.2 | 63.3 | 45.7 | 33.7 | 32.1 | 64.8 |
Faculty Humanities-Social Sci. | 43.5 | 48.2 | 76.1 | 78.9 | 49.5 | 50.2 | 38.6 |
Faculty Science-Engineering | 59.1 | 72.5 | 72.1 | 60.3 | 44.1 | 42.5 | 69.4 |
ALL FACULTY | 60.4 | 73.4 | 69.6 | 57.4 | 40.0 | 39.4 | 59.6 |
Grad Health Sciences | 68.0 | 85.3 | 43.0 | 42.9 | 25.0 | 23.1 | 58.3 |
Grad Humanities-Social Sci. | 47.5 | 59.6 | 53.5 | 59.6 | 37.0 | 41.3 | 55.2 |
Grad Science-Engineering | 57.4 | 78.6 | 58.9 | 54.3 | 41.9 | 36.4 | 73.4 |
ALL GRAD STUDENTS | 56.0 | 72.7 | 52.1 | 53.2 | 35.0 | 34.6 | 61.8 |
However, once again there is significant variation by subject area. Maintaining the quality of the print collection is the overwhelming priority of faculty in humanities and social science disciplines and was little changed from 1998 results. However, in science, engineering and health sciences full text was the overwhelming choice while maintaining the quality of the print collection decreased as a priority. This was especially true in the health sciences where the gap widened significantly for both questions. We find a similar response among graduate students in sciences and health sciences, although the percentage identifying maintaining quality of print collections was unchanged from 1998. However, students in the humanities and social sciences gave equal priority to desktop delivery, maintaining quality of the print collections, and providing electronic access to older journals equal in priority. A new category was added in 2001 on providing electronic access to older journals (it replaced a question dealing with delivery of bibliographic databases through the web) - this was a high priority for those in sciences and health sciences.
Differences between faculty in science and engineering and those in humanities and social sciences are significant in all areas except preservation.
Appendix 1. Composition of broad subject groups | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Health Sciences | Humanities-Social Sciences | Science-Engineering | |||||||||
Faculty | Grad | Faculty | Grad | Faculty | Grad | ||||||
1998 | 2001 | 1998 | 2001 | 1998 | 2001 | 1998 | 2001 | 1998 | 2001 | 1998 | 2001 |
672 | 641 | 128 | 157 | 367 | 305 | 200 | 231 | 408 | 354 | 129 | 174 |
|
|
|
Appendix 2. Science-Engineering faculty and graduate students by academic area | ||||
---|---|---|---|---|
Faculty and grad respondents | Faculty 1998 | Faculty 2001 | Grad 1998 | Grad 2001 |
Arts & Sciences: Sciences | 241 (59%) | 198 (56%) | 56 (43%) | 81 (47%) |
Engineering | 77 (19%) | 76 (21%) | 46 (36%) | 49 (28%) |
Forest Resources | 28 (7%) | 28 (8%) | 7 (5%) | 17 (10%) |
Ocean-Fishery Sciences | 53 (13%) | 42 (12%) | 15 (12%) | 19 (11%) |
Bioengineering | 9 (2%) | 10 (3%) | 5 (4%) | 6 (3%) |
TOTAL | 408 | 354 | 129 | 172 |
________. 1988. User surveys. Systems and Procedures Exchange Center (SPEC) Kit 148. Washington, D.C.: Association of Research Libraries.
________. 1984. User studies in ARL libraries. Systems and Procedures Exchange Center (SPEC) Kit 101. Washington, D.C.: Association of Research Libraries.
________. 1981. User surveys and evaluation of library services. Systems and Procedures Exchange Center (SPEC) Kit 71. Washington, D.C.: Association of Research Libraries.
Berger, K. and Hines R. 1994. What does the user really want? The library survey project at Duke University. Journal of Academic Librarianship, 20 (5/6):306-309.
Cook, C., Heath, F., Thompson, B., & Thompson, R.L. 2001. The search for new measures: The ARL "LibQUAL+TM" study--a preliminary report. Portals: Libraries and the Academy, 1, 103-112.
Cook, C., Heath F., and Thompson B. 2000. LibQUAL+: One instrument in the new measures toolbox. ARL Newsletter: A Bimonthly Report on Research Library Issues and Actions from ARL, CNI, and SPARC, 212, 4-7.
Hernon, P. and Altman, E. 1998. Assessing service quality: Satisfying the expectations of customers. Chicago: American Library Association.
Hernon, P. and Altman E. 1996. Service quality in academic libraries. Norwood, New Jersey: Ablex.
Hernon, P. and Whitman, J. 2001. Delivering satisfaction and service quality: A customer-based approach for libraries. Chicago: American Library Association.
Hiller, S. 2001. Assessing user needs, satisfaction and library performance at the University of Washington. Library Trends, 49 (4), Spring 2001, 605-625.
Hiller, S. and Self, J. 2001. A Decade of User Surveys: Utilizing a Standard Assessment Tool to Measure Library Performance. Paper presented at the 4th Northumbria International Conference on Performance Measures in Libraries and Information Services, Pittsburgh, Pennsylvania, August 2001. Conference Proceedings to be published in 2002 by the Association of Research Libraries.
Nitecki, D. and Franklin B. 1999. Perspectives on new measures for research libraries. Journal of Academic Librarianship, 25 (6), 484-487.
Talbot, D., Lowell G., and Martin, K. 1998. From the user's perspective - the UCSD libraries user survey project. Journal of Academic Librarianship, 24 (5), 357-364.
Van House, N., Weil N., and McClure, C. 1990. Measuring academic library performance: A practical approach. Chicago: American Library Association.