| URLs in this document have been updated. Links enclosed in {curly brackets} have been changed. If a replacement link was located, the new URL was added and the link is active; if a new site could not be identified, the broken link was removed. |
![[Board accepted]](http://www.istl.org/graphics/b2.gif){kind=small}
Assessing Reference: Using the Wisconsin-Ohio Reference Evaluation Program in an Academic Science Library
|
Amy Paster Head, Life Sciences Library alp4@psu.edu
|
Kathy Fescemyer Life Sciences Librarian kaf12@psu.edu
|
|
Nancy Henry Health Sciences Librarian nih1@psu.edu
|
Janet Hughes Biological Sciences Librarian jah19@psu.edu
|
| Helen Smith Agricultural Sciences Librarian hfs1@psu.edu
| |
| Life Sciences Library Penn State University Park, PA | |
Abstract
The Pennsylvania State University Life Sciences Library conducted the Wisconsin-Ohio Reference Evaluation Program survey in October 2005. The results of the survey indicated that the desk personnel do an average job in fulfilling the library user's expectations. Known-item questions were handled well but non-known questions need additional training to ensure patrons are not receiving too much information. This project raised a number of questions concerning user satisfaction and library type. It is hoped that this article can serve as the baseline for future research in the evaluation of science reference question transactions and patron satisfaction.
Introduction
Assessment in libraries is currently a topic of much research and discussion within the profession. The Association of Research Libraries has developed the LibQUAL+tm program and according to their home page (accessed April 13, 2006), it is used to solicit, track, understand, and act upon users' opinions of service quality. However the instrument used is quite broad in its definition of service and does not specifically relate to reference desk transactions.
Richardson (2002) defines quality reference service as user satisfaction, which is driven by library staff behavior. However this leaves out the accuracy and usability of the answer, which is another measure that is often considered important. The Wisconsin-Ohio Reference Evaluation Program (WOREP) has been used to measure reference performance in a number of general academic reference departments and academic music libraries, as well as public libraries. "WOREP is designed to assess the outcome of the reference transaction and to identify factors related to success or lack of success" (WOREP home page 2006). WOREP collects data from both the patron and the staff involved in the reference transaction, and evaluates transaction success as well as patron satisfaction. According to early aggregate WOREP data, factors which affect success at general reference desks in academic libraries include: staffing; time and effort spent on the question; effective communication; type and source of the question; and collection size (Bunge 1991).
The utilization of the WOREP instrument in the academic music library setting (Christensen et. al 2001) resulted in some debate as to whether the instrument’s validity and reliability translated effectively into a more specialized, complex reference transaction environment (Lasocki 2002). However the instrument’s validity in the general reference environment has been established (Murfin & Gugelchuk 1987). The decision to use this general reference survey instrument in the Life Sciences Library assessment process at Penn State also resulted in discussions regarding the effective translation of the instrument into a complex science reference transaction environment.
Throughout the literature, there are few articles addressing the uniqueness of reference desk service in a science library. Moysa (2004) discusses evaluating reference behavior, but not the success of the transaction, and although the article describes the situation in a sci/tech library, the instrument developed is general in nature, focusing on approachability, interest, and positive attitudes. Robbins and Daniels (2001) reported on academic health science libraries using the SERVPERF instrument to measure the perceived quality of reference desk service and compare quality across libraries. However this instrument does not measure the success of the transaction, does not measure factors which might affect patron perceptions of success, and the low rate of return affected the possible comparative usefulness.
Is reference success in sci/tech libraries affected by the same factors as general academic libraries? Over the years WOREP has been available, other sci/tech libraries have used the survey (Radcliff 2006); however the data have not been published, and the libraries have not been aggregated as "sci/tech libraries" for comparative purposes. This article is the baseline for future research in the evaluation of science reference question transactions and patron satisfaction.
Survey Procedures
The University Libraries at the Pennsylvania State University is ranked 14th among 113 North American research libraries (ARL Statistics 2004). This includes 36 libraries at 24 locations throughout Pennsylvania, with the largest campus being at University Park. Overall, the Libraries' cataloged collection now approaches five million items. Approximately 100,000 volumes are added annually to the collection, and over 58,000 serials are received. The Life Sciences Library (LSL) is one of the subject-specific libraries at the University Park campus, and although it is housed in the main library building, it is a separate entity with its own reference desk, reference collection, current periodicals, and unique collections. The LSL collections are approximately 300,000 volumes and cover all aspects of agriculture, biology, biochemistry, nutrition, nursing, and public health. The LSL is a primary collection for the students, faculty and staff in the College of Agricultural Sciences, the College of Health and Human Development, and the Eberly College of Science. At the time of the study, the personnel staffing the Life Sciences Library reference desk consisted of five professional librarians, three full-time staff which were classified as "library assistant" for the purposes of this study, and five part-time staff (four of which were students) which were all classified as "student/other" for the purposes of this study.
The WOREP survey is a two-part form that is filled out by both the patron asking the question, and the library staff answering the question. The patron form asks for general demographic information on the patron (status, major), the satisfaction the patron had with the transaction, whether patrons received the information they needed, and any reasons for dissatisfaction. The library staff form asks for information about the type of question (known item vs. non-known item, subject area), number and type of sources used, question difficulty, and factors affecting the transaction (patron in a hurry, books off the shelf, etc.). Sample forms may be found at the {WOREP home page}.
To prepare all LSL personnel for the survey, the WOREP guidelines were discussed extensively at staff meetings. All questions on the survey were reviewed to ensure that personnel defined criteria in the same way. Because WOREP is designed only to evaluate reference transactions, the differences between a directional transaction and a reference transaction were explicitly defined. In addition, all personnel were asked to complete five sample surveys to become comfortable with the process before the actual survey was initiated. Surveys were distributed between October 24 and November 16, 2005.
In order to minimize increased workload on the reference personnel, only the first two reference (not directional) transactions in each hour, where the patron agreed to participate, were used for the study. All patrons were eligible for the survey whether they were PSU faculty, staff, students, or the general public. Patrons were asked if they would be willing to take a survey to help assess how the personnel at the Reference Desk performed. After securing agreement and ensuring the patron was over 18 years of age (for legal reasons) reference staff handed the patron a survey, a copy of the implied consent form, and a pencil. After the survey was filled out it was placed in the survey box which was housed in an area removed from the Reference Desk. The reference staff member answering the question would fill out their half of the survey form and then place it into the survey box. Following the completion of the minimum 100 surveys, they were returned to the WOREP office at Kent State University to be analyzed.
Results
One hundred and two forms with usable results (out of the 104 handed out) were collected. This is a 98.1% rate of return. The results show how the LSL patron profiles, query profiles, satisfaction profiles compared with other academic libraries who also participated in WOREP surveys. At the time the LSL data were processed, there were 117 participating academic libraries, and 41 of these were classified as "comparable academic" because they were of comparable collection size to the LSL. The results showed actual numbers and percentages for the LSL, and then showed percentages for the comparable academic libraries, the academic library that had the best results, and the average of all participating academic libraries. Results primarily focus on the "non-known item" transactions, in which the patron is requesting "anything or something," in contrast to those "known item" transactions where a particular item is requested. Some of the following data were not explicitly provided by the WOREP report, but were easily calculated from the information provided.
Patron Profiles
In contrast to comparable academic libraries, the LSL surveyed more freshman, fewer sophomores, more juniors and fewer seniors. When combined, the LSL had a lower percentage of freshmen/sophomores asking questions than the comparable or all academic libraries, but a higher percentage of juniors/seniors. Graduate students were more highly represented than in comparable academic libraries, but less than the average for all academic libraries. Faculty, staff and alumni were more highly represented than in comparable and all libraries (Table 1). As would be expected in a life sciences subject-specific library, more surveys were completed by those in the medicine, health, agricultural and biological sciences disciplines (Table 2), a total of 67.1%.
| TABLE 1: Patrons by Status | |||
|---|---|---|---|
| STATUS | LSL | Comparable Academic | All Academic |
| Freshmen | 20.8% | 18.7% | 16.5% |
| Sophomores | 10.9% | 19.1% | 15.3% |
| Juniors | 24.8% | 15.6% | 17.6% |
| Seniors | 16.8% | 19.6% | 21.5% |
| Graduates | 12.9% | 10.9% | 15.1% |
| Continuing Educ. / Non-degree | 0.0% | 3.7% | 2.2% |
| Faculty | 5.9% | 3.0% | 3.2% |
| Staff | 4.0% | 1.9% | 2.1% |
| Alumni | 3.0% | 1.8% | 1.8% |
| Non affiliated | 1.0% | 7.0% | 5.9% |
| TABLE 2: Patrons by Major | |||
|---|---|---|---|
| MAJOR | LSL | Comparable Academic | All Academic |
| Agricultural / Biological Sciences | 38.2% | 4.2% | 4.8% |
| Medicine / Health | 28.9% | 8.3% | 8.6% |
| Technical / Engineering | 9.2% | 6.6% | 7.0% |
| Other Social Sciences | 7.9% | 10.2% | 13.0% |
| Arts & Humanities | 5.3% | 14.6% | 19.1% |
| Education | 3.9% | 14.2% | 11.5% |
| Business / Management | 2.6% | 23.3% | 19.2% |
| Undeclared | 1.3% | 7.5% | 6.3% |
| Interdisciplinary/ Other | 1.3% | 3.7% | 4.8% |
| Math / Physical Sciences | 1.3% | 3.3% | 2.8% |
| Law | 0.0% | 4.6% | 3.3% |
Query Profiles
The LSL survey respondents had more known-item questions than the comparable, or all, academic libraries (Table 3). Other than the "specific book" category, more questions fall into the need "something/anything" category than any other type of question (Table 4), and this is a greater percentage than in comparable or all academic libraries. Online tools such as the OPAC, indexes, full-text articles and online reference tools and web sites comprised over 80% of the sources used to answer the queries (Table 5). The LSL staff also tended to use multiple sources to answer a question more often than comparable or all libraries in the study (Table 6), and were more likely to spend more time on non-known item reference transactions (Table 7). The LSL had a higher percentage of "student/other" answering questions than comparable or all academic libraries (Table 8).
| TABLE 3: Query Type - known item (particular text or specific resource) versus non-known item (something or anything on a topic). | |||
|---|---|---|---|
| QUERY TYPE | LSL | Comparable Academic | All Academic |
| Known Item | 26.5% | 12.0% | 15.7% |
| Non-known Item | 73.5% | 88.0%% | 84.3% |
| TABLE 4: Types of questions asked. | |||
|---|---|---|---|
| TYPE OF QUESTION | LSL | Comparable Academic | All Academic |
| Specific book, serial, score, recording etc. | 26.9% | 12.0% | 15.4% |
| Something, Anything | 18.3% | 16.3% | 13.7% |
| Certain type of source or materials | 9.6% | 11.5% | 11.6% |
| Smaller item in larger publication | 7.7% | 2.5% | 2.6% |
| Short answer facts | 5.8% | 9.0% | 9.5% |
| Facts and statistics | 5.8% | 11.3% | 10.7% |
| Focus on aspect: Biog, Hist, other | 4.8% | 6.9% | 6.1% |
| Explanation of particular source or library | 2.9% | 8.9% | 9.5% |
| Analysis, trends, pro/con, cause/effect. Etc | 2.9% | 5.4% | 4.9% |
| Anything by particular author | 1.9% | 0.9% | 0.9% |
| Criticism, Reviews, Interpretations | 0.00 | 4.9% | 5.6% |
| TABLE 5: Frequency of sources used in LSL for non-known item queries | |
|---|---|
| SOURCE | Frequency used (%) |
| Library Catalog | 25.95 |
| Research Databases | 21.62 |
| Full-text Articles (print or online) | 12.97 |
| Online Reference | 9.73 |
| External Free Web | 8.65 |
| Print Reference | 6.49 |
| Own Knowledge | 5.41 |
| Library-Made Web Sites/Guides | 4.86 |
| Print Indexes | 1.62 |
| Consult | 1.08 |
| Refer | 1.08 |
| Circulating Books (print or online) | 0.54 |
| Vertical File | 0.00 |
| Other | 0.00 |
| TABLE 6: Number of sources used in LSL for non-known item queries | |||
|---|---|---|---|
| # SOURCES USED/RECOMMENDED | LSL | Comparable Academic | All Academic |
| 1 source | 21.6% | 35.2% | 40.9% |
| 2 sources | 32.4% | 30.1% | 29.8% |
| 3 sources | 28.4% | 17.6% | 16.3% |
| 4 sources | 13.5% | 8.7% | 6.6% |
| 5 sources | 4.1% | 8.4% | 6.5% |
| TABLE 7: Time spent on question for non-known item queries | |||
|---|---|---|---|
| TIME SPENT | LSL | Comparable Academic | All Academic |
| 0-3 minutes | 9.5% | 32.7% | 33.9% |
| 3-5 minutes | 41.9% | 42.2% | 40.9% |
| 5-15 minutes | 37.8% | 21.6% | 21.0% |
| over 15 minutes | 10.8% | 4.0% | 4.6% |
| TABLE 8: Staff types answering non-known item questions | |||
|---|---|---|---|
| Staff type | LSL | Comparable Academic | All Academic |
| Librarian | 46.7% | 73.5% | 71.1% |
| Library Assistant | 25.3% | 14.3% | 16.7% |
| Student/Other | 28.0% | 7.6% | 7.2% |
| Staff not specified | 0.0% | 4.7% | 5.0% |
Patron Satisfaction
The patron's satisfaction is determined in WOREP to be "found exactly" only when they met the three conditions of marking that they found "just what I wanted," they answered "yes" to the satisfied question, and that they did not mark any reason for dissatisfaction. In the LSL, patrons were highly satisfied when a known-item question was asked and slightly more satisfied than comparable academic and all academic libraries on the non-known item transactions (Table 9). In general the LSL met the average for satisfaction when a librarian answered the question, surpassed the average when a full-time staff answered the question, and were less satisfied than average when a student/other staff member answered the question (Table 10). LSL Patrons were more satisfied than average on "facts & statistics," "analysis," "specific aspect," "specific material" types of questions, about average on "explanation" and "something/anything" type questions, and less satisfied than comparable or all academic libraries on "short answers" type questions (Table 11). When identifying problems with the transaction, the top four issues were "too much information," "library employees not knowledgeable," "couldn't find information in source," and "staff employees only partly understood," however very few or no problems were reported in other areas such as "not enough information," "found nothing," "staff didn't spend enough time," and "staff not courteous" (Table 12). When the patron and the LSL staff perception of success were matched, they agreed less often than comparable or all libraries (Table 13). Although the LSL did not instruct patrons on multiple new sources as often as other libraries, more patrons reported learning one new source (Table 14) and, as with other libraries, patrons reported significantly often that they learned something new about the library (Table 15).
| TABLE 9: Patron found exactly what was wanted and was satisfied, by query type | |||
|---|---|---|---|
| % "found exactly" and satisfied | |||
| QUERY TYPE | LSL | Comparable Academic | All Academic |
| Known Item | 92.6 | 56.6 | 59.5 |
| Non-known Item | 61.3 | 57.0 | 58.8 |
| TABLE 10: Patron found exactly what was wanted and was satisfied, for non-known item queries, by staff type | |||
|---|---|---|---|
| % "found exactly" and satisfied | |||
| STAFF TYPE | LSL | Comparable Academic | All Academic |
| Librarian | 60.0 | 58.1 | 60.1 |
| Library Assistant | 78.9 | 62.0 | --- |
| Student/Other | 50.0 | 54.8 | 60.3 |
| Staff not reported | --- | 50.2 | 52.7 |
| TABLE 11: Patrons who found exactly what was wanted and were satisfied, by type of question. | |||
|---|---|---|---|
| % "found exactly" and satisfied | |||
| QUESTION TYPE | LSL | Comparable Academic | All Academic |
| Facts & Statistics | 100 | 57.3 | 53.8 |
| Analysis, trends, pro/con, cause/effect, etc | 100 | 50.3 | 49.3 |
| Focus on specific aspect: Biog, Hist, Other | 80 | 50.8 | 55.7 |
| Certain type of source or materials | 78.6 | 55.8 | 58.8 |
| Explanation of particular source or library | 66.7 | 62.7 | 66.1 |
| Something, anything | 50 | 50.5 | 53.4 |
| Short Answer Facts | 33.3 | 62.9 | 66.4 |
| Criticism, Reviews | -- | 56.0 | 55.0 |
| TABLE 12: Most frequent patron reported problems, for non-known item queries. | |||
|---|---|---|---|
| % reported problems | |||
| Patron reported problems | LSL | Comparable Academic | All Academic |
| Too much information found | 14.7 | 1.5 | 1.5 |
| Staff only partly, or not at all, knowledgeable | 13.3 | 8.9 | 9.1 |
| Couldn't find information in source | 9.3 | 4.5 | 4.2 |
| Staff only partly gave enough, or did not give enough at all, explanation or help | 6.7 | 7.3 | 7.6 |
| Staff only partly, or did not at all, understand query | 6.7 | 5.7 | 5.2 |
| Need more in-depth information | 5.3 | 10.3 | 8.5 |
| Want more simple information | 5.3 | 2.9 | 2.9 |
| Information not relevant enough | 4.0 | 4.1 | 4.2 |
| Need information with different viewpoint | 2.7 | 1.9 | 2.1 |
| Not Enough information | 1.3 | 13.0 | 11.7 |
| Staff explanations were only partly, or not at all, clear | 1.3 | 6.0 | 6.1 |
| Found Nothing | 1.3 | 3.2 | 2.9 |
| Staff only partly, or did not at all, spend enough time | 0.0 | 4.3 | 5.0 |
| Staff were only partly, or not at all, courteous | 0.0 | 2.0 | 2.0 |
| Not sure if information is correct | 0.0 | 1.5 | 1.5 |
| TABLE 13: Percent agreement between staff and patrons on perception of success | |||
|---|---|---|---|
| Success perception | LSL | Comparable Academic | All Academic |
| Staff and patron agree on success | 60.0 | 65.4 | 65.2 |
| TABLE 14: Percent of patrons reporting learning new sources as result of transaction (non-known item queries). | |||
|---|---|---|---|
| # of new sources learned | LSL | Comparable Academic | All Academic |
| 2 or more new sources | 29.7 | 37.4 | 35.5 |
| 1 new source | 54.1 | 47.2 | 48.7 |
| no new sources | 18.2 | 15.3 | 15.7 |
| TABLE 15: Percent of patrons reporting learning something about the library as a result of transaction (non-known item queries). | |||
|---|---|---|---|
| Learned something? | LSL | Comparable Academic | All Academic |
| Yes | 85.3 | 78.7 | 78.2 |
| Yes and Partly | 94.7 | 93.6 | 93.4 |
| No | 5.3 | 6.4 | 6.6 |
Discussion
The data provided by the survey are not sufficient to allow statistically significant comparisons. The report from WOREP warns that samples are usually not large enough to make valid distinctions based on a few percentage points of difference. One of the main distinctions in the Life Sciences Library is that it is a specialized subject library within a large university library structure. This may be relevant when comparing the demographics of the status of the patrons served, the subjects they represent, the type of queries (known item or non-known item), the sources used, and the time spent on the question. The "comparable academic" libraries were comparable in collection size to the specific LSL collection of 300,000 volumes, not comparable to the entire Penn State holdings of almost five million volumes. This puts the LSL library with "small" academic libraries, not the "large" universities. This may affect some of the data as library staff used the entire Penn State collection to answer questions. The high standard set by WOREP as "quality" reference service is a success rating of at least 60% in the "found exactly" and "satisfied" measures, a measure the LSL reached. Other studies claim that medium-sized libraries generally score better in this area, and small libraries worse (Stalker & Murfin 1996). What is unknown is the benchmark of a small (by WOREP standards) subject library, affiliated with a large university, especially in the age where electronic information has decreased location disadvantages.The higher percentages of upper level patrons may indicate that the LSL has more demanding clientele with more difficult questions but this may be similar to other science and technology libraries, or even indeed to other subject-specific libraries in a University setting. The majors most represented by the comparable academic libraries are business/management, arts and humanities, education and other social sciences. What is unknown is if these libraries were subject-specific libraries within a larger library, or were general libraries serving all subjects.
Like the LSL, music libraries had a higher percentage of known item queries than general libraries (Christensen et. al 2001). Although the reasons for this have not been determined from the data, perhaps this is common in all specialized libraries. Another area that may be common across science and technology libraries is the fact that there was a higher percentage of the "something, anything" type of question than in comparable or all libraries, and conversely a lower percentages in more general types of questions such as "short answer facts" and "facts & statistics," as well as a logically lower percentage in some questions more germane to humanities libraries such as "criticism, reviews, interpretations," and "focus on aspect: biog, hist, other."
The tendency of the LSL to use multiple sources, and spend more time on questions, might be a positive indication that there are more sources available to answer any question, or a negative indication that the LSL does not have sources completely suitable to that specific information need of the patrons. However, the strength of the collection at Penn State is indicated in several areas. First, for the known item questions at the LSL, an extremely high percentage "found exactly" what was wanted, and the result was better than at all other participating libraries. According to Christensen et.al (2001), larger libraries generally score higher on this measure. The strength of the collection is indicated in another area as well. In the report of problems by patrons, the LSL had a lower percentage of problems in "found nothing" and the "not enough information" categories than both comparable and all academic libraries.
The LSL distributed the reference load more evenly among librarians, assistants and students, and has students covering the desk at a much higher percentage of time than the comparable and all other libraries. The data indicate that the LSL patrons were more satisfied when working with library assistants than with librarians and less satisfied when served by students/others. This situation was similar to comparable academic libraries, but the LSL library assistants scored appreciably higher than assistants in comparable academic libraries and the students lower. This may be reflective of several long-term staff members with significant experience, and a need to train the students in better customer service and subject expertise.
For patrons' reports of problems most percentages or actual numbers were quite low, and in several areas were less than both comparable and all academic libraries. The most frequent single issue was the report of getting "too much information." This falls into the category of the information received (as opposed to the service), and it could be related to the time spent on queries, the type of query, the number of sources used, the staffing level, whether the patron was in a hurry, or even the patron's perception of the amount of information available. The category of "want more simple information" was also a problem a greater percentage of the time than for comparable or all academic libraries, and could be a similar, or related, issue. This may be a reflection of librarians' tendencies towards trying to instruct the patrons (in support of the instructional mission of the PSU University Libraries) instead of simply giving the patrons the requested information. The instructional inclination is evident in that patrons reported learning new sources, and/or something about the library, in a large majority of the transactions.
The two most frequent complaints under the service category were reports that the staff appeared only partly or not knowledgeable, or only partly or did not understand the query. Although these two complaints are also problems in comparable and all academic libraries, they are worthy of being addressed. It is unknown if these complaints were related to the staff type, and it may be a reflection of the LSL higher percentage of students serving the desk and their possible lack of the specialized subject knowledge important in a science library. Further training for all staff on specialized databases and subjects may be one way to decrease this type of issue.
Overall, patron and staff perceptions of the success of the transaction were in agreement 60% of the time, and although this was less than comparable or all academic libraries, it does indicate that the LSL perceptions of the reference process were not overly optimistic or overly pessimistic. However, further staff training to ensure that all staff members are able to recognize when communication difficulties occur, might increase this percentage.
Additional correlations would be helpful in identifying more exact issues that should be addressed in the PSU Life Sciences Library. Specifically it will be useful to determine the how various factors such as the type of question, staffing level, time spent on question, and number of sources used, correlate with the specific type of problem (such as "too much information" or "staff not knowledgeable"). And information about how the staff perception of some of the special conditions, such as "patron in a hurry" and "communication difficult," as well as the staff perception of the difficulty of the question, correlates with specific patron reported problems would be informative. General information that might also be useful includes correlations between the type of query (known item or non-known item) and the type of question, with the patron status or major.
Conclusion
For the immediate Life Sciences Library utilization, the WOREP study determined that the personnel do an average job in fulfilling library user's expectations. An excellent job was done in assisting with known-item questions, but more care should be taken with non-known item questions to ensure that patrons are not overloaded and receive only the amount of information for their needs. Staff should be continually trained on the intricacies of the specialized databases and subjects covered by the LSL. Further correlation between the problems identified, and the various other factors, would need to be done in order to further exploit the data.WOREP shows great promise in comparative situations. Some advantages are that it is easy to administer, inexpensive to use, and the return rate is relatively high (Bunge 1991). Its primary use is in looking at reference success, but it also gives an indication of the types of reference being done by describing the types of patrons served, the types of questions asked, and the ways the questions are answered. Perhaps it can be determined if reference questions and patrons are different in science and technology libraries. Is our patron success affected by the same factors as those of a general reference department, or even of other subject specialty libraries such as music? Are patrons served by science and technology libraries more or less satisfied than those served by arts and humanities or social science libraries? Or are these factors independent of the subject served, and affected more by the type of patron and the type of question?
To be more relevant to science and technology libraries, it is possible that the WOREP survey instrument should be altered to reflect better the situation in those libraries. As was done in the music library situation, indication of the location of the library (branch or within main library) may be a measure of interest. And perhaps a closer look at the specific types, and possibly even the exact titles, of information resources utilized in specialized science libraries could provide richer comparative data and lead to greater knowledge of customer reference service needs in the science library environment. Alteration to increase the subject category specificity might also contribute to more specialized quantitative data collection about science library reference transactions. This benchmarking data could translate into useful information for comparative analysis of science library reference services nationally.
It is hoped that this article will encourage reference assessment in science and technology libraries. In order to truly compare ourselves, we all need to use the same instrument. It is possible that the WOREP survey, with some modification, is the appropriate vehicle for determining the answers to some of these questions.
References
Association of Research Libraries. 2004. ARL Statistics. [Online]. Available: {http://fisher.lib.virginia.edu/arl/} [Accessed April 13, 2006].Bunge, C.A. 1991. Gathering and using patron and librarian perceptions of question-answering success. In: Evaluation of Public Services and Public Services Personnel (ed. by B. Allen), pp.59-83. Champaign, Ill.: University of Illinois Graduate School of Library and Information Science.
Christensen, B.E., Du Mont, M.J. & Green, A. 2001. Taking note: assessing the performance of reference service in academic music libraries: a progress report. Notes 58(1):39-54.
Lasocki, D. 2002. A critique and response to the use of the instrument in the published study "Taking Note: Assessing the performance of Reference Service in Academic Music Libraries" (Notes, September 2001, pp.39-54). Notes 58(3): 699-703.
LibQUAL Home Page. n.d. [Online]. Available: http://www.libqual.org/ [Accessed April 13, 2006].
Moysa, S. 2004. Evaluation of customer service behaviour at the reference desk in an academic library. Feliciter 50(2):60-63.
Murfin, M. E. and Gugelchuk, G. M. 1987. Development and testing of a reference transaction assessment instrument. College & Research Libraries 48:314-38.
Radcliff, C. 2006. List of WOREP participating libraries. E-mail correspondence April 25, 2006.
Richardson, J. V. 2002. Reference is better than we thought. Library Journal 127(7): 41-42.
Robbins, K. & Daniels, K. 2001. Benchmarking reference desk service in academic health science libraries: a preliminary survey. College & Research Libraries 62(4):348-353.
Stalker, J.C., & Murfin, M.E. 1996. Quality reference service: a preliminary case study. The Journal of Academic Librarianship 22(6): 423-429.
Wisconsin-Ohio Reference Evaluation Program. n.d. [Online]. Available: {http://worep.library.kent.edu/} [Accessed April 13, 2006].
