Classics

Developing a Measure of Library Goodness

A Review of:

Orr, R. H. (1973). Measuring the goodness of library services: A general framework for considering quantitative measures. Journal of Documentation, 29(3), 315-332.

Reviewed by:

Gregory A. Crawford
Interim Director

Penn State Harrisburg School of Humanities
Middletown, Pennsylvania, United States of America
Email: gac2@psu.edu

Received: 9 Feb. 2016 Accepted: 20 June 2016

2016 Crawford. This is an Open Access article distributed under the terms of the Creative Commons-Attribution-Noncommercial-Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To discuss the theoretical design of a measure of library quality and value that could be used across functional areas of a library in order to justify and maximize the allocation of resources.

Design – This theoretical article provides background on how to conceptualize and develop a quantitative measure of library goodness.

Setting – The process delineated is applicable to any library, whether public, academic, or special.

Subjects – The intended audience is library management, both at the director and the department head levels.

Methods – The author provided examples and questions in the development of appropriate variables.

Main Results – The author presented a discussion of potential variables. These variables include library capability and utilization.

Conclusion – The article concluded with a discussion of the major desiderata for an effective measure of library goodness: appropriateness, informativeness, validity, reproducibility, comparability, and practicality.

Commentary

Although many think of assessment of libraries as a recent phenomenon, assessment of libraries has been on the minds of library administrators for many years. The chief problem was and still remains how to measure library effectiveness or “goodness”, as Orr called it. Although some of his discussion is naturally dated, Orr developed a theoretical framework for measures of library goodness for directors or department heads to utilize for assessments of the effectiveness of their own libraries.

Orr’s chief goal in this article was to provide a method of developing quantitative variables so that libraries can achieve better services at an acceptable cost. Specifically, these variables could serve as management tools in the tasks of justification (obtaining resources needed to meet specific goals), allocation (allocating these resources effectively), and maximization (using resources to maximal advantage). In his concept of “goodness”, Orr argued that two questions must be answered: “How good is the service?” and “How much good does it do?” (p. 317). He referred to the first question as “quality”, meaning how well the service meets the needs of the user. He called the second measure “value”, which can be judged by the beneficial effects that result from its use. For Orr, the goal of the library, in simple terms, was to maximize the quality of its services and the value that it provided to its community in relation to the resources provided.

Orr proposed four criterion variables that can be measured and used as surrogates for measuring quality and value directly: resources, capability, utilization, and beneficial effects. He asserted the following basic relationships between these variables: as resources increase, the capability of a service increases; as the capability increases, the total uses (utilization) will increase depending also on demand; as the utilization increases, the beneficial effects will increase; and as the beneficial effects increase, the resources will increase, thereby closing the feedback loop. While resources and use are generally the easiest variables to measure, capability (quality) and beneficial effects (value) are more difficult to measure directly.

One of the major problems in measuring goodness, according to Orr, is determining user needs. Users are often not aware of their true needs, making it difficult to use the direct measure of needs as a basis for measuring library goodness. As he noted, it is only possible to record needs that have been recognized and acted upon. Similarly, the capability of the library in meeting needs of the user is complicated by many factors including the limitation that the perception of library patrons affects their use of library resources. If a user perceives the library as having the capability of answering a need, the service will more likely be used.

In the final section of the article, Orr proposed five desiderata for measures of library goodness: appropriateness, informativeness, validity, reproducibility, comparability, and practicality. For appropriateness, he listed several questions that must be answered, including: “Is the measure of the proper type?”, “Are the units and scale suitable?”, and “Are the operations required to implement the measure compatible with the library’s procedures, physical layout, etc.?” (p. 329). By informativeness, Orr meant a measure that helps suggest sources of operational problems or possible solutions. Validity indicates that the measure is free from bias and that the variable truly measures what it is intended to measure. Reproducibility can also be called reliability and indicates that standard techniques were used in order to reduce error. Comparability seeks to answer the question “To what extent will results be affected by factors one would wish to control?”, so that changes can be monitored. Finally, practicality is of great importance since the overall purpose of the measure is to provide data for use in managing the library.

Conclusion

Although this article is now over 40 years old and the services provided by libraries have changed greatly, Orr still provides helpful insights into developing serviceable measures of library quality and value which would assist to any director who must justify the budget of the library and the allocation of its resources. Libraries continue to excel at measuring inputs such as budgets and the size of collections and simple output measures such as circulation and the number of reference questions. These measures, however, do not assist managers in providing a true picture of the importance of libraries to the lives of our users that will resonate with our funders. Any current measure must take into account newer services such as the provision of electronic resources (databases, e-journals, e-books, datasets, etc.), new references services such as chat, and new delivery mechanisms. But, most importantly, Orr does not take into consideration the satisfaction of users with the services and resources provided. This must be incorporated into any discussion of the effectiveness and value of libraries. Thus, the quest for a truly meaningful measure library goodness continues.