85 Usage-Based Collection Evaluation with a Curricular Focus Karen C. Kohn Karen C. Kohn is Collection Development Manager in Landman Library at Arcadia University; e-mail: kohnk@arcadia.edu. © 2013 Karen C. Kohn, Attribution-NonCommercial (http://creativecommons.org/ licenses/by-nc/3.0/) CC BY-NC Systematic evaluation of a library’s collection can be a useful tool for collection development. After reviewing three evaluation methods and their usefulness for our small academic library, I undertook a usage- based evaluation, focusing on narrow segments of our collection that served specific undergraduate courses. For each section, I collected data on the number of books owned, number of checkouts in the past four years, and number of unique books used. Using examples from the data, I discuss possible ways to interpret and act on the data. I also note how the knowledge gained from this evaluation fits into the larger toolkit of librarian competencies for collection development. oing collection development well involves a variety of in- formation: about the library’s mission, the needs of users, strengths and weaknesses of the current collection, and tools for iden- tifying quality resources. An evaluation of a library’s collection can be a useful piece of the collection development toolkit, as it helps the library understand whether the collection is adequate and whether, or in what specific areas, it needs to grow. After reviewing several methods for collection evaluation and considering both their feasibility and the assumptions behind them, I selected usage-based evaluation as the most appropriate method for Ar- cadia University (in Glenside, PA), and I evaluated selected segments of our library collection based on how heavily they were used. The sections I evaluated were chosen to correspond to specific courses taught at the university. This was not a comprehensive evaluation of the library collection but rather a chance to gather information related to particular sections, namely, the print collection that serves undergraduate courses. My goal was to determine through quantitative measures how these materials were being used and to use the results to direct purchasing and weeding. In this article, I discuss how data fit into the larger picture of librarians’ role in collection development and review ex- amples from library literature of the ways in which librarians at other institutions have used data to influence weeding or purchasing. I then explain how I assigned call number ranges to individual courses, namely all the 200- and 300-level courses offered by Arcadia University. This rep- resents the middle levels of Arcadia’s undergraduate curriculum, which ranges from 100-level (introductory) to 400-level (usually restricted to seniors). I collected the following data on each call number range: how many books we own, how crl-295 86 College & Research Libraries January 2013 many checkouts occurred in the past four years, and how many unique books were used. From these figures, I calculated the percentage of books used and the num- ber of uses per book. I summarize below our questions and conclusions regarding ways to interpret the data and the action steps we have agreed to take in response to this evaluation project. The evaluation was particularly im- portant for Landman Library because we have only in the last six years shifted responsibility for collection development from faculty to librarians. Initially the librarians’ role was primarily adminis- trative. Being new to collection develop- ment, most librarians initially limited their role to forwarding catalogs or re- views to faculty members and providing budget updates. In the interest of building our own collection development skills, the librarians have discussed the compe- tencies the task requires, one of which is knowledge of the collection. Possessing data that we can use to inform purchase decisions increases our competence and therefore contributes to librarians’ tak- ing ownership of the tasks of collection development. Confidence that our deci- sions reflect students’ needs enables us to approach the faculty as equals with our own distinct knowledge to contribute to collection development. Librarian and Faculty Roles in Collection Development In an article comparing librarian and faculty approaches to collection develop- ment, Felix Chu1 comments that faculty at his institution don’t expect librarians to be knowledgeable about their particular subjects and view librarians mainly as helping with procedures and keeping track of funds. As recently as two or three years ago, librarians at Landman Library at Arcadia University might have agreed with Chu’s faculty about our role. We, like many librarians at smaller institutions, are often liaisons to departments whose dis- ciplines we have never studied. Further- more, until six years ago, faculty members at Arcadia were responsible for all collec- tion development, so some faculty still do not expect us to be active selectors. This attitude has been transformed in recent years, among both librarians and faculty, as the librarians have become aware of how much more is involved in collection development than knowledge of the dis- cipline and have gained expertise that we can share with faculty. Certainly faculty can, and should, contribute to collection development. As Robert Neville, James Williams III, and Caroline C. Hunt say, “the expertise of the … faculty is vital to the health of the collection.”2 There are several ways faculty can share their knowledge with librarians. They can provide us with a list of significant authors in their field3 or with names of their professional organiza- tions.4 At Arcadia, faculty members have shared many kinds of useful information. In addition to the names of key authors and professional associations, they have provided information on what textbooks they use, so that library purchases can complement rather than duplicate class- room materials. They sometimes tell us about how they teach: for example, one professor believes in teaching methodol- ogy by analyzing existing research rather than using methodology textbooks. They can also provide insight into ideological stances within their field. For instance, a tactful negative comment from a faculty member about a film I recommended about disability was eye-opening for me and helped me to recommend more sensitive materials thereafter. Though knowledge of key authors, teaching meth- ods, and ideological nuance is extremely helpful for selection, it is not, however, sufficient for doing collection develop- ment well. A variety of other kinds of information are needed, most of which are more likely to be found in a librarian’s toolkit than a professor’s. Some of the tools for collection de- velopment are ones that librarians can acquire; others are inherent in our role as librarians. One tool that can be acquired Usage-Based Collection Evaluation with a Curricular Focus 87 without in-depth subject knowledge is a familiarity with sources of reviews.5 A competency that is inherent in many li- brarians’ roles is an awareness of students’ needs.6 If we are involved in reference or instruction, then our regular interactions with students give us a perspective that is complementary to faculty’s. Faculty know what they are assigning and what students turn in, but we know what stu- dents start out looking for before modify- ing their topic to use the materials they’ve been able to find. Also inherent in the li- brarian’s position is a broader perspective on the library and university than faculty members may have. Librarians’ lack of personal stake in collecting on specific topics7 and loyalty to the “community as a whole”8 enables us to balance the needs of different library users. In addi- tion, librarians should be aware, though faculty might not be, of the library’s mis- sion and budget situation, both of which significantly inform collection develop- ment. Knowledge of the big picture, in terms of what is being published, what is needed, and the library’s mission, is key to collection development. Another significant tool that librar- ians can use in collection development is knowledge of what is already in the collection and how it compares to our col- lection goals. This knowledge sometimes comes intuitively9 but can be solidified through a collection evaluation. There are a variety of methods that librarians have used to evaluate their collections, which I will review below. After reviewing other librarians’ methods, I will describe why I chose to do a usage-based evaluation of my own library’s collection. I will discuss below what my colleagues and I felt the data told us and how that has affected our collection development activities. Collection Evaluation Methods There is a wide range of ways one can evaluate a library collection. The three main ways described in the literature are quantitative, qualitative, and usage- based. These three methods measure re- spectively whether a library has “enough” books, the “right” books, or books that are being used. Each method contains its own assumptions about the library’s collection goals. Though the present study is only usage-based, there is no need to select only one method of evaluation. Multiple methods could be combined to evaluate the collection from different angles that might correlate with different aspects of a library’s mission. At one time there was a standard method for doing a quantitative evalu- ation, which was called a conspectus. There were two such tools developed in the 1980s and 1990s, one by Research Libraries Group (RLG) and the other by the Western Library Network (WLN).10 The WLN Conspectus was an online tool that provided a protocol for collecting data related to the size of the collection and recency of books. Though the con- spectus also suggested looking at usage data and comparing one’s collection to core lists, its primary role was as a quantitative analysis tool.11 It provided some hard numbers on what constitutes a sufficient collection—10 percent of the collection should come from the last 10 years—and a numerical measure for clas- sifying a library’s collection, ranging from “1a (supports minimal inquiries)” to “5 (highly specialized, comprehensive).”12 Librarians would need to decide for each subject area whether they were aiming for a level 5 collection or something less am- bitious. The subject areas designated by the conspectus system were “broad sub- ject classifications based on the Library of Congress classification system.”13 Both the RLG and WLN Conspectus systems, which were offered online, have now been superseded by the WorldCat Collection Analysis tool. However, the WorldCat Collection Analysis tool does not provide a uniform system of rating collections. Its purpose is for libraries to evaluate them- selves for internal purposes by comparing themselves with other libraries.14 A few variations on the conspectus method appear in the literature. Bodi 88 College & Research Libraries January 2013 and Maier-O’Shea15 used data from an analysis comparing their library to other academic libraries in Illinois. Howard White16 devised a method of quantitative evaluation using WorldCat’s FirstSearch interface to see whether his library’s hold- ings were mainly books held by many other libraries or books held by few. This method could identify areas of special- ization in one’s library. Unfortunately, the WorldCat interface has changed just enough that White’s method is no longer possible. I opted not to use the conspectus meth- od, or any variation of it, as it seems to be designed for larger libraries than ours. This method ranks libraries from 1 (mini- mal) to 5 (comprehensive). Sample data from White’s17 article put the University of California at Berkeley at level 3, which is considered the instructional level. As Berkeley has 70 times the number of print materials we do and 43 times our materials budget18 and only scored a 3, it seemed obvious that Landman Library’s rank would be 1 (minimal). It did not seem particularly useful to rate ourselves on a scale that spans every academic library, as we would undoubtedly score low, and this score would say nothing about whether we were serving our own users’ needs. Comparison to a selected peer group could have been useful, but we do not have an easily identifiable peer group for comparison, as our university is larger than the others in our primary consortium. An additional reason not to pursue the conspectus method was that the conspectus categories would not necessarily match our curriculum, and it was important to me that our evaluation focus on the needs of our own institution. Even libraries that find quantitative evaluation useful may want to supple- ment with a qualitative evaluation, as the conspectus method does not reveal whether the library has the right books, only how many it has. William Aguilar19 describes two qualitative methods that involve comparing one’s holdings to lists of desired books. He does not use qualita- tive in the standard social sciences sense of observation or interviews. Rather, he is referring to a method for “evaluat[ing] the quality of a collection,”20 rather than simply its size. Standard lists are men- tioned often in the literature,21 though only Aguilar22 mentions the caveat that one should make sure not to use the same lists as evaluation tools that were used as selection tools. Doing so would skew the results, as the library’s selectors would presumably have ordered a large percent- age of the books on the list. Reliance on this method of evaluation could limit a li- brary’s selection tools, as one would have to deliberately not use the core lists for se- lection. Another method, which follows a similar process but is more closely related to patron need, is looking at bibliogra- phies of student papers to see how much of the material they cite was obtained, or could have been obtained, in their own library.23 This would provide some indication of whether or not the library was meeting students’ needs, though it is of course possible that students altered their research based on what they could easily obtain. This method also looks at whether the library has the “right” books but defines “right” to mean “desired by students for their coursework.” I initially wanted to do a qualitative evaluation, but after an attempt to use Choice Outstanding Titles lists as an evalu- ation tool, I encountered the problem of which Aguilar24 warns, namely that the library used Choice as a selection tool and wanted to continue to do so. For certain sections of Choice’s list, our col- lection looked very successful, but this assessment was misleading as we’d used Choice’s recommendations throughout the year in making purchase decisions. Fur- thermore, the subject categories in Choice did not correspond well with our curricu- lum and much of the list was irrelevant. I did not pursue student bibliographies as an evaluation tool, though I would like to do so later. The third method of evaluation, usage- based, is not concerned with comparison Usage-Based Collection Evaluation with a Curricular Focus 89 to other libraries, absolute size, or any external measures of the right books. In- stead, usage-based evaluation is focused on demand, as indicated by usage. Both circulation and interlibrary loan statistics can be considered usage statistics, and these are the two components of usage- based evaluations that appear in the literature. Different libraries have used slightly different variations on these data. John Ochola25 broke down his collection by LC subclassification (that is, the range indicated by the first two letters of the call number) and compared what percentage of the collection each subclass comprised with what percentage of circulation it accounted for. Jennifer Knievel, Heather Wicht, and Lynn Silipigni Connaway26 used the RLG Conspectus categories to divide their collection into twenty-five sections. They compared the number of holdings in each section with the number of items circulated in that section to see what percentage was used. They also looked at the average transactions per item. Arianne Hartsell-Gundy, Masha Misco, and Jeffrey Hartsell-Gundy27 aimed to see how closely their collection followed the 80/20 rule. This rule is a descriptive principle that comes from the business world. As applied to libraries, it predicts that 20 percent of the titles in a collection will account for 80 percent of all circulation.28 Looking at five areas that corresponded to the five concentra- tions within their English department, Hartsell-Gundy et al.29 calculated what percentage of the collection in each area accounted for 80 percent of each collec- tion’s total circulation. Another piece of data commonly used in usage-based evaluations is InterLibrary Loan data. While circulation data reveal the ways in which the library is meet- ing patrons’ needs, InterLibrary Loan data complement this by showing what else users need that their library doesn’t have.30 Taken alone, Aguilar31 points out, low usage of a collection doesn’t indicate whether there is low demand for a par- ticular subject or whether the library has the wrong books. If usage is low and ILL is relatively high, however, it is clear that the library’s holdings are somehow off-base. After reviewing the various evaluation methods in the literature, I decided to start with a usage-based evaluation. As we are a relatively small institution, with just over 3,000 FTE, any comparison to the larger universe of academic libraries would simply tell us that our collection is small, which we already know. Compar- ing our holdings to core lists did not work, for reasons described above. Variations on the quantitative methods (that is, selecting our own comparison group) and a quali- tative study using student bibliographies rather than core lists are possibilities to consider for later. Choosing only one method for the time being still satisfied the goal of improving the librarians’ competencies for collection development by providing information we could use to direct purchasing and weeding. An advantage of using no external measures of our collection’s success (such as comparison to peer groups or selection lists) was the ability to create my own subsections of the collection as units of evaluation. These sections were call num- ber ranges that related to specific courses. The narrow ranges allowed us to look at the collection at a very granular level and in a way that tied closely to our own curriculum. This approach was especially useful for us since some of our depart- ments are small, and the courses offered might not span an entire LC subclass. The granularity of the study will help us to target our future purchases appropri- ately. In addition, using courses as a unit of evaluation also made it easier for us to seek faculty insight to combine with our numerical measures, since we could ap- proach the professor of each course with information pertinent to him or her. Tying a collection evaluation to specific courses is a distinctive approach, as other studies with similar methodologies used larger, more standardized units of evalu- ation.32 Knievel, Wicht, and Connaway, who used categories based on the RLG 90 College & Research Libraries January 2013 Conspectus, say that “[w]ith only twenty- five subject categories, it is not sufficient to use the conspectus to evaluate a collec- tion in detail. However, it is very useful to provide a broad picture of a collection.”33 The present study, in contrast, is a pilot project that introduces a method for look- ing more closely at selected segments of the collection. Methodology L a n d m a n L i b r a r y ’s C o l l e c t i o n D e - velopment Plan states a mission of “maintain[ing] a collection that supports students and the university community, balancing a traditional core with a usage- based collection.”34 This mission would indicate that the most appropriate meth- ods for evaluating our own collection are usage-based and comparison to core lists. This evaluation project focused on circula- tion data as a tool for collection evalua- tion. Though InterLibrary Loan data was available to us, the number of books our patrons borrowed from other institutions was much less than the amount borrowed from our own library, and there were not enough data to lead us to any action steps. Therefore, this article will only report on the circulation data. While recognizing that our collection contains a variety of formats—print books, e-books, print journals, online journals, and audiovisuals—the present evaluation project looked only at print books. This was mainly to keep the scope of the project manageable, as my choice to look at narrow segments of the collection resulted in a large number of segments to evaluate. Including electronic resources would have introduced the question of whether checkouts of print books are comparable to (potentially very brief) full-text views of an e-book. Though the present study looked at only one format, we do regularly collect and review usage data on e-books, online journals, and print journals and use these data for collection development. As this evaluation aimed to measure how well our print collection is serving our own institution, I tied the evaluation closely to the curriculum. Rather than dealing with LC subclasses, as has been done by others,35 I looked at individual courses and the call number ranges most relevant to that course. I began with 200-level courses and then added the 300s. I felt that these would involve more library research than 100-level, or introductory, courses. I did not look at 400-level courses, since these are primar- ily senior thesis courses, which include varied topics and do not easily map to call number ranges. Using Arcadia University’s course catalog, I first generated a list of all 200- and 300-level courses. There were 240 200-level and 233 300-level courses. I assigned each course a call number range or ranges, with the following excep- tions. Twenty 200-level courses and nine 300-level courses were ignored because the call number range would be the same as for another course. For example, if Photography I was assigned a range, then Photography II was not. Certain kinds of courses were presumed not to need books: conversational language, fieldwork, a series of courses called “Psy- chology Journal Club” that are devoted to reading journal articles, studio art or theater, physical activity, and intern- ships. I ignored “special topics” courses if sample topics were not included in the catalog description; and I also ignored independent studies, as topics are too varied and are not listed in the catalog. This left 199 200-level and 181 300-level, or 380 courses total. The call number ranges I assigned varied from very broad to very specific. I allowed myself to assign multiple call numbers to the same course. For example, Criminal Justice 208, Great Trials in His- tory, received multiple narrow ranges cov- ering the Nuremberg Trials, Salem Witch Trials, and the trial of Socrates. (These ranges are shown in table 1 below.) At the opposite end of the spectrum was Biology 205, Human Anatomy (also shown in table 1), for which the entire LC subclass of QM Usage-Based Collection Evaluation with a Curricular Focus 91 could be relevant. I was not concerned that some ranges were broader than others, because I was not evaluating our collection by the absolute number of books we have for each course. The evaluation focused instead on ratios of books borrowed to books held within a particular section, so a small section was at no disadvantage when compared to a large section. Having assigned call number ranges to 380 courses, I then collected data on the number of books held by our library in each range. I found much of my in- spiration from Knievel et al.36 Knievel and her colleagues also looked at aver- age checkouts per book and percentage of items borrowed. My evaluation was more targeted and, I believe, unique, in that the ranges I looked at were, in most cases, much narrower LC subclasses or conspectus categories. I had the benefit of several reports that our IT department had written using SQL, a common programming language for querying a database. The reports allowed me to type in a call number range and to select the desired dates and item locations. I included all circulating books and excluded reference books and audiovisual materials. I also excluded e-books, because, although they have catalog records, their usage is not marked in the catalog. As noted above, the library reviews e-book usage separately. The number I used was for items held as of the date I began data collection: December 15, 2010. Though it took several months to collect all the data, I always used this date for consistency. The numbers I collected were: number of items in range, number of checkouts in range in the last four years (including renewals, but not in-house uses), number of unique items checked out in the last four years. From this I de- rived two ratios: percentage of collection used (unique items checked out divided by items held) and usage per book (total number of checkouts divided by number of unique items checked out). I calculated an additional ratio that did not turn out to be useful, which was checkout ratio (total number of checkouts, which could include multiple checkouts of the same book, divided by items held). I used four years as my range because in this period of time almost all the courses should have run at least once. There was one piece of data that I thought would be very useful but could not get, and that was the average age at which a book stops circulating. Millson- Martula37 notes a few studies that calcu- late at what point in a book’s life circula- tion drops, but he does not explain how this is calculated. Arcadia University’s In- stitutional Researcher, whom I consulted, had some thoughts on how to calculate this, but I was not able to find a way to measure this on a large scale. Data Of the 380 courses included in this study, there are 35 for which Landman Library owns no books. No further statistics were collected for these courses, although this information was brought to the attention of the librarians. Of the remaining 345 courses, the average number of books held for each course was 175.17 ( as shown in table 1 below). Courses where books were checked out had an average of 34.35 percent of books being used (also shown in table 1). The numbers varied significantly by discipline. For math courses the average percentage of books used was 19.91 per- cent, whereas for art the average was 45.98 percent. These are an interesting pair of disciplines to compare because neither has been weeded heavily in recent years, and failure to weed would be an obvious cause for having a lot of unused titles in a section. In Education, where there has been recent weeding, the average percent- age of books used is 30.08 percent, falling between the percentages for math and art. The average checkouts per circulated book (in other words, total checkouts di- vided by total number of books checked out) was 2.5. Nine courses had books that were used an average of four or more times. 92 College & Research Libraries January 2013 Data on a selection of individual courses are included in table 1 below. Discussion The goal of this evaluation was not to as- sess whether Landman Library was meet- ing any particular target for the percent- age of books used, but rather to enable our collection development activities to become better informed. After collecting all the data described above, I provided each librarian with a spreadsheet of data relating to his or her departments and convened a meeting to discuss action steps. As mentioned above, the possible action steps for us were to weed or buy more books. There was an additional option of talking to faculty if we felt we needed more information about the needs of a particular course. After our meeting, each librarian developed his or her own guidelines for deciding which actions to pursue. I will present the common themes to our decisions, using example courses as illustrations. (All courses used as ex- amples are included in table 1.) As noted above, what constituted high circulation varied by department. For example, the departments to which I am a liaison are Education and Sociology/ Anthropology. For Education, the aver- age percentage of books borrowed was 30.08 percent, whereas for Sociology/ Anthropology it was 34 percent. For these departments, if more than 30 percent of the books for a specific course were used, I made a note to collect more books on that topic. An example of this would be Anthropology 285, Aesthetics, listed in the table below. We only have one book on this topic, and it has been used three times. I plan to order more books for this course. The question came up during our dis- cussions that it is possible the checked-out books were used for a course other than the one we were evaluating. There is no way to tell if this is the case, though it is quite likely to happen sometimes. It is not clear, however, that our actions would be any different if we found that, for example, our one book on aesthetics had been used for something other than the aesthetics course. The book is serving a need of our users, so we should continue to buy books on this topic and can do so knowing that they are relevant to our curriculum. Despite not knowing for sure whether the borrowed books were used for the course we had in mind, Arcadia librar- ians all felt that, in general, high circula- tion implied that we should continue to collect in this area. The literature shows other libraries implementing the same conclusion in a variety of ways. Several libraries ensure higher purchasing levels by allocating more funds to the areas with high circulation.38 Ochola39 adjusted approval plans to bring in more books on subjects that had high circulation. Knievel et al.40 use higher-than-expected circulation data to justify buying books in areas where faculty believe there is no need. They quote Chuck Hamaker, say- ing, “both computer science and math faculty say don’t buy books, we need journals; usage patterns say something quite different to the selectors.”41 When librarians see that books on a particular subject are being used, the consensus in the literature seems to be that the library should use whatever means makes sense to ensure the purchase of more books on this subject. For now, I do not plan to make any changes to the ways money is allocated among departments. As mentioned above, faculty at Arcadia University had sole responsibility for collection devel- opment until 2005; as a result, most of them think of the money that the library allocates to their department as “their” money and pay close attention to how much they are “given.” Making sig- nificant changes would be controversial. Instead, each department’s allocation will remain the same, but we will most likely make changes to the ways we spend that allocation. We do not have approval plans, so librarians generate lists of books to order and can do so with specific courses in mind. Usage-Based Collection Evaluation with a Curricular Focus 93 D ep ar tm en t C ou rs e N um be r C ou rs e N am e C al l N um be r R an ge (s ) To ta l B oo ks O w ne d To ta l C he ck ou ts B oo ks B or ro w ed Pe rc en ta ge Ti tle s B or ro w ed C he ck ou ts Pe r B oo k A ve ra ge 17 5. 17 11 0. 12 43 .6 5 34 .3 5 2. 5 A nt hr op ol og y 28 5 A es th et ic s N 72 .A 56 1 3 1 10 0% 3 B io lo gy 20 5 H um an A na to m y Q M 10 8 82 34 31 .4 8% 2. 41 C he m is tr y 20 1 O rg an ic C he m is tr y I: S tr uc tu re , M ec ha ni sm s an d R ea ct io ns Q D 24 1- 44 1 45 8 41 18 3. 93 % 2. 28 C he m is tr y 21 2 C on ce pt ua l Ph ys ic s II Q C 17 4. 2- 17 4. 24 , Q C 23 1, Q C 50 1- 76 6 10 8 2 1 0. 93 % 2 C ri m in al J us tic e 20 8 G re at T ri al s in H is to ry K F8 91 0- 90 66 , F 74 . S1 , E 45 1, B 31 0- 31 8, Q B 36 .G 2, K Z 11 75 - 11 79 , K F2 24 .S 3, K FM 24 78 .8 .W 5, T R 14 0. M 34 7, K D 37 2. W 53 18 3 74 27 14 .7 5% 2. 74 E du ca tio n 21 1 A ss es sm en t a nd In te rv en tio n in In fa nc y an d To dd le rh oo d L B 11 37 , L B 11 24 - 11 32 38 4 3 7. 89 % 1. 33 E ng lis h 21 2 W ri tin g Po et ry an d Fi ct io n PN 11 01 12 1 1 8. 33 % 1 Ps yc ho lo gy 20 5 A du lt Ps yc ho pa th ol og y R C 51 1- 56 9. 5. V 53 2 91 9 28 8 54 .1 4% 3. 19 94 College & Research Libraries January 2013 When usage is low, the preferred re- sponse of librarians at other institutions seems to depend on the existence of re- mote storage and the philosophy of the library. Many librarians move materials with low circulation to remote storage.42 Libraries that do not have off-site storage or that have less commitment to keeping historical materials can weed materials that have not been used.43 Of the literature reviewed here, only Hartsell-Gundy et al.44 explicitly stated that they do not see lack of usage as indicating that an item should be removed from the collection. Having no definite action step resulting from low circulation, they focused their analysis more on the ranges where they saw high circulation. The librarians at Arcadia University were in agreement on what to do with unused materials. Unlike Hartsell-Gundy et al.,45 the librarians at Landman Library do believe in removing from the collection those books that have not circulated for a number of years. How far back in the book’s history we should look varies by discipline, and we retain classic books re- gardless of use. We do not have the option of remote storage; but, as our mission is to build a “usage-based collection,”46 re- taining unused materials, even remotely, would not make sense for us anyway. An example of a section in which a very low percentage of books were used would be Criminal Justice 208, Great Trials in History (listed in table 1 above). Relevant books for this course may be in several different areas of the library, as the subject matter ranges from the trial of Socrates to the Nuremberg Trials. We have a total of 183 books for the course, of which 27 (14.75%) were used during the four years for which data were collected. I will likely remove from the collection the books from this section that have not circulated in some arbitrary number of years, such as ten. My colleagues have begun weeding sections with low circulation as well. We were less clear on what low usage implied about future purchases. In very few cases were we comfortable conclud- ing that we should simply stop buying books for a particular course. It is pos- sible that a course whose books had low usage does not have assignments that require library research, or it could mean that the research is done using mainly journal articles. Another possibility is that the books we own somehow miss the mark, and students want different books. For some subjects, we also have access to collections of e-books, whose usage could result in less use of print books. A closer look at the data, combined with our knowledge of the rest of the library’s holdings and of the departments’ overall library habits, provided some direction but did not always answer our questions. Returning to the example of Criminal Justice 208, Great Trials in History (see table 1 above), we can see how the data suggest additional purchasing, despite low overall usage. Although 14.75 percent of the books for this course were used, these books were checked out an average of 2.74 times per book, a bit above the average of 2.5 checkouts per circulated book. This means we have a small portion of the collection circulating heavily and should probably purchase more books like the ones that are being used at the same time that we are weeding the un- used books. This requires a further step of identifying what is different about the books being used from the ones not be- ing used. Looking at a report by title or talking to the professor could be useful. For courses where few books were used and that have a low number of checkouts per circulated book, the deci- sion at first seems simpler: we should purchase less, or only sparingly. The librarians at Arcadia University felt that this might not be the only valid conclu- sion, however. Minimal purchasing might make sense for Chemistry 212, Conceptual Physics, for which only 1 of the 108 books we own (0.93%) circulated. This book was used twice, yielding a low ratio of checkouts per circulated book: 2 compared to the average ratio of 2.5. This course is a lab science and likely does Usage-Based Collection Evaluation with a Curricular Focus 95 probably want to weed some despite the high percentage of books used. English 212, Writing Poetry and Fiction, illustrates the opposite scenario. For this course, one of the twelve available books was used (8.33%), yet weeding the eleven unused books seems less than urgent. After re- viewing our data, we all seemed to feel that raw numbers did matter when decid- ing what to weed, especially when the number of books owned for a particular course was high compared to the average books owned per course. Though courses with few books will, in most cases, be exempted from weeding, we were divided on whether we should buy more books for these courses. Initially I felt that a low absolute number of books for a course did not necessarily imply we needed more books. This was particularly true if the few books we owned had very low usage, as with the twelve books for English 212 mentioned above. Low absolute numbers concerned some other librarians, though, and we all felt it would not hurt to ask the faculty whether they thought we needed more. In addition, we all thought that percentages were less meaningful when based on a low absolute number, so the courses with low numbers might need their data supplemented with more impressionistic input from faculty. One last outcome of the evaluation, though not an immediate action step, will likely be demonstrating to faculty who believe they do not need books that there is reason to purchase after all. For example, 18 books were checked out related to Organic Chemistry, in the call number range QD241-441. Although this was only 3.93 percent of the total number of books (458) owned in that range, it is also evidence against spending that de- partment’s entire budget on journals and leaving no money for book purchases. Having data on hand will help us to stand firmer when telling faculty we do not agree with their request to purchase only journals for their department. We expect the conversations with faculty that result from this evaluation not involve writing papers other than lab reports. It seems reasonable to guess that this course just doesn’t need books. This would not be a reasonable assump- tion for Education 211, Assessment and Intervention in Infancy and Toddlerhood, however. For this course, 3 of 38 (7.89%) of books are used. These three books were used a total of four times, yielding a ratio of 1.33 checkouts per circulated book. It is not clear from the course description what kinds of assignments this course has; but, as the Education department in general uses books, there is no reason to guess that this course does not. A conversation with the professor might be the best next step in this case. We found we were very rarely comfort- able deciding to ignore a course when it came to book purchasing. Talking to the faculty about the course’s assignments was the action step most of us preferred in situations where data alone suggested low demand for books. Faculty can tell us if the course in fact has no library as- signment or if there is something wrong with the books in our collection that is not apparent to a nonexpert in the subject. As Bodi and Maier-O’Shea47 say, conversa- tions provide a kind of qualitative data that can fill in gaps where numbers do not tell the full story. After reaching a loose consensus on what actions to take in response to cases of low and high usage, we needed to decide which courses required the most urgent action. A question the librarians had when deciding which courses to target was to what extent we should consider raw numbers. Although I had initially focused on ratios, raw numbers became relevant when we began identifying sections to weed. Psychology 205, Adult Psychopa- thology, illustrates the point. As the chart shows, 54.14 percent of the books for this course were used, well above the average of 34.35 percent. Yet there are 532 books in this section, considerably more than the average of 175.17 books per course. This means there is a lot of shelf space being taken up by unused books, and we 96 College & Research Libraries January 2013 to have a different tenor from most con- versations we’ve had in the past. Rather than taking a tone of asking or offering (for instance, asking, “Would you like to buy any books from this list?”), we will be approaching them with information and suggestions. Initial responses from faculty suggest that they appreciate the input we are giving them based on the evaluation. This project should therefore improve our relationship with faculty. More important, it is my hope and expectation that we are improving our service to students as well. Conclusion This collection evaluation generated a wealth of data that librarians at Arcadia University’s Landman Library have already begun using to make decisions about purchasing and weeding. We hope to bring some of the information we’ve gathered to faculty in the coming months and to ask for their additional perspec- tive on their needs and on our collection. It is thus expected that this project will contribute to better purchasing decisions, increased confidence in our authority to make such decisions, and a shifting sense among faculty about librarians’ role in collection development. A future step that we have discussed is creating collection development plans for each department. This would comple- ment the present study, whose narrow focus allowed us to look very closely at parts of our collection, with a more holistic view of how we are serving each department. We recognize that each de- partment is served by multiple kinds of materials in various formats: electronic journals, printed books, e-books, and DVDs or streaming video. The depart- mental plans would note the size of the collection in each format, along with some broad usage data, and would also include input from faculty on what publishers or authors they like and how our collection can support their teaching strategies. The plans would embody our philosophy of combining faculty’s expertise with librarians’ knowledge of the collection, as enhanced by the current project. Notes 1. Felix T. Chu, “Librarian-Faculty Relations in Collection Development,” Journal of Academic Librarianship 23 (Jan. 1997): 15–20. 2. Robert Neville, James Williams III, and Carolyn C. Hunt, “Faculty-Library Teamwork in Book Ordering,” College & Research Libraries 59 (Nov. 1998): 527. 3. Ibid., 524–33. 4. Jim Agee, “Collection Evaluation: A Foundation for Collection Development,” Collection Building 24, no. 3 (2005): 92–95. 5. T.H. Connell, “Comparing the Circulation of Library Materials Ordered by Faculty and Librarians,” Collection Management 14, no. 1/2 (1991): 73–84. 6. Connell, “Comparing the Circulation of Library Materials”; Chu, “Librarian-Faculty Relations,” 15–20; Sonia Bodi and Katie Maier-O’Shea, “The Library of Babel: Making Sense of Collection Management in a Postmodern World,” Journal of Academic Librarianship 31 (Mar. 2005): 143–50. 7. Chu, “Librarian-Faculty Relations,” 15–20; Neville, Williams, and Hunt, “Librarian-Faculty Teamwork,” 523–32. 8. Debbi Dinkins, “Circulation as Assessment: Collection Development Policies Evaluated in Terms of Circulation at a Small Academic Library,” College & Research Libraries 64 (Jan. 2003): 47. 9. Bodi and Maier-O’Shea, “The Library of Babel,” 143–50. 10. Howard D. White, “Better Than Brief Tests: Coverage Power Tests of Collection Strength,” College & Research Libraries 69 (Mar. 2008): 155–74. 11. Janet G. Webster, “Oregon State University Libraries’ Collection Assessment Project,” paper presented at the IAMSLIC 2000: Tides of Technology Conference, Fort Pierce, FL, 2001. 12. Ibid. 13. Jennifer. E. Knievel, Heather Wicht, and Lynn Silipigni Connaway, “Use of Circulation Statistics and Interlibrary Loan Data in Collection Management,” College & Research Libraries 67 Usage-Based Collection Evaluation with a Curricular Focus 97 (Jan. 2006): 35–49. 14. White, “Better Than Brief Tests,” 155–74. 15. Bodi and Maier-O’Shea, “The Library of Babel,” 143–50. 16. White, “Better Than Brief Tests,” 155–74. 17. Ibid. 18. “Library Comparison Report,” National Center for Education Statistics, available online at http://nces.ed.gov/surveys/libraries/compare/index.asp [accessed 31 May 2011]. 19. William Aguilar, “The Application of Relative Use and Interlibrary Demand in Collection Development,” Collection Management 8 (Spring 1986): 15–24. 20. Aguilar, “Relative Use and Interlibrary Demand.” 21. Aguilar, “Relative Use and Interlibrary Demand”; Webster, “Oregon State University Li- braries”; Bodi and Maier-O’Shea, “The Library of Babel,” 143–50; Agee, “Collection Evaluation” 92–95. 22. Aguilar, “Relative Use and Interlibrary Demand,” 15–24. 23. Agee, “Collection Evaluation” 92–95; Aguilar, “Relative Use and Interlibrary Demand,” 15–24. 24. Aguilar, “Relative Use and Interlibrary Demand,” 15–24. 25. John N. Ochola, “Use of Circulation Statistics and Interlibrary Loan Data in Collection Management,” Collection Management 27, no. 1 (2002): 1–13. 26. Knievel, Wicht, and Connaway, “Use of Circulation Statistics,” 35–49. 27 Arianne Hartsell-Gundy, Masha Misco, and Jeffrey Hartsell-Gundy, “Humanities by the Numbers: Evaluating Usage Data of Collection Areas,” paper presented at ACRL2011, Philadel- phia, PA, April 2011. 28. Richard L. Trueswell, “Some Behavioral Patterns of Library Users: The 80/20 Rule,” Wilson Library Bulletin 43, no. 5 (1969): 458–61. 29. Hartsell-Gundy, Misco, and Hartsell-Gundy, “Humanities by the Numbers.” 30. Jeffrey M. Mortimore, “Access-Informed Collection Development and the Academic Library: Using Holdings, Circulation, and Ill Data to Develop Prescient Collections,” Collection Management 30, no. 3 (2005): 21–37. 31. Aguilar, “Relative Use and Interlibrary Demand,” 15–24. 32. Ochola, “Use of Circulation Statistics,” 1–13; Knievel, Wicht, and Connaway, “Use of Circulation Statistics,” 35–49. 33. Knievel, Wicht, and Connaway, “Use of Circulation Statistics,” 35–49. 34. Karen Kohn, “Collection Development Plan,” (internal policy document, Landman Library, Arcadia University, [Glenside] PA, 2010). 35. Ochola, “Use of Circulation Statistics,” 1–13. 36. Knievel, Wicht, and Connaway, “Use of Circulation Statistics,” 35–49. 37. C. Millson-Martula, “The Effectiveness of Book Selection Agents in a Small Academic Library,” College and Research Libraries 46 (Nov. 1985): 504–10. 38 Hartsell-Gundy, Misco, and Hartsell-Gundy, “Humanities by the Numbers”; D.P. Carrigan, “Data-Guided Collection Development: A Promise Unfulfilled,” College & Research Libraries 57 (Sept. 1996): 429–37; Webster, “Oregon State University Libraries.” 39. Ochola, “Use of Circulation Statistics,” 1–13. 40. Knievel, Wicht, and Connaway, “Use of Circulation Statistics,” 35–49. 41. Ibid., 36. 42. Ochola, “Use of Circulation Statistics,” 1–13; Knievel, Wicht, and Connaway, “Use of Circulation Statistics,” 35–49; Carrigan, “Data-Guided Collection Development,” 429–37. 43. Webster, “Oregon State University Libraries”; Ochola, “Use of Circulation Statistics,” 1–13. 44. Hartsell-Gundy, Misco, and Hartsell-Gundy, “Humanities by the Numbers.” 45. Hartsell-Gundy, Misco, and Hartsell-Gundy, “Humanities by the Numbers.” 46. Kohn, “Collection Development Plan.” 47. Bodi and Maier-O’Shea, “The Library of Babel,” 143–50.