smith.p65 344 College & Research Libraries September 2003 344 Assessing Collection Usefulness: An Investigation of Library Ownership of the Resources Graduate Students Use Erin T. Smith Erin T. Smith is Cataloging and Systems Librarian in McGill Library at Westminster College; e-mail: smithet@westminster.edu. This research was conducted between March 2001 and July 2002, at which time the author held the position of Science Reference Librarian with the University of Georgia Libraries. A two-phase assessment tool was developed for evaluating the “fit” of the University of Georgia Libraries’ collections with the needs of their patrons. First, a citation analysis of a sample of 2001 theses and disser- tations revealed the specific titles and materials used by a disciplinary cross section of graduate students. Second, searches were conducted in the OPAC and in electronic resources to determine library ownership of those cited works. The process then was repeated with a sample of graduate works completed in 1991 in order to investigate whether the further stretching of library budgets due to the influx of electronic re- sources has had an impact on the usefulness of the overall collection. o date, collection developers have used citation analysis for two types of research. It has been used to determine the most im- portant journals in a discipline as part of a journal collection evaluation. It has also been used to determine the ratio of mono- graphs to serials cited by researchers in a discipline in order to develop a library bud- get allocation formula for that discipline. This study represents the combination and broadening of these two methodolo- gies in order to quantitatively measure the overall health of a collection. Rather than focusing on either the importance of indi- vidual journal titles or the ratio of mono- graphs to serials, this project was designed to assess the usefulness of the University of Georgia Libraries’ collections to an ex- tremely important user group at any re- search institution—graduate students. By looking at the citation patterns in a disciplinary cross section of theses and dissertations written by University of Georgia students in concert with library ownership of the works they cited, this study sought to answer one simple, but absolutely critical, question: Do we own the things our students use? Literature Review Since P. L. K. Gross and E. M. Gross pub- lished their landmark article in 1927, ci- tation analysis has been used to determine the value of individual journal titles to specific disciplines and subdisciplines.1 In fact, this type of study was conducted so frequently that one of the primary appli- cations of citation analysis described in Linda Smith’s seminal article was its use in library decisions as to whether “to ac- quire or not acquire a particular title, to Assessing Collection Usefulness 345 continue or to discontinue a subscription, to weed or not to weed a backset.”2 However, in the late 1970s, as skyrock- eting subscription prices began to consume more and more of acquisitions budgets, citation analysis became a budget alloca- tion tool for determining the ratio of mono- graphs to serials.3 Academic libraries con- cerned about maintaining “balanced” col- lections used bibliographic analysis to de- termine the proportion of materials cited by researchers in a discipline as an indica- tor of the relative importance of various material formats. The ratio of materials cited in the literature then was used as part of a formula for determining the appro- priate ratio of monographs to serials in the acquisitions budget for the library collec- tion in that discipline.4 Harry M. Kriz was one of the first to employ this more holistic application of ci- tation analysis in his 1978 study of the West Virginia University College of Engineer- ing.5 Other researchers soon followed, each examining a distinct discipline or subdis- cipline. By 1990, Robin B. Devin and Martha Kellogg found that “more than 50 such articles have appeared … cover[ing] almost the entire spectrum of subjects from the fine arts to engineering.”6 Disciplinary-specific data were invalu- able to collection developers because, as Joseph J. Kohut said in his article on allo- cating academic book budgets, the appro- priate ratio of monographs to serials var- ies greatly by subject discipline.7 In gen- eral, however, the studies analyzed by Devin and Kellogg provided quantitative evidence of what librarians already sus- pected from experience: The humanities and social sciences tend to be more book dependent, whereas science and technol- ogy fields tend to be more journal depen- dent.8 Robert D. Stueart and George B. Miller found that the “cultures” also differ in the age of materials cited: “the areas of science and technology also tend to be less inter- ested than the humanities in older mate- rial and consequently to feel less need for retrospective purchases.”9 This issue of time span, defined by Rosalind Walcott as “the number of years that a researcher in a field reaches back into the literature to find useful materials,” became particularly im- portant to libraries making decisions about collection weeding and the remote storage of older materials.10 Then, in the late 1990s, the challenging task of allocating appropriate funds to monographs and serials was further com- plicated by the addition of electronic re- sources to academic library budgets. Sur- veys of both faculty and students have shown that the World Wide Web (Web) has had a tremendous impact on the way the academic community conducts research.11 However, bibliometric methods have not been a particularly effective tool for assess- ing the use of electronic resources because, more often than not, bibliographies do not indicate how a cited work was accessed. The notable exception was a study pub- lished by Philip M. Davis and Suzanne A. Cohen in 2001, in which the authors com- pared the bibliographies of undergradu- ate term papers written in 1996 and 1999.12 They found that not only had the Web be- come a significant resource in student bib- liographies in 1999, but also that the stu- dents cited significantly fewer books than they had just three years earlier.13 The Davis–Cohen study is also note- worthy because it examines the research habits of a segment of the academic popu- lation largely untouched by bibliographic analysis—the students. The vast major- ity of bibliographic studies focus on ei- ther faculty citation patterns or citations in the published literature. Undergradu- ate studies are difficult to conduct because most papers are graded and then returned to the students. However, academic li- braries have a wealth of data on gradu- ate student citation patterns in the form of theses and dissertations, a resource that surprisingly few researchers outside the sciences have exploited. Walcott was one of the few to take ad- vantage of this largely untapped data source. In her 1994 citation analysis of the importance of individual journal titles to the biological sciences at Stony Brook Uni- versity, Walcott stated that she chose to 346 College & Research Libraries September 2003 study graduate students because they represented the “largest and most active group of users of the library collection.”14 In 1999, Sherri Edwards chose to investi- gate citation patterns in polymer science and polymer engineering theses and dis- sertations in order to determine that field’s most important material formats and journal titles.15 In addition, a study by Louise S. Zipp indicated that the data derived from re- search on graduate students do more than provide valuable insight into student re- search behavior. Zipp found that citation analyses of theses and dissertations are a reliable “surrogate for faculty publication citations in evaluations of the research portion of library collection use” because the research interests of graduate students often reflect the interests of their faculty advisors.16 Methodology Phase 1: Citation Analysis The goal of this study was two pronged: one, to evaluate the usefulness of the cur- rent library collection as a whole; and two, to investigate whether the usefulness of the collection had changed during the past ten years because of the introduction of electronic resources and the increases in periodical expenditures. A search was conducted in the libraries’ OPAC to iden- tify all theses and dissertations written at the University of Georgia in 1991 and 2001. The theses and dissertations then were grouped into general categories based loosely on the division of subject areas among the libraries’ collection de- velopment staff: arts and humanities, education, sciences, and social sciences. This search resulted in a total of 647 works in the 1991 population: 121 in the arts and humanities, 132 in education, 241 in the sciences, and 153 in the social sci- ences. The total for 2001 was 748 theses and dissertations: 156 in the arts and hu- manities, 121 in education, 324 in the sci- ences, and 146 in the social sciences. A sample of thirty works from each population (60 total) was chosen to keep the data set small enough to complete the study in time for the results to be avail- able for budgeting decisions, but large enough to provide a reasonably accurate snapshot of the usefulness of the collec- tion to patrons and the appropriateness of current budget allocation policies. A stratified sample based on the four broad disciplinary categories was selected randomly from each population. The re- sulting pool included thirteen theses and seventeen dissertations for each year: seven works in the arts and humanities, six in education, ten in the sciences, and seven in the social sciences. Bibliographies from the 2001 theses and dissertations were printed from the university’s Electronic Theses and Disser- tation Library, when available.17 All 1991 bibliographies and those 2001 works not available online were photocopied. Each bibliography then was assigned an iden- tification number, and all personal infor- mation was removed to preserve student confidentiality. If a bibliography had seventy-five or fewer citations, all citations were analyzed; if it had more than that number of citations, only the first seventy-five were included in the study. Unpublished primary resources, such as personal interviews, correspon- dence, and lecture notes, were not included in the analysis. In addition, secondary sources were given preference for inclusion in the study over published primary docu- ments when analyzing history theses and dissertations (for example, primary news- paper and magazine articles were included only if a bibliography listed fewer than sev- enty-five secondary sources). Citations were coded based on the for- mat of the material: monograph, periodi- cal, conference proceeding or paper, news- paper or magazine, theses or dissertation, or Web site. These more precise terms were employed rather than adopting one cat- egory for all serials in accordance with the ALA Glossary of Library and Information Sci- ence.18 If the cited work did not fit any of the six format categories listed above, the designation “other” was assigned. The date of publication for each citation also was re- corded in order to calculate time span. Assessing Collection Usefulness 347 Phase 2: Library Ownership The second phase of the study was to de- termine library ownership of the works cited in the sample of theses and disserta- tions. Print ownership of all 1991 cited works and all 2001 works, with the excep- tion of Web sites, was determined by an OPAC search. Electronic ownership of all serials cited in 2001 works was determined by a search in the University of Georgia’s Electronic Journal Locator (EJL).19 For the purposes of this study, print ownership was defined as possession of the specific edition cited in the case of nonserial materials or the specific issue in the case of serials. Electronic owner- ship was defined as access to the complete full-text of the specific article cited. As the data-gathering portion of this study was conducted in the spring of 2002, the author could assume with confidence that any item currently listed in the OPAC also was available to graduate students fin- ishing theses and dissertations in 2001. However, this same assumption could not necessarily be made about those students who completed their studies in 1991. Unfortunately, due to the libraries’ con- version to a new automation system in 1999, the date of acquisition was not avail- able for any work originally cataloged in the old system. But, because it is not UGA’s policy to devote substantial funding to ret- rospective purchases and because the col- lection has not recently undergone a ma- jor weeding project, the study proceeded on the assumption that if works cited by 1991 theses or dissertations were currently held, they probably were available to the students while they were conducting their research. However, the author acknowl- edges that a firm date of acquisition would have resulted in more accurate 1991 own- ership statistics. Results A total of 3,363 citations were examined: 1,768 from 1991 bibliographies and 1,595 citations from 2001 bibliographies. Of the 1991 works, one had twenty-five or fewer citations, ten had twenty-six to fifty cita- tions, three had fifty-one to seventy-five citations, and sixteen had seventy-six or more citations. Of the 2001 works, six had twenty-five or fewer citations, seven had twenty-six to fifty citations, six had fifty- one to seventy-five citations, and eleven had seventy-six or more citations. On av- erage, 53.2 citations were examined from each 2001 work and 58.9 citations from each 1991 work. While it should be noted that the total number of citations went down quite dra- matically between 1991 and 2001, this study was not designed to investigate this issue; therefore, any speculation on the cause of this decline falls beyond the scope of this project. Format of Materials Cited Table 1 illustrates the distribution of cita- tions by material format. Theses and dis- sertations in both years cited more periodi- cals than any other source, accounting for 54 percent of citations in 1991 and 48 per- cent in 2001. Monographs were cited with the second greatest frequency, with 37 per- cent of citations in 1991 and 38 percent in 2001. The most noticeable change in the past ten years was the appearance of Web pages in bibliographies. Accounting for 3.5 per- cent of total citations in 2001, Web pages were cited more frequently than newspa- pers and magazines (2%), theses and dis- sertations (2%), or “other” resources (3%).20 Disciplinary analysis revealed two dis- tinct citation patterns. Both the arts and humanities and the sciences had one dominant format that accounted for the vast majority of citations. Around three- quarters of arts and humanities citations were to monographs in both 1991 and 2001, whereas the bulk of science citations in both years were to periodicals (79% in 1991, 64% in 2001). On the other hand, citation patterns in both education and the social sciences in- dicated a more balanced reliance on the two major material formats. For these two disciplines, periodicals were the most- cited format in both 1991 and 2001, mak- ing up 60 percent of social science cita- tions and almost half of education cita- tions. However, in both years, mono- 348 C ollege & R esearch L ib raries S ep tem b er 2003 TABLE 1 Material Formats Cited by UGA Theses and Dissertations Conference Newspapers & Papers & Theses & Other Monographs Periodicals Magazines Proceedings Dissertations Web Pages Resources N (%) (%) (%) (%) (%) (%) (%) All Disciplines 1991 1,768 37 54 1 2 2 � 4 2001 1,595 38 48 2 3.5 2 3.5 3 Arts & Humanities 1991 368 75 20 1 2 1 � 1 2001 377 71 19 6 <1 1 0 3 Education 1991 450 37 45 1 1 4 � 12 2001 424 37 49 <1 2 3 6 3 Sciences 1991 547 14 79 0 4 1 � 2 2001 444 18 64 0 7 2 6 3 Social Sciences 1991 403 36 60 1 1 1 � 1 2001 350 32 60 2 3 1 1 1 Assessing Collection Usefulness 349 graphs also were very important, ac- counting for about a third of citations in both education and the social sciences. Analysis also revealed that some dis- ciplines used the Web much more than others did. In 2001, Web sites accounted for an amazing six percent of citations in both education and the sciences, whereas this format received only one percent of social science citations and were not cited at all in the arts and humanities theses and dissertations analyzed. Change in the four other minor format categories also was evident in the disciplin- ary analysis. For example, every discipline except arts and humanities made more ref- erences to conference papers and proceed- ings in 2001 than they had ten years earlier. However, the biggest change was in edu- cation. In 1991, 12 percent of citations fell within the “other” category due to fairly high usage of ERIC documents on micro- form; however, in 2001, only three percent of citations were categorized as “other.” Time Span Figure 1 shows the distribution of cited works by age (the number of years be- tween the completion date of the thesis or dissertation and the publication date of the cited work). Between 1991 and 2001, there was a 75 percent increase in the number of citations made to works published in the same year of theses or dissertation comple- tion: forty-two citations were made to cur- rent-year publications in 2001, compared to only twenty-four in 1991. In fact, there was an overall reduction in time lag in the past ten years: 1991 works relied most heavily on materials published three to four years before the completion date of the graduate work, whereas 2001 works relied on materials published two to three years before the completion date. On the other hand, 2001 works demonstrated a longer total time span, requiring sixty-five years to cover 99 percent of citations, as opposed to 1991 works that only needed fifty years to cover 99 percent of citations. However, the general distribution of cited works by age remained constant. In both years, approximately 30 percent of total citations were to materials published within five years of completion date, 50 percent of citations were covered by 10 years, and around 80 percent of citations were covered by 20 years. A diminished time lag also was evident within each discipline. Current-year cita- 0 20 40 60 80 100 120 Current Year 5 10 15 20 25 30 35 40 45 50 Age of Citation (Years) N u m b er o f C it at io n s 1991 2001 FIGURE 1 Time Span of Works Cited by UGA Theses and Dissertations 350 C ollege & R esearch L ib raries S ep tem b er 2003 Time Span - Arts and Humanities 0 5 10 15 20 25 C ur re nt Y ea r 5 10 15 20 25 30 35 40 45 50 Age of Cited Work (Years) N u m b er o f C it at io n s 1991 2001 Time Span - Education 0 5 10 15 20 25 30 35 40 C ur re nt Y ea r 5 10 15 20 25 30 35 40 45 50 Age of Cited Work (Years) N u m b er o f C it at io n s 1991 2001 Time Span - Sciences 0 10 20 30 40 50 60 C ur re nt Y ea r 5 10 15 20 25 30 35 40 45 50 Age of Cited Work (Years) N u m b er o f C it at io n s 1991 2001 Time Span - Social Sciences 0 5 10 15 20 25 30 35 C ur rr en t Y ea r 5 10 15 20 25 30 35 40 45 50 Age of Cited Work (Years) N u m b er o f C it at io n s 1991 2001 FIGURE 2 Disciplinary Differences in Time Span of Works Cited Assessing Collection Usefulness 351 tions increased by 25 percent in social sci- ence theses and dissertations, 50 percent in arts and humanities works, 80 percent in science works, and 100 percent in edu- cation works. As in the composite evalu- ation, the 2001 theses and dissertations in each of the disciplines reached further back in time with their citations than the 1991 works did. Figure 2 represents the distribution of cited works by age for all four disciplin- ary categories. In both years, the theses and dissertations in the sciences and edu- cation were much more time dependent, exhibiting significant drop-offs in citation frequency after five to ten years; citation frequencies were flatter in the social sci- ences and in the arts and humanities, re- maining relatively steady for a period of ten to fifteen years. Library Ownership Interestingly, for both the 1991 and 2001 sample groups, the University of Georgia Libraries owned 87 percent of works cited by theses and dissertations (table 2). De- spite the fact that the total number of cita- tions to the two major formats decreased, library ownership of these formats changed very little over the past ten years. Ownership of cited monographs increased by two percent while ownership of cited periodicals decreased by three percent. However, change was quite evident in the minor formats. Not only did 2001 the- ses and dissertations cite more newspa- pers and magazines and more conference proceedings and papers, but library own- ership of these formats increased as well, the former by nine percent and the latter by an amazing 77 percent. On the other hand, library ownership of the two for- mats that were cited less frequently in 2001, theses and dissertations and mate- rials in the “other” category, had de- creased since 1991, the former by 12 per- cent and the latter by 20 percent. Interestingly, ownership patterns var- ied between the disciplines with one domi- nant format and those with a more equal distribution of citations between mono- graphs and periodicals (table 2). In the arts and humanities, library ownership of cited works decreased significantly, from 89 percent in 1991 to 80 percent in 2001. Own- ership of the most important format— monographs—decreased by six percent. Library ownership of periodicals, which accounted for about 20 percent of citations in each year, decreased by 24 percent. Science, the other discipline with one dominant format, also exhibited a de- crease in ownership of cited materials, but to a lesser degree—from 91 percent to 89 percent. Periodical ownership dropped by only one percent, whereas ownership of monographs cited by science theses and dissertations, which accounted for less than a quarter of total citations in both years, was down by 19 percent. However, library ownership in the two disciplines with a more even distribution of citations between the two major for- mats improved during the past ten years. Total ownership in both education and the social sciences increased, the former by 12 percent and the latter by two per- cent. In education, periodical ownership increased by two percent and monograph ownership increased by 28 percent. In the social sciences, the percentage of cited monographs owned by the library also increased—from 88 percent in 1991 to 97 percent in 2001. Ownership of periodicals cited by social science graduate works decreased, but only by one percent. Table 3 illustrates the electronic avail- ability of serials cited in 2001 theses and dissertations. The complete full text of 31 percent of all cited articles was available online—31 percent of periodical articles, and 25 percent of newspaper and maga- zine articles. In addition, 57 percent of cited articles published in 1995 or later were available online (Table 4). The library also owned the print version of the majority of these recent articles; however, four percent were only available electronically. Disciplinary analysis indicated that more articles cited in social science graduate works were available electronically than any other discipline (41%). The full text of 32 percent of all articles cited in education works, 25 percent of science articles, and 20 352 C ollege & R esearch L ib raries S ep tem b er 2003 TABLE 2 UGA Libraries Ownership of Works Cited by Theses and Dissertations Total Newspapers & Conference Theses & Other Owned Monographs Periodicals Magazines Papers & Dissertations Resources (%) (%) (%) (%) Proceedings (%) (%) All Disciplines 1991 87 84 94 77 31 65 51 (1,768) (667) (951) (13) (35) (31) (71) 2001 87 86 91 84 55 57 41 (1,595) (613) (769) (32) (55) (30) (41) Arts & Humanities 1991 89 91 85 100 83 0 33 (368) (278) (74) (4) (6) (3) (3) 2001 80 85 65 83 100 40 60 (377) (267) (71) (23) (1) (5) (10) Education 1991 76 69 89 25 0 75 61 (450) (165) (204) (4) (5) (16) (54) 2001 85 88 91 100 10 31 57 (424) (156) (206) (1) (10) (13) (14) Sciences 1991 91 85 96 � 27 100 22 (547) (79) (433) (0) (22) (6) (9) 2001 89 69 95 � 69 90 17 (444) (80) (283) (0) (32) (10) (12) Social Sciences 1991 91 88 96 100 0 33 0 (403) (145) (240) (5) (2) (6) (5) 2001 93 97 95 88 50 100 20 (350) (110) (209) (8) (12) (2) (5) Note: Figure in parentheses is the base N for the percentage listed above it. Assessing Collection Usefulness 353 percent of arts and humanities articles was available via resources provided by the li- braries. In terms of recent articles, gradu- ate students in education, the sciences, and the social sciences had electronic access to the majority of the works they cited (54%, 57%, and 66%, respectively). More than half of the recent articles cited by every discipline except the arts and sciences were available from the li- braries in both print and electronic ver- sions. Thus, it was surprising that arts and sciences had the highest percentage of recent articles available from the library exclusively through online services (10%). No less surprising was the finding that the discipline with the highest percent- age of recent articles available only in print was not the arts and humanities but, rather, the sciences, with 39 percent. Discussion and Conclusions The results of the citation analysis phase of the project concurred with previously pub- lished disciplinary-specific studies. In terms of material formats, theses and dissertations in the sciences relied most heavily on peri- odicals, works in the arts and humanities relied just as heavily on monographs, and works in the social sciences and education TABLE 3 Percentage of Articles Cited by 2001 Works Available Electronically via UGA Libraries, by Serial Format Periodical Newspaper or All Serials (%) Magazine (%) (%) All Disciplines 31 25 31 (769) (32) (801) Arts & Humanities 23 13 20 (71) (23) (94) Education 66 100 32 (206) (1) (207) Sciences 25 � 25 (283) (0) (283) Social Sciences 41 50 41 (209) (8) (217) Note: Figure in parentheses is the base N for the percentage listed above it. TABLE 4 Percentage of Recent Articles Cited by 2001 Theses and Dissertations Available via UGA Libraries, by Method of Access N Print Electronic Both Total Available Only (%) Only (%) Formats (%) Electronically (%) All Disciplines 313 33 4 52 57 Arts & Humanities 30 17 10 30 40 Education 104 33 2 52 54 Sciences 105 39 2 54 57 Social Sciences 74 30 9 57 66 Note: Recent articles were those published in 1995 or later. 354 College & Research Libraries September 2003 cited the two major formats more equally. In terms of time span, the sciences and edu- cation drew most heavily from recent ma- terials, whereas the age of the material was a minimal factor in both the arts and hu- manities and the social sciences. However, this study is among the first citation analyses to demonstrate the impact that the Web and other electronic resources have had on graduate research. The most direct evidence was in the bibliographies of the theses and dissertations themselves. Citations to Web sites went from zero in 1991 to 3.5 percent of total citations in 2001 and were cited more frequently than news- papers and magazines, other theses and dis- sertations, and “other” resources. In addition, the introduction of elec- tronic resources is likely responsible for the change in the bibliographies of education theses and dissertations during the past ten years. The reduction in ERIC micro- form usage in 2001 education theses and dissertations represents a shift to the elec- tronic version of ERIC and a significant change in educational research methods. The dramatic increase in the number of ci- tations made to materials published in the same year as thesis or dissertation comple- tion also may be linked to diminishing time lag due to frequently updated elec- tronic abstracts and indexes as well as to the electronic dissemination of preprints and current periodical articles. However, this change also could be explained by an overall increase in the speed of scholarly communication, including everything from personal correspondence via e-mail to faster production times due to improved publishing technology. The second phase of the project pro- vided a wealth of valuable and previously unquantified local information. First and foremost, the finding that there was no change in the percentage of library- owned materials cited in theses and dis- sertations indicates that there has been very little change in the usefulness of the libraries’ collection to graduate students in the past ten years. Furthermore, considering that (1) bud- get increases at the University of Georgia (as with most ARL libraries) have failed to keep pace with price increases, especially for journals, and (2) those already tight budgets also have had to absorb the cost of electronic resources, this study indicates that the current allocation of funds among the various material formats has had very little, if any, negative effect on the useful- ness of the collection to graduate stu- dents.21 In fact, the finding that library ownership of cited monographs has actu- ally increased since 1991, while periodical ownership decreased, implies that decades of concern about the creation of unbal- anced, periodical-dominated library col- lections as the result of the reallocation of materials budgets to cover increases in an- nual subscription costs has been unwar- ranted. The analyses of library ownership of works cited by graduate students in edu- cation and the social sciences also indicate that current collection development poli- cies are appropriate. Not only has total ownership of the works cited by these two disciplines increased since 1991, but in education it has increased by 12 percent. Specifically, the finding that library own- ership of monographs cited by students in both disciplines has increased signifi- cantly since 1991, while ownership of cited periodicals has not changed significantly, is particularly supportive of the current allocation of funds among the various material formats in these two disciplines. However, the analysis of the other two disciplines revealed a decline in library collection usefulness to graduate students in both the sciences and the arts and hu- manities, with ownership of total works cited by the latter down by an alarming 11 percent. Particularly worrisome is the find- ing that library ownership of the primary format cited by each discipline had de- creased since 1991, with the ownership of cited science periodicals down slightly and the ownership of cited arts and humani- ties monographs down more significantly. Considering these disciplinary differ- ences, the application of this methodol- ogy to individual disciplines, and perhaps even an investigation of subdisciplines, Assessing Collection Usefulness 355 could provide invaluable information about the current state of the various sub- ject collections. At the very least, the rep- lication of this study using a much larger sample would provide even more accu- rate results by reducing the probability of sampling error. But to fully understand these issues and to answer the question of how useful the library is to students, more research must be done. However, the current study provided invaluable—and previously unavail- able—insight into the overall health of the library collection. This assessment tool, forged by the combination of citation analysis and local ownership data, pro- vided a snapshot of the current useful- ness of the University of Georgia Librar- ies’ collections and brought the libraries one step closer to the answer. Notes 1. P. L. K. Gross and E. M. Gross, “College Libraries and Chemical Education,” Science 66 (Oct. 1927): 385–89. 2. Linda Smith, “Citation Analysis,” Library Trends 30 (summer 1981): 83–106. 3. Herbert S. White, “Strategies and Alternatives in Dealing with the Serials Management Budget,” in Serials Collection Development: Choices and Strategies (Ann Arbor, Mich.: Pierian Press, 1981), 31–32. 4. Robin B. Devin and Martha Kellogg, “The Serial/Monograph Ratio in Research Libraries: Budgeting in Light of Citation Studies,” College & Research Libraries 51 (Jan. 1990): 46–54. 5. Harry M. Kriz, “Subscriptions vs. Books in a Constant Dollar Budget,” College & Research Libraries 39 (1978): 105–9. 6. Devin and Kellogg, “The Serial/Monograph Ratio in Research Libraries,” 52. 7. Joseph J. Kohut, “Allocating the Book Budget: A Model,” College & Research Libraries 35 (Sept. 1974): 192–99. 8. Devin and Kellogg, “The Serial/Monograph Ratio in Research Libraries,” 50–51. 9. Robert D. Stueart and George B. Miller, eds., Collection Development in Libraries: A Treatise (Greenwich, Conn.: JAI Press, 1980): v. 1, 60. 10. Rosalind Walcott, “Local Citation Studies: A Shortcut to Local Knowledge,” Science & Technology Libraries 14 (1994): 1–14. 11. See Cecelia M. Brown, “Information-seeking Behavior of Scientists in the Electronic Infor- mation Age: Astronomers, Chemists, Mathematicians, and Physicists,” Journal of the American Society for Information Science 50 (Aug. 1999): 929–43; Sally A. Rogers, “Electronic Journal Usage at Ohio State University,” College & Research Libraries 62 (Jan. 2001): 25–34; Carol Tenopir and Donald W. King, “The Use and Value of Scientific Journals: Past, Present, and Future,” Serials 14 (July 2001): 113–20. 12. Philip M. Davis and Suzanne A. Cohen, “The Effect of the Web on Undergraduate Cita- tion Behavior 1996–1999,” Journal of the American Society for Information Science and Technology 52 (Feb. 2001): 309–14. 13. Ibid. 14. Walcott, “Local Citation Studies,” 2. 15. Sherri Edwards, “Citation Analysis as a Collection Development Tool: A Bibliometric Study of Polymer Science Theses and Dissertations,” Serials Review 25 (1999): 11–21. 16. Louise S. Zipp, “Thesis and Dissertation Citations as Indicators of Faculty Research Use of University Journal Collections,” Library Resources & Technical Services 40 (Oct. 1996): 335–42. 17. Theses and dissertations held in the University of Georgia ETD Library are currently only available in full text to users on campus. 18. A periodical is “a serial appearing or intended to appear indefinitely at regular or stated intervals, generally more frequently than annually, each issue of which is numbered or dated consecutively and normally contains separate articles, stories, or other writings. Newspapers disseminate general news, and the proceedings, papers, or other publications of corporate bod- ies primarily related to their meetings are not included in this term.” Heartsill Young, ed., ALA Glossary of Library and Information Science (Chicago: ALA, 1983), 166. 19. UGA Libraries Electronic Journal Locator (http://www.libs.uga.edu/ejournals/). 20. Items categorized as “other” included ERIC documents, lecture notes, “in press” articles, and so on. 21. Martha Kyrillidou and Mark Young, eds., ARL Statistics 2000–01: A Compilation of Statistics from the One Hundred and Twenty-Four Members of the Association of Research Libraries (Washington, D.C.: Association of Research Libraries, 2002), 10.