47 Relevance of Library Collections for Graduate Student Research: A Citation Analysis Study of Doctoral Dissertations at Notre Dame Jessica Kayongo and Clarence Helm Jessica Kayongo is Reference Librarian, Anthropology in the Hesburgh Libraries, e-mail: jkayongo@nd.edu; and Clarence Helm is Data Manager in the Computer Vision Research Laboratory, at the University of Notre Dame, email: chelm@nd.edu. © Jessica Kayongo and Clarence Helm This study focused on determining the extent to which collections of the Hesburgh Libraries of Notre Dame met the needs of graduate students. This study data (2005–2007) consisted of a citation analysis of 248 dissertations and focused on the following questions: What were the graduate students citing in their dissertations? Did the library own the cited items? How did the disciplines compare in their citation patterns? The data showed that over 90 percent of the 39,106 citations were to books and journals. The libraries owned 67 percent of the items graduate students cited in their dissertations. The libraries owned 83 percent of the Arts & Humanities, 90 percent of the Engineering, 92 percent of the Science, and 75 percent of the Social Sciences sources in the top 1,000 most cited titles, indicating a need for funding for further development of Social Sciences collections in the Hesburgh Libraries. he University of Notre Dame is a private Catholic univer- sity, founded in 1842, and lo- cated in Notre Dame, Indiana. The Graduate School, established in 1918, offers thirty-two masters’ and twenty-five doctoral degree programs.1 Professional degrees are administered separately by the School of Architecture, the Law School, and the College of Business. During the period studied (2005–2007), the Graduate School enrolled, on aver- age, 1,951 students annually. Roughly 68 percent of these graduate students were Ph.D. students. On average, 41 percent and 24 percent of the doctoral students enrolled were female and international students, respectively. Doctoral students enrolled as percent of the total number of graduate students varied by discipline as follows: Engineering (93%), Science (98%), Humanities (62%), and Social Sci- ence (48%).2 The first mention of a “college library” at Notre Dame was in 1869, when it con- tained 7,000 volumes. The collection grew and was relocated several times until a new separate library building for the uni- versity library was built and dedicated in 1917, which coincided with the start of the Graduate School in 1918. By 1920, the li- brary collection had grown to over 100,000 crl-211 48 College & Research Libraries January 2012 volumes. The current library system, the Hesburgh Libraries, has one main library building and nine branches with over three million monographs and 45,000 serials, a $22 million budget and 60 library faculty members. The University of Notre Dame is a member of the Association of Research Libraries (ARL).3 This study focused on Notre Dame’s graduate students’ doctoral dissertations in Science, Arts and Humani- ties, Social Sciences, and Engineering. Dissertations produced by students in the professional schools of architecture, law, and business were excluded. Citation analysis is a very powerful tool that demonstrates the researchers’ pathways toward, and support for, their final conclusions. “Citations are sign- posts left behind after information has been utilized and as such provide data by which one may build pictures of user behavior.”4 Items cited in students’ dis- sertations show the resources they used in their dissertation research and writing. Studies in library and information science and other disciplines have used citation analysis as a way to examine various is- sues related to research and researchers’ information-seeking behavior. Research- ers in library and information science use citation analysis to identify and as- sess user needs; determine patterns of author usage of sources; tally and rank referenced materials; and develop library collections for different disciplines based on identified needs and usage counts. The objective of this study was to find out the extent to which the collections of the Hesburgh Libraries of Notre Dame met the needs of the graduate doctoral student population. This was done using citation analysis of the doctoral graduate students’ dissertations and focused on the following questions: What were the doctoral graduate students citing in their dissertations? Did the library own the cited items? How did citation patterns differ among academic disciplines? Literature Review Analysis of citation data can be used to create core journal lists, which can be used to make collection decisions in support of research and teaching. However, errors arise from mistakes made in the citations. For example, names of authors, titles of articles, and journal titles are often listed incorrectly and inconsistently in pub- lished works. The problem this creates in terms of analyzing data is duplication of entries in datasets, which might un- derestimate or overestimate use. Not- withstanding possible methodological problems, citation analysis is a valuable tool for librarians and libraries in making decisions about collection development, collection maintenance, library services, and library budget considerations.5 A number of studies have used cita- tion analysis specifically for collection development and creating core lists (Wilder; Kelsey and Diamond; Burright et al.; Waugh and Ruppel; Pancheshnikov; Nabe and Imre; Ralston et al.; and Gao et al.).6 Yet other studies have focused merely on assessing the match between the library collection and the citation patterns of graduate students or other pa- trons (Smith; Fuchs et al.; Pancheshnikov; and Wilson and Tenopir).7 Knight-Davis and Sung looked at changes in students’ citation patterns over time.8 Other studies have dealt with citation patterns per se (Gooden; Sam and Tackie; Kuruppu and Moore; and Cox);9 comparing citation patterns in different disciplines (Sinn);10 and testing Bradford’s Law of Scattering (Tonta and Al).11 Last, some researchers have studied faculty publications (Kelsey and Diamond; Burright et al.; Wilson and Tenopir; Pancheshnikov; and Ralston et al.).12 In terms of sample size, Kuruppu and Moore had the largest number of disser- tations and total citations of all the stud- ies reviewed.13 The longest time frame sampled was the study by Sinn covering a 23-year period between 1980 and 2002.14 There have been a variety of sampling strategies used in these types of studies. One strategy was to select all documents (dissertations, theses, journal articles, Relevance of Library Collections for Graduate Student Research 49 and/or research papers) within a particu- lar time frame. This strategy was used by Burright et al. on neuroscientists at the University of Maryland;15 Cox on gradu- ate theses at Indiana University School of Dentistry;16 Tonta and Al using masters’ theses and doctoral dissertations pro- duced at Turkey’s Hacettepe University Department of Librarianship;17 Waugh and Ruppel’s study based on research pa- pers, theses, and dissertations produced by graduate students in the Workforce Education and Development (WED) de- partment at Southern Illinois University, Carbondale;18 Sinn’s using mathematics and statistics dissertations produced at Bowling Green;19 Nabe and Imre in their study on biology dissertations at South- ern Illinois;20 Pancheshnikov’s study on literature citations in faculty publications and student theses at the University of Saskatchewan; 21 and Ralston et al. in their study at the Indiana University School of Medicine.22 Another strategy was to take a sample of the documents from a specified time frame. A third option was to sample a certain number of citations out of the total number of citations found in the documents fitting a particular selection criterion. Some studies have used a combination of strategies. Among those using the second and/or third strategy was Gooden, who sampled chemistry dissertations produced and housed in the Science & Engineering Library at Ohio State University;23 Knight-Davis and Sung, who used a sample of Eastern Illinois University undergraduate student portfolios;24 and Gao et al., who sampled dissertations from library and informa- tion science, biology, photogrammetry and remote sensing, and stomatology at Wuhan University.25 Others, such as Smith, used a random sample of theses and dissertations produced at the Univer- sity of Georgia by the education, arts and humanities, sciences, and social sciences disciplines;26 Fuchs et al. did a “classic bibliographic citation analysis” on disser- tations produced in civil engineering and educational psychology at the University of Texas at Austin;27 and Sam and Tackie examined usage of different formats of materials in dissertations produced in the Department of Information Studies at the University of Ghana, Legon.28 In their citation analysis, Kuruppu and Moore used half of the dissertations emanating from nine agriculture and biology subject disciplines at Iowa State University.29 Most of the studies reviewed made no specific mention of computer pro- grams used for data entry, including, for example, Smith; Tonta and Al; Sam and Tackie; Kuruppu and Moore; and Pancheshnikov.30 There were studies that used a spreadsheet or database (Waugh and Ruppel; Fuchs et al.; and Knight- Davis and Sung).31 Other studies used Microsoft Access (Burright et al.; Sinn; and Ralston et al.)32 or Microsoft Excel (Haycock; Cox; Gao et al.; Nabe et al.; and Wilson and Tenopir).33 Methodology This citation analysis study focused on dissertations authored by doctoral gradu- ate students at the University of Notre Dame for a three-year period from 2005 to 2007. A Notre Dame Faculty Research Pro- gram grant funded this study. Because the grant period was 2008, the dissertations studied were for the most recent three years preceding that year: 2005, 2006, and 2007. The library owns print copies of all Notre Dame dissertations and electronic copies of most Notre Dame dissertations completed in recent years. For this study, only those dissertations for which the library had an electronic copy were used. Thus, the study examined the 248 disser- tations that were produced in 2005, 2006, and 2007. The dissertations were pro- duced in nineteen departments, with the most (26 apiece) coming out of Biological Sciences, Chemical & Biomolecular Engi- neering, and Electrical Engineering. All electronic copies of dissertations are ac- cessible from the library’s Web site in the Electronic Theses & Dissertations (ETD). From the ETD, a PHP script was used 50 College & Research Libraries January 2012 to capture bibliographic data about the dissertations, including dissertation title, dissertation author, author’s department, page length, and year of the dissertation. For each dissertation, every biblio- graphic reference cited in the dissertation was entered into a database. Specific forms (see figure 1) were created in a Mi- crosoft Access database to make it easier to collect bibliographic data, including citation title, citation author (for nonse- rial sources), citation year, and citation type. Visual Basic for Applications (VBA) code was added to the forms to compare the data being entered against preexist- ing entries to eliminate redundancy and to standardize the form fields. In cases where a record was found to already ex- ist in the database, it was linked to the newly entered citation. Where there was no match for a newly entered citation, a new record was created. To expedite the process of data entry, autocomple- tion was also used. In the event that the person doing data entry needed to edit or delete an entry, a subform allowed for that. The researcher, along with five research assistants, gathered biblio- graphic data from the references cited in each dissertation, including citation title (cited publication), citation author, citation year, and citation type (such as journal or newspaper). This information was entered into the Access database. In total, 39,106 citations were entered, which translated into 27,652 discrete citations. In other words, the 39,106 figure represented the total corpus of citations collected whereas the 27,652 figure represented a grouping of unique items, those items that shared the same title, year, citation type, and author (for books). Grouping is explained more fully later in this section. The researchers checked the data for accuracy by comparing and correcting each reference gathered using additional bibliographic data from the library’s online catalog. For every cited reference, researchers manually searched the catalog to determine whether the library owned a copy of the cited source. If a cited item was not found in the catalog, an attempt was made to validate the original citation. To ensure that the library did not own a correctly cited item and that the citation was correct, a verification of its existence was done (by searching WorldCat, for instance). This process, in essence, elimi- nated citation errors made by dissertation authors. Ownership of an item meant that the library had a subscription, either print or electronic to the full text of the cited source. So, for example, links to electronic books and open access jour- nals would also be considered “owned.” Ownership status was determined at the time of the study in 2008, that is, whether the library currently had subscriptions to the items that the graduate students had cited in their dissertations completed in 2005, 2006, and 2007. Researchers did not collect “point in time” data due to time and financial limitations. Finally, the data were cleaned to elimi- nate any redundancies that may have been created using a variety of techniques. Redundancies in collected data were partly attributable to dissertation authors’ use of abbreviations or acronyms of source titles. Cleanup methods included using an algorithm to compare string dis- tance, Access queries (such as title within title and initial article), and spell-check. Researchers corrected items within the database after reviewing the reports and making necessary adjustments. A dupli- cate records search was done to remove duplicate records, and the database was updated accordingly. These steps were repeated until all redundancy checks and queries returned no errors. Collected citations were classified in two different ways based on the biblio- graphic data and disciplines. There were three groups based on the bibliographic data: Group 1 contained the 27,652 dis- crete citations mentioned previously in this section and included items that shared the same year, title, type, and author. Year was included in this first group so that, for example, the same Relevance of Library Collections for Graduate Student Research 51 figure 1 Sample Data entry form 52 College & Research Libraries January 2012 journal, from two different years, was seen as two discrete (separate) sources. Group 2 contained 18,942 discrete cita- tions and included those items that shared the same title, type, and author. In this second group, all journals with the same title, even if they were from different years, were seen as one source. In es- sence, Group 2 allowed for a more valid comparison between journal titles and other materials than Group 1 did. Group 3 contained 11,278 discrete citations and varied by only identifying the cited items owned by Notre Dame’s libraries. For citations referring to books, author data were collected to distinguish between books with the same title. Author data were not collected for journal references because the journal at large, as opposed to each individual article contained within the journal, was the focus of this citation analysis. The second way the data were classi- fied was in groupings by discipline. There were four groups created and classified by discipline: Arts and Humanities, Engineering, Science, and Social Sci- ences (see table 1). Table 1 also shows the descriptive data for the dissertations in each disciplinary group. Additionally, all data in the bibliographic and disciplin- ary groups were subjected to a weighted index. Weighting was used to ensure that an individual dissertation citing an item an inordinate number of times would not bias the results. In other words, weighting was done in recognition of the fact that one publication with numerous citations to a particular source would create an im- precise ranking order. So, weighting was used to get a true measure of which items were being cited the most by different graduate students, in general, and by dis- cipline. The weighted index formula from the Waugh and Ruppel study was used.34 This weighting formula ranked items in order of those cited the most times by the greatest number of graduate students. All the data collected and processed was then analyzed using the Statistical Package for the Social Sciences (SPSS).35 Results The average length of dissertation overall was 196.9 pages. History dissertations were the longest with an average page length of 426.3 pages, and psychology dissertations were the shortest with an average page length of 83.6 pages. The longest dissertation in this study was from Civil Engineering & Geological Sci- ences with 813 pages, while the shortest one was from Mathematics with 39 pages. On average, there were 0.83 citations per page (total number of pages divided by the total number of end references in the bibliography) overall, 1.23 citations per page in Biological Sciences, and 0.30 citations per page in Medieval Studies (see table 1). The average age of citations overall was 19.1 years. The mean age of citation sources by discipline was as follows: Arts & Humanities (33.4 years), Engineering (11.4 years), Science (10.8 years) and Social Sciences (15.7 years). On average, graduate students in the departments of History and of History and Philosophy of Science tended to cite older material, 43.3 and 43.2 years, respectively. However, graduate students in the department of Computer Science & Engineering cited relatively newer (7.1 years old) materials (see table 1). Of the 39,106 total citations, 55.2 percent were journal citations and 36.8 percent were book citations (see table 2). Classification groups [Group 1 (same year, title, type, and author), Group 2 (same title, type, and author) and Group 3 (same title, author, and type and owned by Notre Dame’s libraries)], all showed that books and journals were the most frequently cited types of materials as fol- lows: Group 1 (46% journals, 44% books), Group 2 (64% books, 22% journals), and Group 3 (66% books, 31% journals) (see table 3). Classification groups based on dis- ciplines also showed that books and journals were the most frequently cited sources, accounting for 96 percent, 78 percent, 92 percent, and 88 percent of all Relevance of Library Collections for Graduate Student Research 53 TABLe 1 Number and Characteristics of Dissertations used by Department and Disciplinary Areas Discipline Department Number of Dissertations Dissertation Length Total Citations Average Citations per Page Average Citation AgeAverage Longest Shortest Arts and Humanities History 9 426.3 706 310 3,547 0.98 43.3 Theology 16 336.9 506 226 4,693 0.81 24.5 History & Philosophy of Science 2 293.0 308 278 580 1.00 43.2 English 12 277.0 487 177 2,893 0.90 29.4 Philosophy 6 235.8 404 182 771 0.48 14.7 Medieval Studies 1 233.0 233 233 65 0.30 30.7 Arts & Humanities 46 321.4 706 177 12,549 0.85 33.4 Engineering Civil Engineering and Geological Sciences 15 231.5 813 96 1,810 0.66 12.7 Chemical and Biomolecular Engineering 26 180.7 427 99 4,440 0.96 11.8 Computer Science and Engineering 9 173.3 246 120 978 0.63 7.1 Electrical Engineering 26 147.2 237 80 2,175 0.59 11.3 Aerospace and Mechanical Engineering 16 137.4 243 98 1,273 0.61 14.0 engineering 92 171.3 813 80 10,676 0.68 11.4 Science Chemistry and Biochemistry 11 198.1 272 130 2,025 0.95 10.2 Biological Sciences 26 166.2 442 97 5,018 1.23 8.9 Physics 22 160.7 285 55 2,125 0.64 16.1 Mathematics 9 102.2 152 39 301 0.39 14.9 Science 68 161.1 442 39 9,469 0.86 10.8 Social Sciences Political Science 12 288.2 507 186 2,525 0.78 20.3 Sociology 5 242.4 414 155 1,076 1.00 13.7 Economics 6 178.7 299 71 1,045 1.00 13.1 Psychology 19 83.6 131 51 1,766 1.13 13.0 Social Sciences 42 174.5 507 51 6,412 0.87 15.7 TOTAL 248 196.9 813 39 39,106 0.80 19.1 54 College & Research Libraries January 2012 the citations in Arts and Humanities, Engineering, Science, and Social Sci- ences dissertations, respectively. The most frequently cited types of materials by discipline were: Arts & Humani- ties, books (73%) and journals (23%); Engineering, journals (64%) and books (14%); Science, journals (78%) and books (14%); and Social Sciences, journals (46%) and books (42%). When examined at a departmental level, the same pat- tern emerged where journals and books were the most cited items across all departments, except in Computer Sci- ence & Engineering, where conference proceedings were the most frequently cited (32%), followed by journals (31%), online sources (14%), and books (11%). Two departments, while maintaining the journals and books pattern, had citation types in third place that were notable. The third most highly cited items in Electrical Engineering were conference proceedings (14%) and in Economics were papers (11%). Further analysis, using the weighted index, indicated the most cited titles overall. Of the top twenty most cited titles overall, the journals of Science and The Journal of Biological Chemistry appeared most frequently (see table 5). There was an average of 4.9 citations to Science (for the year 2000), for 29 dis- sertations, which was 11.7 percent of all the dissertations. To apply the weighted index to the data in the disciplinary groups, the weighting formula was adjusted to limit analysis to those dissertations within each group, as follows: Group Weight = (T/G) x C (where T = number of gradu- ate students citing each item within the disciplinary group; G = number of dissertations in that disciplinary group, and C = number of times the item was cited in that disciplinary group). In terms of the number of dissertations in each disciplinary group, there were 46 in Arts and Humanities, 92 in Engi- neering, 68 in Science, and 42 in Social Sciences. TABLe 2 Citation Type by frequency and Percentage Citation Type frequency Percent Journal 21,597 55.2% Book 14,406 36.8% Conf. Proc. 856 2.2% Online Source 601 1.5% Dissertation 361 0.9% Paper 258 0.7% Report 238 0.6% Other 206 0.5% Newspaper 121 0.3% Thesis 78 0.2% Manuscript 73 0.2% Magazine 69 0.2% Conf. Presentation 52 0.1% Computer Program 29 0.1% Manual 26 0.1% Patent 24 0.1% Admin. Materials 16 0.0% Presentation 15 0.0% Mimeograph 13 0.0% Course Notes 10 0.0% E-mail 8 0.0% Preprint 8 0.0% Speech 5 0.0% Poster 4 0.0% Computer File 4 0.0% Newsletter 4 0.0% Lecture 4 0.0% Poll 3 0.0% Interview 3 0.0% Document 3 0.0% Electronic Device 2 0.0% Video 2 0.0% Map 2 0.0% Project Summary 1 0.0% Image 1 0.0% Memorandum 1 0.0% Testimony 1 0.0% Radio broadcast 1 0.0% Total 39,106 100.00% Relevance of Library Collections for Graduate Student Research 55 TABLe 3 unique Citation Types by Bibliographic grouping Citation Type group 1* group 2** group 3*** frequency Percent frequency Percent frequency Percent Journal 12,771 46% 4,236 22% 3,444 31% Book 12,131 44% 12,131 64% 7,492 66% Conf. Proc. 630 2% 505 3% 117 1% Online Source 557 2% 548 3% 17 0% Dissertation 342 1% 342 2% 0 0% Paper 248 1% 239 1% 22 0% Report 230 1% 230 1% 7 0% Other 200 1% 196 1% 10 0% Newspaper 101 0% 84 0% 83 1% Thesis 74 0% 74 0% 21 0% Manuscript 72 0% 72 0% 0 0% Magazine 62 0% 51 0% 32 0% Conf. Presentation 51 0% 51 0% 1 0% Computer Program 27 0% 27 0% 0 0% Patent 24 0% 24 0% 0 0% Manual 24 0% 24 0% 0 0% Presentation 15 0% 15 0% 0 0% Admin. Materials 15 0% 15 0% 0 0% Mimeograph 13 0% 13 0% 0 0% Course Notes 9 0% 9 0% 1 0% Email 8 0% 8 0% 0 0% Preprint 8 0% 8 0% 0 0% Speech 5 0% 5 0% 1 0% Poster 4 0% 4 0% 26 0% Computer File 4 0% 4 0% 0 0% Newsletter 4 0% 4 0% 1 0% Lecture 4 0% 4 0% 2 0% Poll 3 0% 3 0% 0 0% Interview 3 0% 3 0% 0 0% Document 2 0% 2 0% 0 0% Electronic Device 2 0% 2 0% 0 0% Video 2 0% 2 0% 0 0% Map 2 0% 2 0% 1 0% Project Summary 1 0% 1 0% 0 0% Image 1 0% 1 0% 0 0% Memorandum 1 0% 1 0% 0 0% Testimony 1 0% 1 0% 0 0% Radio broadcast 1 0% 1 0% 0 0% Total 27,652 100% 18,942 100% 11,278 100% *Group 1 = same year, title, type, and author; ** Group 2 = same title, type, and author; and ***Group 3 = same title, type, and author and owned by Hesburgh Libraries 56 College & Research Libraries January 2012 TA B L e 4 Ty pe s of C it at io ns b y D ep ar tm en t a nd D is ci pl in ar y A re as D is ci pl in e D ep ar tm en t Jo ur na l B oo k C on f. P ro c. O nl in e So ur ce D is se rt at io n P ap er r ep or t N ew sp ap er O th er To ta l N A rt s & H um an it ie s H is to ry 13 % 81 % 0% 0% 2% 0% 0% 1% 2% 3, 54 7 E ng lis h 16 % 81 % 0% 1% 1% 0% 0% 1% 1% 2, 89 3 M ed ie va l S tu di es 22 % 77 % 0% 0% 2% 0% 0% 0% 0% 65 H is to ry a nd P hi lo so ph y of S ci en ce 22 % 75 % 1% 0% 2% 0% 0% 0% 0% 58 0 T he ol og y 29 % 69 % 0% 0% 1% 0% 0% 0% 0% 4, 69 3 Ph ilo so ph y 37 % 54 % 1% 0% 1% 5% 0% 0% 2% 77 1 A ve ra ge fo r A rt s & H um an it ie s* 23 % 73 % e ng in ee ri ng A er os pa ce a nd M ec ha ni ca l E ng in ee ri ng 59 % 22 % 9% 1% 2% 4% 2% 0% 3% 1, 27 3 E le ct ri ca l E ng in ee ri ng 62 % 15 % 14 % 3% 2% 0% 1% 0% 3% 2, 17 5 C iv il E ng in ee ri ng a nd G eo lo gi ca l S ci en ce s 78 % 14 % 2% 1% 1% 0% 2% 0% 2% 1, 81 0 C om pu te r S ci en ce a nd E ng in ee ri ng 31 % 11 % 32 % 14 % 1% 0% 5% 0% 5% 97 8 C he m ic al a nd B io m ol ec ul ar E ng in ee ri ng 88 % 9% 0% 1% 0% 0% 0% 0% 1% 4, 44 0 A ve ra ge fo r e ng in ee ri ng * 64 % 14 % Sc ie nc e M at he m at ic s 51 % 34 % 4% 2% 3% 0% 1% 0% 6% 30 1 Ph ys ic s 74 % 13 % 1% 6% 3% 0% 1% 0% 3% 2, 12 5 B io lo gi ca l S ci en ce s 94 % 5% 0% 0% 0% 0% 1% 0% 0% 5, 01 8 C he m is tr y an d B io ch em is tr y 95 % 3% 0% 0% 0% 0% 0% 0% 1% 2, 02 5 A ve ra ge fo r Sc ie nc e* 78 % 14 % So ci al Sc ie nc es Po lit ic al S ci en ce 25 % 67 % 0% 1% 0% 1% 1% 0% 4% 2, 52 5 So ci ol og y 51 % 42 % 0% 1% 2% 0% 1% 0% 3% 1, 07 6 E co no m ic s 41 % 31 % 0% 6% 0% 11 % 1% 2% 6% 1, 04 5 Ps yc ho lo gy 68 % 29 % 0% 0% 0% 0% 0% 0% 2% 1, 76 6 A ve ra ge fo r So ci al S ci en ce s* 46 % 42 % O ve ra ll A ve ra ge 55 % 37 % 2% 2% 1% 1% 1% 0% 2% *A ve ra ge s ar e pr ov id ed o nl y fo r j ou rn al s an d bo ok s Relevance of Library Collections for Graduate Student Research 57 TABLe 5 Most frequently Cited items for All Dissertations Title Year Dissertation Weighted index % of all Dissertations 1 Science 2000 29 4.9113 11.69% 2 Science 2002 26 4.7177 10.48% 3 The Journal of Biological Chemistry 2004 17 2.8790 6.85% 4 Science 2001 21 2.7097 8.47% 5 The Journal of Biological Chemistry 2003 17 2.6048 6.85% 6 Nature 2002 16 2.3871 6.45% 7 The Journal of Biological Chemistry 2001 15 2.3589 6.05% 8 Science 1999 19 1.9153 7.66% 9 Proceedings of the National Academy of Sciences of the United States of America 2000 18 1.8871 7.26% 10 The Journal of Biological Chemistry 1999 18 1.8871 7.26% 11 The Journal of Biological Chemistry 2002 16 1.8710 6.45% 12 Proceedings of the National Academy of Sciences of the United States of America 2001 14 1.6371 5.65% 13 Nature 2001 16 1.6129 6.45% 14 Journal of the American Chemical Society 2004 14 1.5806 5.65% 15 Nature 1999 16 1.5484 6.45% 16 The Journal of Physical Chemistry B 2004 10 1.5323 4.03% 17 Science 1997 17 1.5081 6.85% 18 The Journal of Biological Chemistry 2000 13 1.3629 5.24% 19 Journal of the American Chemical Society 2003 12 1.3548 4.84% 20 The Journal of Physical Chemistry B 2002 12 1.3548 4.84% 21 Science 1998 16 1.3548 6.45% 22 Proceedings of the National Academy of Sciences of the United States of America 1999 16 1.3548 6.45% 23 The Journal of Biological Chemistry 1996 16 1.3548 6.45% 24 Proceedings of the National Academy of Sciences of the United States of America 2002 14 1.2984 5.65% 25 Applied Physics Letters 2001 13 1.2056 5.24% 58 College & Research Libraries January 2012 The data in tables 6, 7, 8, and 9 show the top twenty most frequently cited items by discipline with the weighted index and the group weighted index. In Arts & Humanities, the top twenty most frequently cited items were overwhelm- ingly religious sources. In Engineering, Science, The Journal of Physical Chemistry B, and Applied Physics Letters were the most frequently cited items. In Science, several issues of The Journal of Biological Chemistry were cited most frequently. In Social Sci- ences, psychology sources, such as Child Development, were cited most frequently. In this study, library ownership meant that the library had either a print or an electronic copy of a particular cited item. Of the 27,672 unique materials cited (Group 1), Hesburgh Libraries owned 18,461, or 67 percent. The weighted index was applied to determine which items were the top 1,000 cited overall. Of the top 1,000 items cited overall, Hesburgh Libraries owned 93 percent. When the group weighted index was applied to the top 1,000 citations by the various disciplinary groups, library ownership was as follows: Arts & Humanities (83%), Engineering (90%), Science (92%), and Social Sciences (75%) (see table 10). Discussion and Conclusions Results of this study show that the Uni- versity of Notre Dame owns 67 percent of the materials Notre Dame graduate students cite in their dissertations. To get a more accurate assessment of their actual needs, however, a weighted index was applied. Of the top 1,000 citations overall (using a weighted index), library ownership rose to 93 percent. Further breakdown by group shows some varia- tion in ownership and access by the dif- ferent disciplines, with Science faring the best (92%), followed by Engineering (90%), Arts & Humanities (83%), and Social Sciences (75%). Smith (2003), in her study of Univer- sity of Georgia graduate student theses and dissertations, reported an 87 percent overall ownership and the following percentages of ownership by disciplin- ary group: Social Sciences (93%), Science (89%), and Arts & Humanities (80%).36 The major difference between the Geor- gia statistics and Notre Dame’s is the much lower library ownership (75%) of materials cited by the Social Sciences graduate students. Elsewhere, studies at the University of Texas-Austin found that the library owned 84.9 percent of the materials cited in graduate student theses and dissertations for education psychology students.37 The Notre Dame figure of 90 percent library ownership of materials cited by Engineering gradu- ate students is, however, comparatively much higher than that reported by Fuchs et al., who found that the library owned 71.2 percent of what University of Texas- Austin civil engineering students were citing.38 The gap in ownership of materials at Notre Dame, cited by students in the physical sciences as compared to owner- ship of materials cited by students in the social sciences, might be attributable to differences in internal library budgetary allocations or to differences in endow- ments for particular subjects. Although endowments are generally restricted by donor preferences, other sources of fund- ing can be administratively reallocated as is deemed necessary. Age of the most cited materials in this study differed from findings in other studies on graduate student citations. In a citation analysis of Chemistry and Chemi- cal Engineering dissertations, Vallmitjana and Sabate found that the mean age of citations was 14 years.39 The mean age of citations for Notre Dame’s most compa- rable group, Science, was 10.8 years. The Texas-Austin educational psychology student citations were, on average, 13.7 years old, whereas the Notre Dame’s Social Sciences group citations averaged 15.7 years. However, the mean age of cited items by the civil engineering students in the Fuchs et al. study, at the University of Texas-Austin,40 was comparable to Notre Dame’s Engineering group (11.34 vs. 11.4 years, respectively). Relevance of Library Collections for Graduate Student Research 59 TA B L e 6 To p 25 C it at io n it em s fo r A rt s an d H um an it ie s D is ci pl in es C it at io n T it le Y ea r To ta l C it at io ns N um be r of D is se rt at io ns C it in g th is r ef er en ce D is ci pl in ar y g ro up W ei gh te d in de x O ve ra ll W ei gh te d in de x A nc ho r B ib le d ic tio na ry / D av id N oe l F re ed m an , e di to r- in -c hi ef ; a ss oc ia te e di - to rs , G ar y A . H er io n, D av id F . G ra f, Jo hn D av id P le in s ; m an ag in g ed ito r, A st ri d B . B ec k. 19 92 12 5 1. 30 0. 24 C os m os C ru m bl in g: A m er ic an R ef or m a nd th e R el ig io us Im ag in at io n / R ob er t H . A bz ug . 19 94 6 6 0. 78 0. 15 T he D em oc ra tiz at io n of A m er ic an C hr is tia ni ty / N at ha n O . H at ch . 19 89 6 6 0. 78 0. 20 E xi le : O ld T es ta m en t, Je w is h, a nd C hr is tia n C on ce pt io ns / ed ite d by J am es M . Sc ot t. 19 97 11 3 0. 72 0. 13 A m er ic an n at io na l b io gr ap hy /g en er al e di to rs , J oh n A . G ar ra ty , M ar k C . C ar ne s. 19 99 9 3 0. 59 0. 11 Fr om re ve la tio n to c an on : s tu di es in th e H eb re w B ib le a nd S ec on d Te m pl e lit er a- tu re / by J am es C . V an de rK am . 20 00 13 2 0. 57 0. 10 T he p ol iti ca l u nc on sc io us : n ar ra tiv e as a s oc ia lly s ym bo lic a ct / Fr ed ri c Ja m es on . 19 81 5 5 0. 54 0. 10 W ith ou t G od , w ith ou t c re ed : t he o ri gi ns o f u nb el ie f i n A m er ic a / J am es T ur ne r. 19 85 5 5 0. 54 0. 10 C at ho lic is m a nd A m er ic an fr ee do m : a h is to ry / Jo hn T . M cG re ev y. 20 03 5 5 0. 54 0. 10 T he W ill ia m a nd M ar y Q ua rt er ly . 20 02 11 2 0. 48 0. 09 T he D ea d Se a sc ro lls : fi ft y ye ar s af te r t he ir d is co ve ry : pr oc ee di ng s of th e Je ru - sa le m C on gr es s, J ul y 20 -2 5, 1 99 7 / e di te d by L aw re nc e H . S ch iff m an , E m an ue l To v, a nd J am es C . V an de rK am . 20 00 7 3 0. 46 0. 08 Ph ilo so ph ic al p er sp ec tiv es 20 00 10 2 0. 43 0. 08 T he C om m un ity o f t he R en ew ed C ov en an t: th e N ot re D am e Sy m po si um o n th e D ea d Se a Sc ro lls / ed ite d by E ug en e U lr ic h an d Ja m es V an de rK am 19 94 5 4 0. 43 0. 08 (c on tin ue d) 60 College & Research Libraries January 2012 TA B L e 6 (c on tin ue d) To p 25 C it at io n it em s fo r A rt s an d H um an it ie s D is ci pl in es C it at io n T it le Y ea r To ta l C it at io ns N um be r of D is se rt at io ns C it in g th is r ef er en ce D is ci pl in ar y g ro up W ei gh te d in de x O ve ra ll W ei gh te d in de x T he C at ho lic B ib lic al q ua rt er ly 19 81 5 4 0. 43 0. 08 E nc yc lo pe di a of th e D ea d Se a sc ro lls / L aw re nc e H . S ch iff m an , J am es C . V an de rK am , e di to rs in c hi ef 20 00 6 3 0. 39 0. 07 Pe rs ia a nd T or ah : T he T he or y of Im pe ri al A ut ho ri za tio n of th e Pe nt at eu ch /J am es W . W at ts , e di to r 20 01 9 2 0. 39 0. 07 T he G os pe l o f L uk e / L uk e Ti m ot hy J oh ns on ; D an ie l J . H ar ri ng to n, e di to r 19 91 4 4 0. 35 0. 06 H is to ri og ra ph y an d Se lf -D efi ni tio n: J os ep hu s, L uk e- A ct s, a nd A po lo ge tic H is to - ri og ra ph y 19 92 4 4 0. 35 0. 06 A m er ic a' s G od : f ro m J on at ha n E dw ar ds to A br ah am L in co ln / M ar k A . N ol l 20 02 4 4 0. 35 0. 06 Jo ur na l f or th e St ud y of th e Ps eu de pi gr ap ha 19 88 4 4 0. 35 0. 06 A G re ek -E ng lis h le xi co n / c om pi le d by H en ry G eo rg e L id de ll an d R ob er t S co tt 19 96 4 4 0. 35 0. 06 H on or & S la ve ry : L ie s, D ue ls , N os es , M as ks , D re ss in g as a W om an , G if ts , St ra ng er s, H um an ita ri an is m , D ea th , S la ve R eb el lio ns , T he P ro sl av er y A rg um en t, B as eb al l, H un tin g, a nd G am bl in g in th e O ld S ou th /K en ne th S . G re en be rg 19 96 4 4 0. 35 0. 06 C ra dl e of th e M id dl e C la ss : T he F am ily in O ne id a C ou nt y, N ew Y or k, 1 79 0– 18 65 / M ar y P. R ya n 19 81 4 4 0. 35 0. 06 M ak in g th e A m er ic an S el f: J on at ha n E dw ar ds to A br ah am L in co ln / D an ie l W al ke r H ow e 19 97 4 4 0. 35 0. 06 T he C hr on ic le r a s A ut ho r: S tu di es in T ex t a nd T ex tu re / ed ite d by M . P at ri ck G ra ha m & S te ve n L . M cK en zi e 19 99 16 1 0. 35 0. 06 Relevance of Library Collections for Graduate Student Research 61 TABLe 7 Top 25 Citation items for engineering Disciplines Citation Title Year Total Citations Number of Dissertations Citing this reference Disciplinary group Weighted index Overall Weighted index 1 Science 2002 16 10 4.52 4.72 2 The Journal of Physical Chemistry B 2004 35 8 3.80 1.53 3 Science 2000 12 8 3.78 4.91 4 Applied Physics Letters 1999 27 9 2.93 1.17 5 Science 1999 14 10 2.89 1.92 6 Applied Physics Letters 2001 19 10 2.68 1.21 7 The Journal of Physical Chemistry B 2002 20 9 2.61 1.35 8 The Journal of Physical Chemistry B 2005 21 8 2.28 0.93 9 Applied Physics Letters 2003 15 9 1.96 1.16 10 Journal of the American Chemical Society 2004 12 8 1.83 1.58 11 IEEE transactions on information theory 2001 22 6 1.67 0.65 12 Science 1997 9 6 1.66 1.51 13 The Journal of Physical Chemistry B 2001 16 8 1.57 0.69 14 Journal of the American Chemical Society 2003 12 6 1.57 1.35 15 Journal of Chemical Physics 2004 13 8 1.55 0.80 16 Applied Physics Letters 2000 13 8 1.41 0.65 17 Applied Physics Letters 2002 13 7 1.41 0.73 18 Nature 1999 8 4 1.39 1.55 19 Science 1998 7 6 1.22 1.35 20 The Journal of Physical Chemistry B 2003 11 6 1.20 0.73 21 Journal of the Electro- chemical Society 2005 21 5 1.14 0.42 22 Science 2001 5 4 1.14 2.71 23 Applied Physics Letters 2005 13 7 1.13 0.45 24 Applied Physics Letters 2004 10 6 1.09 0.60 25 Journal of Applied Physics 2004 10 7 1.09 0.56 62 College & Research Libraries January 2012 TABLe 8 Top 25 Citation items for Science Disciplines Citation Title Year Total Citations Number of Dissertations Citing this reference Disciplinary group Weighted index Overall Weighted index 1 Science 2000 30 21 12.79 4.91 2 Science 2002 29 16 11.09 4.72 3 The Journal of Biological Chemistry 2004 42 17 10.50 2.88 4 The Journal of Biological Chemistry 2003 37 16 9.25 2.60 5 Science 2001 27 17 8.34 2.71 6 The Journal of Biological Chemistry 2001 37 14 8.16 2.36 7 Nature 2002 31 12 7.29 2.39 8 The Journal of Biological Chemistry 2002 29 16 6.82 1.87 9 The Journal of Biological Chemistry 1999 24 16 6.35 1.89 10 Proceedings of the National Academy of Sciences of the United States of America 2000 23 16 6.09 1.89 11 Proceedings of the National Academy of Sciences of the United States of America 2001 26 12 5.35 1.64 12 The Journal of Biological Chemistry 2000 26 13 4.97 1.36 13 The Journal of Biological Chemistry 1996 20 15 4.71 1.35 14 Nature 2001 19 13 4.47 1.61 15 Proceedings of the National Academy of Sciences of the United States of America 1999 18 13 4.24 1.35 16 Proceedings of the National Academy of Sciences of the United States of America 2002 18 10 3.71 1.30 17 The Journal of Biological Chemistry 1998 18 11 3.44 1.05 18 Nature 2000 18 12 3.44 1.00 19 Journal of the American Chemical Society 2004 16 6 3.29 1.58 20 Science 1997 13 11 3.25 1.51 21 Science 1999 11 9 3.07 1.92 22 Nature 1999 13 10 3.06 1.55 23 Science 1994 13 11 3.06 1.16 24 Proceedings of the National Academy of Sciences of the United States of America 1996 16 11 3.06 0.94 25 Proceedings of the National Academy of Sciences of the United States of America 1998 17 11 3.00 0.87 Relevance of Library Collections for Graduate Student Research 63 TABLe 9 Top 25 Citation items for Social Sciences Disciplines Citation Title Year Total Citations Number of Dissertations Citing this reference Disciplinary group Weighted index Overall Weighted index 1 Parenting and the child's world : influences on academic, intel- lectual, and social-emotional de- velopment / {edited by} John G. Borkowski, Sharon Landesman Ramey, Marie Bristol-Power 2002 12 6 1.71 0.29 2 Child development 1984 9 7 1.50 0.25 3 Child development 2000 6 6 0.86 0.20 4 Interwoven lives : adolescent moth- ers and their children / Thomas L. Whitman ... {et al.} ; foreword by Sharon Landesman Ramey 2001 6 6 0.86 0.15 5 Developmental psychology 1997 6 6 0.86 0.15 6 Child development 1994 7 5 0.83 0.14 7 Child development 2003 7 5 0.83 0.14 8 Child development 2005 7 5 0.83 0.14 9 Parenting, science and practice 2004 6 5 0.71 0.12 10 Child development 2002 6 5 0.71 0.12 11 Psychological methods 1996 7 4 0.67 0.11 12 Psychological bulletin 1990 5 5 0.60 0.10 13 Journal of Personality and Social Psychology 1986 5 5 0.60 0.10 14 Multivariate behavioral research 1990 5 5 0.60 0.10 15 After virtue : a study in moral theory / by Alasdair MacIntyre 1981 3 3 0.21 0.10 16 Nature 1999 3 2 0.14 1.55 17 Nature 2001 2 1 0.05 1.61 18 Science 1998 2 1 0.05 1.35 19 Nature 2002 1 1 0.02 2.39 20 Nature 1998 1 1 0.02 0.90 21 Science 1991 1 1 0.02 0.40 22 IEEE transactions on information theory 1999 1 1 0.02 0.23 23 The democratization of American Christianity / Nathan O. Hatch 1989 1 1 0.02 0.20 24 Proceedings of the National Academy of Sciences of the United States of America 1990 1 1 0.02 0.12 25 Nature 1993 1 1 0.02 0.10 64 College & Research Libraries January 2012 The most frequently cited types of ma- terials overall were books (55%) and jour- nals (37%); and by discipline were: Arts & Humanities, books (73%) and journals (23%); Engineering, journals (64%) and books (14%); Science, journals (78%) and books (14%); and Social Sciences, journals (46%) and books (42%). The Notre Dame findings are comparable to the findings in the study of theses and dissertations by Smith at the University of Georgia. That study found book and journal citation patterns were as follows: All Disciplines, books (48%) and journals (38%); and Arts & Humanities, books (71%) and journals (19%).41 The Burright study of neurosci- ence faculty and graduate students’ work showed that cited references in their publications were 82 percent from journal articles and 8 percent from books,42 which is similar to Notre Dame’s Science group, journals (88%) and books (7%). Notre Dame’s Science group citation pattern is also comparable to Kuruppu and Moore’s study of agriculture and biology dis- sertations at Iowa State, which reported that student citations were 80.5 percent journals and 10.5 percent books.43 The effect of electronic access on cita- tion patterns has been studied by Smith (2003). She found that citations to Web sites were nonexistent in the 1991 theses and dissertations but were 3.5 percent of the citations in the 2001 set of theses and dissertations. There was also a clear shift from the use of ERIC microform to the electronic version of ERIC in the educa- tion theses and dissertations.44 Other research on undergraduates found that students cited more Web sources and fewer books than they had three years previously.45 It is difficult to measure through citation analysis precisely how big of an impact electronic access has had on researchers because researchers do not necessarily acknowledge that they used an electronic version. The effect of electronic materials on library ownership has also been studied by Smith (2003) at the University of Georgia. Smith found that the library owned more monographs and fewer periodicals in 2001 than it did in 1991, a finding that surprised her.46 Fur- ther analysis on Notre Dame dissertations will involve selecting dissertations, from a point in time before electronic access became the norm, to compare citation patterns and ownership statistics to those dissertations that were used in this study. In conclusion, citation analysis gives the library researcher a deeper look into the user’s research patterns and library re- search needs, which should be important in recommending materials for selection and deselection. A previous study of graduate students at Notre Dame found that students identified materials relevant to their research by searching a database or reading an article or book and that, if the library did not own an item, 64 per- TABLe 10 Hesburgh Libraries’ Ownership of Cited items by Disciplinary group* group Citations Owned % Owned % Top 1,000 Cited items Owned ** Arts and Humanities 10,818 7,297 63 90 Engineering 7,032 4,395 67 83 Science 5,369 4,093 76 92 Social Sciences 5,185 3,378 65 75 Overall 27,652 18,461 67 93 * There were 752 citations that were in more than one group, 702 of which were owned. ** Weighted index was applied. Relevance of Library Collections for Graduate Student Research 65 cent of them would try to get it through interlibrary loan.47 Although this citation analysis study was conducted on Notre Dame’s graduate student population, the findings should be relevant to librarians elsewhere in developing collection de- velopment policies and in thinking about budgeting for different user populations. The findings in this study indicate a need for additional funding for further development of the social sciences col- lections in the Hesburgh Libraries of Notre Dame to support graduate stu- dent research (theses and dissertation writing) and publication endeavors at the university. The marked difference in library ownership of materials cited by graduate students in the physical sciences versus those in the social sciences can be addressed through library budget- ary increases and/or adjustments. Since this study focused on graduate students from various departments and colleges at Notre Dame, it should be a fairly accurate indicator of their current and future needs and, therefore, impact decisions regard- ing collection development funding. It will enable those involved in collection development to provide and maintain high-quality collections for graduate stu- dents. This is not to suggest that libraries should have comprehensive collections to be useful to graduate students. Com- prehensive collections are impractical because of funding hardships in libraries. There are other ways to support graduate student research: for example, ordering unowned materials through interlibrary loan, assuming this is more cost-effective than purchasing a particular item. Using data such as that collected through cita- tion analysis to drive collection strategies and to develop quality collections is a bet- ter approach than striving for comprehen- sive collections. Ultimately, developing and maintaining a high-quality library collection is paramount in meeting gradu- ate student needs. Notes 1. “About Notre Dame,” University of Notre Dame, available online at http://nd.edu/aboutnd [accessed 16 September 2010]. 2. “Fact Book,” University of Notre Dame Institutional Research, available online at www. nd.edu/~instres/home3/fact_book.shtml [accessed 10 October 2010]. 3. “Chronology of Notre Dame Libraries,” Hesburgh Libraries, available online at www.library. nd.edu/about/history/timeline.shtml [accessed 16 September 2010]. 4. Linda Smith, “Citation Analysis,” Library Trends 30 (1981): 83–106. 5. Thomas E. Nisonger, “Journals in the Core Collection: Definition, Identification, and Ap- plications,” The Serials Librarian 51 (2007): 51–73. 6. See Stanley J. Wilder, “A Simple Method for Producing Core Scientific and Technical Journal Title Lists,” Library Resources & Technical Services 44 (2000): 92–96; Paul Kelsey and Tom Diamond, “Establishing a Core List of Journals for Forestry: A Citation Analysis from Faculty at Southern Universities,” College & Research Libraries 64 (2003): 357–77; C. Keith Waugh and Margie Ruppel, “Citation Analysis of Dissertation, Thesis, and Research Paper References in Workforce Educa- tion and Development,” The Journal of Academic Librarianship 30 (2004): 276–84; Marian Burright, Trudie Hahn, and Margaret Antonisse, “Understanding Information Use in a Multidisciplinary Field: A Local Citation Analysis of Neuroscience and Cognitive Science Research,” College & Re- search Libraries 66 (2005): 198–210; Yelena Pancheshnikov, “A Comparison of Literature Citations in Faculty Publications and Student Theses as Indicators of Collection Use and a Background for Collection Management at a University Library,” The Journal of Academic Librarianship, 33 (2007): 674–83; Jonathan Nabe and Andrea Imre, “Dissertation Citations in Organismal Biology at Southern Illinois University at Carbondale: Implications for Collection Development,” Issues in Science & Technology Librarianship 55 (2008); Rick Ralston, Carole Gall, and Frances A. Brahmi, “Do Local Citation Patterns Support Use of the Impact Factor for Collection Development?” Journal of the Medical Library Association 96 (2008); 374–78; Shi-Jian Gao, Wang-Zhi Yu, and Feng-Ping Luo, “Citation Analysis of PhD Thesis [sic] at Wuhan University, China,” Library Collections, Acquisi- tions, & Technical Services 33 (2009): 8–16. 7. See Erin T. Smith, “Assessing Collection Usefulness: An Investigation of Library Ownership of the Resources Graduate Students Use,” College & Research Libraries 64 (2003): 344–55; Beth Fuchs, 66 College & Research Libraries January 2012 Cristina Thomsen, Randolph Bias, and Donald Davis. “Behavioral Citation Analysis: Toward Col- lection Enhancement for Users,” College and Research Libraries 67 (2006): 304–24; Pancheshnikov, “A Comparison of Literature Citations,” 674; Conception Wilson and Carol Tenopir, “Local Citation Analysis, Publishing and Reading Patterns: Using Multiple Methods to Evaluate Faculty Use of an Academic Library’s Research Collection,” Journal of the American Society for Information Science & Technology, 59 (2008): 1393–1408. 8. Stacey Knight-Davis and Jan S. Sung, “Analysis of Citations in Undergraduate Papers,” Col- lege & Research Libraries 69 (2008): 447–58. 9. See Angela M. Gooden, “Citation Analysis of Chemistry Doctoral Dissertations: An Ohio State University Case Study,” Issues in Science and Technology Librarianship 32 (2001), available online at www.istl.org/01-fall/refereed.html [accessed 16 September 2010]; Joel Sam and S.N.B. Tackie, “Citation Analysis of Dissertations at the Department of Information Studies, University of Ghana, Legon,” African Journal of Library Archives and Information Science 17 (2007): 123–32; Pali U. Kuruppu and Debra C. Moore, “Information Use by PhD students in Agriculture and Biology: A Dissertation Citation Analysis,” portal: Libraries and the Academy 8 (2008): 387–405; Janice E. Cox, “Citation Analysis of Graduate Dental Theses References: Implications for Collection Develop- ment,” Collection Management 33 (2008): 219–34. 10. Robin N. Sinn, “A Local Citation Analysis of Mathematical and Statistical Dissertations,” Science & Technology Libraries 25 (2005): 25–37. 11. Yasar Tonta and Umut Al, “Scatter and Obsolescence of Journals Cited in Theses and Dis- sertations of Librarianship,” Library & Information Science Research 28 (2006): 281–96. 12. Kelsey and Diamond, “Establishing a Core List of Journals for Forestry,” 357; Burright, Hahn, and Antonisse, “Understanding Information Use in a Multidisciplinary Field,” 198; Wilson and Tenopir, “Local Citation Analysis, Publishing and Reading Patterns,” 1393; Pancheshnikov, “A Comparison of Literature Citations,” 674; Ralston et al., “Do Local Citation Patterns Support Use of the Impact Factor for Collection Development?” 374. 13. Kuruppu and Moore, “Information Use by PhD Students in Agriculture and Biology,” 391. 14. Sinn, “A Local Citation Analysis of Mathematical and Statistical Dissertations,” 29. 15. Burright et al., “Understanding Information Use in a Multidisciplinary Field,” 202. 16. Cox, “Citation Analysis of Graduate Dental Theses References,” 225. 17. Tonta and Al, “Scatter and Obsolescence of Journals,” 285. 18. Waugh and Ruppel, “Citation Analysis of Dissertation, Thesis, and Research Paper Refer- ences,” 279. 19. Sinn, “A Local Citation Analysis of Mathematical and Statistical Dissertations,” 29. 20. Nabe and Imre, “Dissertation Citations in Organismal Biology.” 21. Pancheshnikov, “A Comparison of Literature Citations,” 674. 22. Ralston et al., “Do Local Citation Patterns Support Use of the Impact Factor for Collection Development?” 374. 23. Gooden, “Citation Analysis of Chemistry Doctoral Dissertations.” 24. Knight-Davis and Sung, “Analysis of Citations in Undergraduate Papers,” 448. 25. Gao et al., “Citation Analysis,” 10. 26. Smith, “Assessing Collection Usefulness,” 346. 27. Fuchs et al., “Behavioral Citation Analysis,” 307. 28. Sam and Tackie, “Citation Analysis of Dissertations,” 126. 29. Kuruppu and Moore, “Information Use by PhD Students in Agriculture and Biology,” 391. 30. See Smith, “Assessing Collection Usefulness,” 346; Tonta and Al, “Scatter and Obsoles- cence of Journals,” 285; Sam and Tackie, “Citation Analysis of Dissertations,” 126; Kuruppu and Moore, “Information Use by PhD Students in Agriculture and Biology,” 391; Pancheshnikov, “A Comparison of Literature Citations,” 675. 31. See Waugh and Ruppel, “Citation Analysis of Dissertation, Thesis, and Research Paper Ref- erences,”279; Fuchs et al., “Behavioral Citation Analysis,” 307; Knight-Davis and Sung, “Analysis of Citations in Undergraduate Papers,” 448. 32. See Burright et al., “Understanding Information Use in a Multidisciplinary Field,” 202; Sinn, “A Local Citation Analysis of Mathematical and Statistical Dissertations,” 29; Ralston et al., “Do Local Citation Patterns Support Use of the Impact Factor for Collection Development?” 375. 33. See Laurel A. Haycock, “Citation Analysis of Education Dissertations for Collection Develop- ment,” Library Resources & Technical Services 48 (2004): 102–06; Cox, “Citation Analysis of Graduate Dental Theses References,” 225; Gao et al., “Citation Analysis,” 10; Nabe and Imre, “Dissertation Citations in Organismal Biology”; Wilson and Tenopir, “Local Citation Analysis, Publishing and Reading Patterns,” 1394. 34. Weight = (T/N) x C (where T = number of graduate students citing each item, N = number of dissertations in this study, and C = number of times the item was cited). Waugh and Ruppel, Citation Analysis of Dissertation, Thesis, and Research Paper References,” 280. Relevance of Library Collections for Graduate Student Research 67 35. Julie F. Pallant, SPSS Survival Manual: A Step by Step Guide to Data Analysis Using SPSS (Sydney, Australia: Allen & Unwin, 2007). 36. Smith, “Assessing Collection Usefulness,” 352. 37. Fuchs et al., “Behavioral Citation Analysis,” 317. 38. Ibid. 39. Nuria Vallmitjana and L.G. Sabate, “Citation Analysis of Ph.D. Dissertation References as a Tool for Collection Management in an Academic Chemistry Library,” College & Research Libraries 69 (2008): 72–82. 40. Fuchs et al., “Behavioral Citation Analysis,” 311. 41. Smith, “Assessing Collection Usefulness,” 348. 42. Burright et al., “Understanding Information Use in a Multidisciplinary Field,” 204. 43. Kuruppu and Moore, “Information Use by PhD Students in Agriculture and Biology,” 393. 44. Smith, “Assessing Collection Usefulness,” 354. 45. Philip M. Davis and Suzanne A. Cohen, “The Effect of the Web on Undergraduate Cita- tion Behavior 1996–1999,” Journal of the American Society for Information Science and Technology 52 (2001): 309–14. 46. Smith, “Assessing collection usefulness,” 354. 47. Jessica Kayongo and Clarence Helm, “Graduate Students and the Library: A Survey of Research Practices and Library Usage at the University of Notre Dame,” Reference & User Services Quarterly 49 (2010): 341–49.