nisonger2.indd The Perception of Library and Information Science Journals by LIS Education Deans and ARL Library Directors: A Replication of the Kohl– Davis Study Thomas E. Nisonger and Charles H. Davis Analyzing the collective opinion of presumed experts, often termed a per- ception study, is a frequently used approach for rating journals or evaluating education programs. Replicating the 1985 Kohl–Davis study, seventy-one library and information science (LIS) journals are ranked according to their mean rating on a 1 to 5 ordinal scale by deans of ALA-accredited educa- tion programs and by the directors of ARL libraries (surveyed during the summer of 2003). Comparison of the results with the 1985 study found considerable continuity in journal perceptions over the past two decades, but more so by directors than deans. A weak to moderate correlation was found between deans’ ratings and Journal Citation Reports citation scores, whereas thecorrelations between directors’ perceptions and citationdata were weak to nonexistent. The findings confirm a hierarchy of prestige among LIS journals, but the hierarchical order differs somewhat between deans and directors. lthough, in theory, every research article should be judged on its own merits, the journal in which it is pub- lished o en serves as a proxy indicator of research quality. The evaluation of schol- arly journals is important for selection and cancellation decisions by librarians, the evaluation of faculty and librarians for promotion and tenure as well as an- nual performance reviews, manuscript submission decisions by authors, moni- toring of their journals by editors and publishers, and familiarizing new doctoral students or outsiders (such as members of a university-wide promotion and tenure commi ee evaluating faculty from other departments) with a field’s journals. Thomas E. Nisonger is Professor and Charles H. Davis is a Senior Fellow, both in the School of Library and Information Science at Indiana University; e-mail: nisonge@indiana.edu and davisc@indiana.edu, respectively. All correspondence should be sent to Professor Nisonger. The authors gratefully acknowledge the assistance of Rebecca Kennedy, an MLS student and graduate assistant in Indiana University’s School of Library and Information Science (SLIS), who compiled the data; Yung-Rang Cheng, a Ph.D. student and adjunct faculty member in SLIS, who conducted the statistical analysis with SPSS; and Debora Shaw, Executive Associate Dean and Professor in SLIS, who provided valuable feedback on various dra s of the manuscript. 341 mailto:davisc@indiana.edu mailto:nisonge@indiana.edu 342 College & Research Libraries July 2005 Both journal rankings, which place a set of journals in hierarchical order according to some type of evaluative measure, and journal ratings, which calculate evaluative scores for a set of journals without placing them in explicit hierarchical order, can help evaluate the scholarly journals of a discipline. The two most frequently used journal ranking/rating criteria in library and information science (LIS) as well as other disciplines are citation data (such as impact factor or total citation count in the Institute for Scientific Information’s Journal Citation Reports) and the percep- tions of domain experts. This research replicates an analysis and ranking of LIS journals based on the perceptions of LIS education program deans and directors of large research libraries, which was published in 1985 by David F. Kohl and Charles H. Davis.1 The Kohl–Davis study, as is demonstrated in the literature review, served as a model for several subsequent perception-based rankings of LIS journals by various con- stituencies and was last replicated about a decade ago by Virgil L. P. Blake.2 The benefits and drawbacks of journal perception studies have been debated in the literature and briefly summarized by Thomas E. Nisonger.3 Proponents argue that expert perception can reflect subtle nuances of journal value not readily cap- tured by citation data or other objective measures and that perception rankings reflect the collective judgment of domain specialists whose knowledge of the field and its journals may reasonably be pre- sumed. Critics of the perception approach contend that respondents may be biased, unfamiliar with the titles they are rating or have outdated perceptions, that impor- tant titles might have been omi ed from the list for evaluation, and that the criteria on which journals are being rated may be vague or ill defined. Literature Review Nisonger identified 178 rankings or rat- ings of LIS journals published between 1952 and 1997.4 A er citation-based rank- ings (nine different citation methods were used), perception studies (25 contained in 12 published studies) were the most frequently employed approach. Other ranking methods included productivity (i.e., the number of articles contributed to an indexing or abstracting database) and readership. In fact, the earliest LIS journal evalua- tions identified in this investigation used the perception method. More than half a century ago in 1952, Alice I. Bryan, as part of the Public Library Inquiry sur- vey, rated a list of eight journals based on the percentage of 1,837 professional librarians and 461 subprofessionals who judged the title “had made very helpful contributions towards the effectiveness of their library work during the past year.”5 Mary Lee Bundy’s survey of public library directors asked them to name published articles considered “particularly good” and regular columns or features they “like especially.”6 She then listed fourteen journals according to the number of times their articles were mentioned and twelve titles in order of the times their columns or features were wri en on the survey forms. C. W. Hanson and Patricia Tilbury asked participants at the 1962 Aslib conference in the United Kingdom to list the three journals they “most look forward to see- ing” and ranked the top twenty-five by tabulating their responses.7 In order to generate a citation pool for a document delivery test, Rudolf Jacob Penner asked the deans of the seven Canadian LIS education programs to list the twenty journals they deemed “most important for research and education” in the field and then ranked twenty-four titles based on the number of times mentioned.8 Some journal rankings or ratings have been compiled from the subjective judg- ment of faculty in a single LIS education program. Charles T. Meadow and Mary Ann Zaborowski presented a list of jour- nals to four Drexel University LIS faculty and in 1979 published a ranking of the top ten journals according to the num- ber of votes received.9 Robert M. Hayes Perception of Library and Information Science Journals 343 published ratings for 140 journals based on the number of UCLA LIS faculty who identified them as “central” or “periph- eral” to their specialties.10 Kohl and Davis, whose methodology was modeled on evaluations of LIS educa- tion programs by Herbert S.White11 12 13 surveyed the sixty-six deans of schools/ institutions with ALA-accredited library programs and the eighty-five directors of ARL institutions in the fall of 1982 to determine if there were a perceived “hierarchy of prestige” among the field’s journals.14 Respondents were asked to rate a list of thirty-one journals, culled from a core journal listing by Jesse H. Shera,15 on a 1 to 5 ordinal scale concerning “how important publication in that journal was for the consideration of promotion and tenure at their institution” and to indicate in no particular order the five most prestigious journals, termed the “top five” method.16 The Kohl–Davis methodology served as a model for several subsequent journal rankings. Renee Tjoumas asked public library directors to rate a list of fi y-six periodicals on a 1 to 5 ordinal scale con- cerning their “usefulness for the perfor- mance of work-related duties.”17 Virgil L. Blake compared the ratings of fi y-five journals by LIS faculty specializing in school media with those of district-level school library media coordinators.18 Tjou- mas and Blake then compared the la er’s journal ratings by LIS school media fac- ulty with a new set of evaluations by fac- ulty specializing in public librarianship.19 Finally, Blake replicated the Kohl–Davis methodology by surveying in 1992 the deans of schools with ALA-accredited LIS education programs and the directors of ARL libraries, asking them to rate a list of fifty-seven journals.20 Note that the original Kohl–Davis article and the later studies using its methodology found, among other things, that a hierarchy of journal prestige does indeed exist and that for many journals perceptions of their prestige vary among different stakeholder groups and longitudinally over time. The most recent perception ranking of LIS journals was published in 1997 by E. E. Nkereuwem.21 Nigerian academic librarians rated a list of journals on a 0 to 10 scale according to the “quality” of their articles. Twenty-six journals were ranked based on “journal impact,” calculated by multiplication of the mean rating by the proportion of respondents sufficiently familiar with the journal to rate it plus ad- dition of the mean rating to the result. Journal rankings based on the percep- tions of subject experts (variously termed “perception,” “prestige,” or “subjective” studies) have been compiled in numerous social science disciplines and professional fields other than library and information science. Examples include political science by Michael W. Giles, Francie Mizell, and David Pa erson,22 sociology by Norval D. Glenn,23 geography by David Lee and Arthur Evans,24 criminal justice by David Shichor, Robert M. O’Brien, and David L. Decker,25 human development by J. Craig Peery and Gerald R. Adams,26 economics by Jean-Louis Malouin and J.-Francois Outreville,27 behavioral aspects of man- agement by Marian M. Extejt and Jona- than E. Smith,28 marketing by Robert H. Luke and E. Reed Doke,29 real estate by Joe Albert and P. R. Chandy,30 business ethics by Andrew C. Wicks and Robbin Derry,31 accounting by Lawrence D. Brown and Ronald J. Huefner,32 and social work by Ram A. Cnaan, Richard K. Caputo, and Yochi Shmuely.33 Typically, these studies rank a list of journals through a 4-, 5-, or 10-point quality scale. In addition to rating journals per se, these studies have investigated such questions as the correspondence between citation and perception rankings, the longitudinal stability of perception rank- ings, and the extent to which specialists give higher ratings to journals in their own areas. Methodology The methods Kohl and Davis used in the original study were replicated with an expanded set of journal titles. In early July, http:Shmuely.33 http:Nkereuwem.21 http:journals.20 http:librarianship.19 http:coordinators.18 http:method.16 http:journals.14 http:specialties.10 344 College & Research Libraries July 2005 2003, questionnaires were mailed to the fi y-six deans, directors, or department chairs of schools with ALA-accredited LIS education programs (identified through the ALA Web site) and the directors of the 120 ARL libraries (member institutions were determined through the ARL Web site and their directors identified through the member library Web sites directly linked to the ARL Web site). A second questionnaire was sent in late September to those who did not respond to the first mailing.34 Deans of ALA-accredited LIS educa- tion programs and ARL library directors were selected to replicate the Kohl–Davis study. Moreover, these populations may reasonably be assumed to be familiar with the quality of various LIS journals as well as the promotion and tenure policies at their institutions. There were two parts to the survey instrument. Part one asked respondents to rate a list of seventy-one journals on a 1 (low) to 5 (high) ordinal scale according to their perception of “how important pub- lication in each journal is for promotion and tenure at your institution.” Respon- dents were instructed not to rate (i.e., to indicate NF [not familiar]) for those titles “which you do not have enough familiar- ity with to rate.” They also were given the opportunity to suggest additional titles not on the original list that they believed should have been included. The list of seventy-one journals includ- ed the titles in the original Kohl–Davis study, if still active, and those covered in the “information and library science” sub- ject category in the 2001 Journal Citation Reports (the most current edition available when the questionnaire was designed in the early summer of 2003). A number of titles were added to give representation to all-electronic journals (e.g., First Mon- day and D-Lib Magazine) and topics that had emerged since the initial Kohl–Davis study (e.g., Internet Research). In contrast to the Kohl–Davis study, Canadian jour- nals were included on the list if they met these criteria. In applying these criteria, a few likely journals (e.g., Portal: Libraries and the Academy) were inevitably omi ed from the list. Ulrich’s Periodicals Directory and Google Web searches were used to verify that all seventy-one titles were still active and listed under the currently correct title. Of the thirty-one titles in the first Kohl–Davis study, seventeen were still being pub- lished under the same title. The others had either ceased publication, such as Wilson Library Bulletin, Drexel Library Quarterly, and the Library of Congress Quarterly Jour- nal or changed names. For example, Journal of Library History, Philosophy, & Comparative Librarianship became Libraries & Culture and RQ changed to Reference & User Services Quarterly. In some cases, the title changes were relatively minor, such as from the Journal of the American Society for Information Science to the Journal of the American Society for Information Science and Technology (JASIST). In instances of recent name changes, the former title also was indicated on the questionnaire to avoid confusion. In part two of the survey, the ARL library directors and LIS education program heads were asked to list, in no particular order, the five most prestigious journals “to have published in for promo- tion and tenure purposes at your institu- tion.” No further instructions were given, so respondents were free to list titles on the original list, titles not so listed, or a combination of the two categories. Kohl and Davis as well as other investigators have referred to this technique as the “top five method.” It is the authors’ understanding that all schools with ALA-accredited programs currently have a promotion and tenure system in place. The questionnaire sent to directors of ARL libraries differed slightly from that mailed to the heads of LIS education programs, as it asked whether their institution had promotion and ten- ure systems and instructed respondents whose institutions lacked these policies to rate the seventy-one titles and list the top five “according to the prestige asso- http:mailing.34 Perception of Library and Information Science Journals 345 ciated with publishing in it.” Finally, the questionnaires to both groups concluded with an open-ended question asking for general comments. This investigation’s analysis is based on the mean rating of each journal by LIS deans and the mean ratings by library di- rectors. Two methods were used to calcu- late the mean ratings. In the first method, used in the Kohl–Davis study and by Blake, blank responses were counted as zero, predicated on the assumption that a respondent’s nonfamiliarity with a title reflected negatively on its status. In the second method, blank responses were simply disregarded so that journals highly rated by smaller numbers of respondents would not be disadvantaged. Results A er a second mailing in September 2003, thirty-seven usable responses from LIS deans (a 66.1% response rate) and fi y-six usable responses from library directors (a 46.7% rate) had been received by the end of December 2003, for a 52.8 percent over- all response rate. These rates are some- what lower than the 71.1 percent response rate from deans and the 50.6 percent rate from directors in the Kohl–Davis study 35 and the 75.8 and 59 percent response rates from deans and directors respectively in Blake.36 Yet, this investigation’s overall response rate was higher than those in numerous perception-based journal rank- ings in LIS and other disciplines, such as 22.2 percent in Bundy,37 33 percent in Hanson and Tilbury,38 42 percent in Cnaan, Caputo, and Shmuely,39 49.3 percent in Brown and Huefner,40 and 25.5 percent in Wicks and Derry.41 Mean Ratings of the Journals Table 1 displays the seventy-one titles in rank order according to their mean rating by ARL directors and then by LIS educa- tion deans, with “not familiar” or blank responses counted as 0. Table 2 follows the identical format, but the NF or blank responses were disregarded in calculating each journal’s mean rating. There was a .735 correlation in the directors’ ratings by the two methods and .737 for the deans. However, some journal rankings changed between the two tables. In the deans’ ratings, Sciento- metrics increases from 43rd to 7thand MIS Quarterly from a three-way tie for 45th to 10th from table 1 to table 2, reflecting the fact that many respondents are unfamil- iar with these titles, but those who are familiar with them rate them highly. In contrast, some well-known titles receiving midlevel ratings from a large proportion of the respondents decrease in rank from table 1 (where they are not penalized by 0s from nonraters) to table 2. For instance, in the directors’ ratings, American Libraries falls from 18th to 66th and Library Journal declines from tied for 11th to 45th. The results are mixed regarding all- electronic journals. Some titles did not fare especially well. For example, Cyber- metrics, ranked 70th, three-way tie for 46th (throughout this article, a journal’s rank in table 1 is reported first, followed by the rank in table 2 with tied positions so indicated) by directors and 70th, 56th (tied) by deans, and First Monday ranked 47th, 57th by directors and 52th, 59th by deans. Other all-electronic titles made quite respectable showings in some rankings. School Library Media Research ranked 16th, 16th by deans, but its 60th, 68th ranking by directors is no surprise given its focus on school libraries. D-Lib Magazine placed in position 22 in table 1’s directors’ rankings, but less impressive is its three-way tie for 46 by directors in table 2 and the 57th, 54th rankings by deans. In summary, firm conclusions regarding scholarly acceptance of electronic journals cannot be reached due to the small sample size. One should also note that eighteen directors listed the all-electronic journal Libres (second only to Portal: Libraries and the Academy) as among those that should have been included on the list of seventy- one titles. (See subsequent subsection on additionally suggested titles.) Exactly half (28 of 56) of the responding ARL directors reported their library had http:Derry.41 http:Blake.36 346 College & Research Libraries July 2005 TABLE 1 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are counted as 0. ARL Directors LIS Deans Rank Journal Title Mean Rating Rank Journal Title Mean Rating 1 College & Research Libraries 4.46 1 JASIST 4.41 2 Library Trends 4.41 1 Library Quarterly 4.41 3 Journal of Academic Librarianship 4.39 3 ARIST 3.97 4 Library Quarterly 4.25 3 Library & Information Science Research 3.97 5 Reference & User Services Quarterly 4.04 5 Journal of Documentation 3.81 6 Library Resources & Technical Services 3.86 6 Library Trends 3.62 7 JASIST 3.82 7 Journal of Academic Librarianship 3.49 8 ARIST 3.70 7 Information Processing & Management 3.49 9 Library Coll. Acq. & Tech. Services 3.57 9 ASIST Proceedings 3.46 10 Information Technology & Libraries 3.52 10 Reference & User Services Quarterly 3.43 11 Collection Management 3.39 11 College & Research Libraries 3.41 11 Library Journal 3.39 12 J. Education for Library & Info. Science 3.30 13 Reference Services Review 3.27 13 Libraries & Culture 3.24 14 Government Information Quarterly 3.09 14 Journal of the Medical Library Association 3.19 15 Journal of the Medical Library Association 3.04 15 Library Resources & Technical Services 3.11 16 Aslib Proceedings 2.91 16 School Library Media Research 3.08 17 Libri 2.84 17 Journal of Information Science 2.95 18 American Libraries 2.82 17 Libri 2.95 19 Information Outlook 2.79 19 J. Amer. Medical Informatics Association 2.92 20 Journal of Documentation 2.73 20 School Library Journal 2.81 Perception of Library and Information Science Journals 347 TABLE 1 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are counted as 0. ARL Directors LIS Deans Rank Journal Title Mean Rating Rank Journal Title Mean Rating 20 Library & Information Science Research 2.73 21 Aslib Proceedings 2.78 22 D-Lib Magazine 2.71 22 Canadian J. Info. & Library Science 2.73 23 J. Education for Library & Info. Science 2.66 23 Information Research 2.70 24 Libraries & Culture 2.57 23 Public Libraries 2.70 25 Journal of Government Information 2.52 25 Government Information Quarterly 2.68 26 Journal of Information Science 2.38 25 Information Technology & Libraries 2.68 26 Journal of Scholarly Publishing 2.38 27 Journal of Scholarly Publishing 2.49 28 J. Librarianship & Information Science 2.34 28 Information Outlook 2.46 29 Online 2.30 29 J. Librarianship & Information Science 2.41 30 Interlending & Document Supply 2.18 30 Journal of Government Information 2.38 30 Law Library Journal 2.18 30 Law Library Journal 2.38 32 Microform & Imaging Review 2.14 30 Reference Services Review 2.38 33 ASIST Proceedings 2.11 33 Online Information Review 2.24 34 Information & Management 2.05 34 Library Journal 2.19 35 Canadian J. Info. & Library Science 2.02 35 Collection Management 2.16 35 Econtent 2.02 36 The Information Society 2.14 35 Journal of Information Technology 2.02 36 Journal of Information Ethics 2.14 38 Library & Information Science 2.00 38 International J. Information Management 2.11 39 Harvard Library Bulletin 1.86 39 Library Coll. Acq. & Tech. Services 2.08 39 International J. Information Management 1.86 40 Library & Information Science 2.05 348 College & Research Libraries July 2005 TABLE 1 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are counted as 0. ARL Directors LIS Deans Rank Journal Title Mean Rating Rank Journal Title Mean Rating 41 Information Processing & Management 1.77 40 Online 2.05 42 Journal of Management Information Systems 1.75 42 Journal of Management Information Systems 2.00 43 Journal of Information Ethics 1.73 43 Scientometrics 1.97 44 J. Amer. Medical Informatics Association 1.71 44 Information & Management 1.89 45 School Library Journal 1.68 45 Journal of Information Technology 1.86 46 Online Information Review 1.64 45 Knowledge Organization 1.86 47 First Monday 1.61 45 MIS Quarterly 1.86 48 Information Research 1.57 48 Information Systems Research 1.84 48 International Information & Library Review 1.57 49 American Libraries 1.78 48 Scientist 1.57 49 Internet Research 1.78 51 International J. Geographical Info. Science 1.52 51 International Information & Library Review 1.70 51 Social Science Information 1.52 52 First Monday 1.62 53 Public Libraries 1.50 53 Journal of Health Communication 1.58 54 Information Systems Research 1.48 54 Microform & Imaging Review 1.57 54 International Journal of Legal Information 1.48 55 Harvard Library Bulletin 1.46 56 Electronic Library 1.45 55 Program: Electronic Library & Info. Systems 1.46 57 The Information Society 1.41 57 D-Lib Magazine 1.43 58 Internet Research 1.39 58 Electronic Library 1.32 58 Social Science Computer Review 1.39 58 International Journal of Legal Information 1.32 60 School Library Media Research 1.38 60 Econtent 1.27 Perception of Library and Information Science Journals 349 TABLE 1 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are counted as 0. ARL Directors LIS Deans Rank Journal Title Mean Rating Rank Journal Title Mean Rating 61 Information Systems Journal 1.34 60 Information Systems Journal 1.27 62 Knowledge & Organization 1.32 60 Telecommunications Policy 1.27 63 MIS Quarterly 1.29 63 Interlending & Document Supply 1.19 64 Telecommunications Policy 1.27 63 Social Science Information 1.19 65 Zeitschrift B. B. 1.16 65 Zeitschrift B. B. 1.14 66 Journal of Health Communication 1.11 66 Scientist 0.89 67 Restaurator 0.96 67 Social Science Computer Review 0.86 68 Scientometrics 0.91 68 International J. Geographical Info. Science 0.78 69 Program: Electronic Library & Info. Systems 0.89 69 Restaurator 0.62 70 Cybermetrics 0.79 70 Cybermetrics 0.59 71 NFD Info. Wissenschaft und Praxis 0.45 71 NFD Info. Wissenschaft und Praxis 0.41 a tenure system. The ratings by the two categories (directors in libraries with a tenure system and libraries without ten- ure) were strongly correlated, .925 when blank responses are counted as 0 and .804 when they are disregarded. Thus, we conclude further analysis regarding the possible influence of a tenure system on library director journal perceptions is unnecessary. Differences between Deans’ and Directors’ Ratings and Rankings T-tests indicated that the mean ratings by deans and directors differed at the .05 significance level for twenty-six journals, 36.6 percent of the seventy-one listed, when nonrated titles are counted as 0, and for twenty-nine titles, 40.8 percent of those listed, when nonrated journals are disregarded.42 For the names of these journals, see tables 3 and 4. Among the titles on both lists are such well-known publications as American Libraries, Library Journal, College & Research Libraries, Journal of Academic Librarianship, and Information Processing & Management. In the 1985 Kohl–Davis study, deans and directors differed on eleven of thirty-one titles (35.5%),43 eight of which are listed in table 3 and nine in table 4 (with School Library Media Research counted as equivalent to its earlier title School Library Media Quarterly). Although a purist might argue that mean scores are of more statistical impor- http:disregarded.42 350 College & Research Libraries July 2005 TABLE 2 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are not considered. ARL Directors LIS Deans Rank Journal Title Mean Rating Rank Journal Title Mean Rating 1 College & Research Libraries (53) 4.72 1 JASIST (35) 4.66 2 Library Trends (55) 4.49 2 Library Quarterly (36) 4.53 3 JASIST (48) 4.46 3 Information Processing & Management (30) 4.30 4 Journal of Academic Librarianship (55) 4.39 4 Library & Info. Science Research (35) 4.20 5 Reference & User Services Quarterly (52) 4.35 5 Journal of Documentation (34) 4.15 6 Library Quarterly (55) 4.33 6 ARIST (36) 4.08 7 Information Technology & Libraries (47) 4.19 7 Scientometrics (18) 4.06 8 Journal of the Medical Lib. Association (41) 4.15 8 Library Trends (34) 3.94 8 Library Resources & Technical Services (52) 4.15 9 J. Amer. Medical Informatics Assoc. (28) 3.86 10 Reference Services Review (45) 4.07 10 MIS Quarterly (18) 3.83 11 ARIST (51) 4.06 11 Libraries & Culture (32) 3.75 12 Library Coll. Acq. & Tech. Services (50) 4.00 12 College & Research Libraries (34) 3.71 13 Library & Information Science Research (39) 3.92 12 Lib. Resources & Technical Services (31) 3.71 14 Journal of Scholarly Publishing (34) 3.91 14 Information Research (27) 3.70 15 Libraries & Culture (37) 3.89 15 Journal of Academic Librarianship (35) 3.69 16 J. Amer. Medical Informatics Assoc. (25) 3.84 16 School Library Media Research (31) 3.68 17 Journal of Documentation (40) 3.83 17 Reference & User Services Quarterly (35) 3.63 18 Journal of Information Science (35) 3.80 18 Telecommunications Policy (13) 3.62 19 Information Systems Research (22) 3.77 19 The Information Society (22) 3.59 19 Journal of Information Technology (30) 3.77 20 Information Systems Research (19) 3.58 Perception of Library and Information Science Journals 351 TABLE 2 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are not considered. ARL Directors LIS Deans Rank 21 Journal Title J. Librarianship & Information Science (35) Mean Rating 3.74 Rank 20 Journal Title Journal of the Medical Lib. Association (33) Mean Rating 3.58 22 Collection Management (51) 3.73 22 ASIST Proceedings (36) 3.56 23 International J.Geographical Info. Sci. (23) 3.70 23 Journal of Scholarly Publishing (26) 3.54 23 Law Library Journal (33) 3.70 24 Journal of Information Science (31) 3.52 23 Libri (43) 3.70 24 J. Management Information Systems (21) 3.52 26 Government Information Quarterly (47) 3.68 26 Zeitschrift B. B. (12) 3.50 27 Information Processing & Management (27) 3.67 27 Knowledge Organization (20) 3.45 28 Journal of Government Information (39) 3.62 28 J. Education for Lib. & Info. Science (36) 3.39 29 Library & Information Science (31) 3.61 29 J. Librarianship & Information Science (27) 3.30 30 Information Systems Journal (21) 3.57 29 Library & Information Science (23) 3.30 31 J. Education for Library & Info. Science (42) 3.55 31 Journal of Information Technology (21) 3.29 31 Social Science Computer Review (22) 3.55 32 Library Coll. Acq. & Tech. Services (24) 3.21 33 Aslib Proceedings (46) 3.54 32 Libri (34) 3.21 34 Information Research (25) 3.52 34 Program: Electronic Lib.& Info. Systs. (17) 3.18 34 International Information & Lib. Review (25) 3.52 35 Law Library Journal (28) 3.14 34 Knowledge Organization (21) 3.52 35 Social Science Information (14) 3.14 34 Scientist (25) 3.52 37 Aslib Proceedings (33) 3.12 38 Information & Management (33) 3.48 37 International J. Info. Management (25) 3.12 39 Information Outlook (45) 3.47 37 International J. of Legal Information (16) 3.12 352 College & Research Libraries July 2005 TABLE 2 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are not considered. ARL Directors LIS Deans Rank 39 Journal Title International J. Information Management (30) Mean Rating 3.47 Rank 40 Journal Title Government Information Quarterly (32) Mean Rating 3.09 41 Journal of Information Ethics (28) 3.46 40 Information Technology & Libraries (32) 3.09 42 Journal of Health Communication (18) 3.44 42 Canadian J. Info. & Library Science (33) 3.06 43 The Information Society (23) 3.43 43 Information & Management (23) 3.04 44 Social Science Information (25) 3.40 43 Journal of Information Ethics (26) 3.04 45 Library Journal (53) 3.39 45 Journal of Government Information (29) 3.03 46 Cybermetrics (13) 3.38 45 Public Libraries (33) 3.03 46 D-Lib Magazine (45) 3.38 45 Reference Services Review (29) 3.03 46 J. Management Information Systems (29) 3.38 48 Internet Research (22) 3.00 49 International J. of Legal Information (25) 3.32 49 School Library Journal (35) 2.97 50 Online (39) 3.31 50 Information Systems Journal (16) 2.94 51 Interlending & Document Supply (37) 3.30 51 Journal of Health Communication (20) 2.93 52 ASIST Proceedings (36) 3.28 52 Social Science Computer Review (11) 2.91 53 Internet Research (24) 3.25 53 Collection Management (28) 2.86 54 Canadian J. Info. & Library Science (35) 3.23 54 D-Lib Magazine (19) 2.79 54 Econtent (35) 3.23 55 Information Outlook (33) 2.76 54 Telecommunications Policy (22) 3.23 56 Cybermetrics (8) 2.75 57 First Monday (28) 3.21 56 Scientist (12) 2.75 58 Scientometrics (16) 3.19 58 International Info. & Lib. Review (23) 2.74 59 Restaurator (17) 3.18 59 First Monday (22) 2.73 60 MIS Quarterly (23) 3.13 60 Online (28) 2.71 Perception of Library and Information Science Journals 353 TABLE 2 Average Rating of Journal Prestige in Terms of Value for Tenure and Promotion by Directors and Deans: “Not familiar” and blank responses are not considered. ARL Directors LIS Deans Rank 61 Journal Title Electronic Library (26) Mean Rating 3.12 Rank 61 Journal Title Online Information Review (31) Mean Rating 2.68 62 Zeitschrift B. B. (21) 3.10 62 Electronic Library (20) 2.45 63 Microform & Imaging Review (40) 3.00 63 International J. Geographical Info. Sci. (12) 2.42 64 Online Information Review (31) 2.97 63 Microform & Imaging Review (24) 2.42 65 Program: Electronic Lib.& Info. Systems (17) 2.94 65 Interlending & Document Supply (19) 2.32 66 American Libraries ((55) 2.87 66 Library Journal (35) 2.31 67 Harvard Library Bulletin (37) 2.81 67 Restaurator (10) 2.30 68 School Library Media Research (33) 2.33 68 Econtent (21) 2.24 69 School Library Journal (41) 2.29 69 NFD Info. Wissenschaft und Praxis (7) 2.14 70 NFD Info. Wissenschaft und Praxis (11) 2.27 70 Harvard Library Bulletin (26) 2.08 70 Public Libraries (37) 2.27 71 American Libraries (35) 1.89 The number of respondents who rated the title (i.e., did not have blank or NF responses), is given in parentheses. tance than ranking position, the la er can nevertheless illustrate relative differences in perceptions between the two groups. In table 1, only one journal, Library Quarterly, ranks among the five highest in both the deans’ and directors’ ratings, but six of the top ten overlap between the two groups: Library Trends, Journal of Academic Librari- anship, Library Quarterly, Reference & Users Services Quarterly, JASIST, and the Annual Review of Information Science and Technol- ogy (ARIST). In table 2, only JASIST ranks in both groups’ top five and only three titles, JASIST, Library Trends, and Library Quarterly, overlap in the top ten. Deans, not unexpectedly, tended to rate information science journals higher than did directors. For example, they ranked Scientometrics 43rd, 7th compared to 68th, 58th by directors and Information Processing & Manage- ment 7th (tied), 3rd contrasted to 41st, 27th by directors It is no surprise that directors rated practitioner-oriented journals higher than did deans. Library Collections, Acquisitions, & Technical Services was ranked 9th, 12th by direc- tors contrasted to 39th, 32nd (tied) by deans, whereas Information Technology & Libraries was ranked 10th, 7th by directors and 25th (tied), 40th (tied) by deans. Overall, there was a .700 correlation between deans’ and directors’ ratings 354 College & Research Libraries July 2005 when blank responses are counted as 0 (table 1). This is somewhat lower than the .791 correlation in the first Kohl–Davis (1985) study, which calculated nonre- sponses as 0s, suggesting a longitudinal decline in the agreement between these two constituencies. With blank responses disregarded (table 2), the correlation between directors’ and deans’ ratings decreases to .545, demonstrating there is less agreement when journal famil- iarity is eliminated as a variable. (A longitudinal comparison with the earlier investigation is impossible because it did not use this method of calculating mean scores.) TABLE 3 Journals Whose Ratings by Directors and Deans Differed Significantly: “Not familiar” and blank responses are counted as 0. Journals Directors’ Rating Deans’ Rating Significance Level American Libraries 2.82 1.78 <.01 Canadian Journal of Information & Library Science 2.02 2.73 .044 Collection Management 3.39 2.16 <.01 College & Research Libraries 4.46 3.41 <.01 D-Lib Magazine 2.71 1.43 .001 Econtent 2.02 1.27 .030 Information Processing & Management 1.77 3.49 <.01 Information Research 1.57 2.70 .006 Information Technology & Libraries 3.52 2.68 .013 Interlending & Document Supply 2.18 1.19 .005 International Journal of Geographical Info. Science 1.52 0.78 .050 Journal of Academic Librarianship 4.39 3.49 <.01 Journal of Documentation 2.73 3.81 .005 Journal of the American Medical Informatics Association 1.71 2.92 .005 Library & Information Science Research 2.73 3.97 .002 Library Collections, Acquisitions, & Technical Services 3.57 2.08 <.01 Library Journal 3.39 2.19 <.01 Library Resources & Technical Services 3.86 3.11 .020 Library Trends 4.41 3.62 .004 ASIST Proceedings 2.11 3.46 <.01 Public Libraries 1.50 2.70 <.01 Reference & User Services Quarterly 4.04 3.43 .031 Reference Services Review 3.27 2.38 .014 School Library Journal 1.68 2.81 <.01 School Library Media Research 1.38 3.08 <.01 Scientometrics 0.91 1.97 .008 Perception of Library and Information Science Journals 355 TABLE 4 Journals Whose Ratings by Directors and Deans Differed Significantly: “Not familiar” and blank responses are not considered. Journals Directors’ Rating Deans’ Rating Significance Level American Libraries 2.87 1.89 <.01 Collection Management 3.73 2.86 <.01 College & Research Libraries 4.72 3.71 <.01 Econtent 3.23 2.24 .01 Electronic Library 3.12 2.45 .049 Government Information Quarterly 3.68 3.09 .021 Harvard Library Bulletin 2.81 2.08 .011 Information Outlook 3.47 2.76 .001 Information Processing & Management 3.67 4.30 .018 Information Technology & Libraries 4.19 3.09 <.01 Interlending & Document Supply 3.30 2.32 <.01 International Information & Library Review 3.52 2.74 .021 International Journal of Geographical Info. Science 3.70 2.42 .004 Journal of Academic Librarianship 4.39 3.69 .001 Journal of Government Information 3.62 3.03 .017 Journal of the Medical Library Association 4.15 3.58 .021 Law Library Journal 3.70 3.14 .031 Library Collections, Acquisitions, & Technical Services 4.00 3.21 .003 Library Journal 3.39 2.31 <.01 Library Resources & Technical Services 4.15 3.71 .040 Library Trends 4.49 3.94 .011 Microform & Imaging Review 3.00 2.42 .013 Online 3.31 2.71 .007 Public Libraries 2.27 3.03 .008 Reference & User Services Quarterly 4.35 3.63 <.01 Reference Services Review 4.07 3.03 <.01 School Library Journal 2.29 2.97 .011 School Library Media Research 2.33 3.68 <.01 Scientometrics 3.19 4.06 .038 Internal Consensus among Deans and among Directors In order to examine the consensus among respondents, the two most frequently mentioned rating levels were summed and divided by the total number of re- sponses. For illustration, thirty-three of thirty-seven deans rated Journal of the American Society for Information Science and Technology as 4th or 5th, equaling an internal consensus of 89.3 percent. This technique, termed a “heuristic approach,” 356 College & Research Libraries July 2005 TABLE 5 Degree of Internal Consensus by ARL Library Directors in Journal Ratings: “Not familiar” and blank responses are counted as 0. Journal Title Top Adjacent Totals1 Percentage of Total Standard Deviation Journal of Academic Librarianship 50 (4,5) 89.3 0.846 Library Trends 50 (4,5) 89.3 1.023 College & Research Libraries 49 (4,5) 87.5 1.250 NFD Information—Wissenschaft und Praxis 49 (0,1) 87.5 1.043 Library Quarterly 48 (4,5) 85.7 1.116 Cybermetrics 44 (0,1) 78.6 1.522 Journal of the American Society for Info. Science & Technology 44 (4,5) 78.6 1.759 Reference & User Services Quarterly 43 (4,5) 76.8 1.348 Restaurator 43 (0,1) 76.8 1.695 Scientometrics 42 (0,1) 75.0 1.587 Library Resources & Technical Services 40 (4,5) 71.4 1.394 Program: Electronic Library & Information Systems 40 (0,1) 71.4 1.473 Zeitschrift fur Bibliothekswesen und Bibliographie 39 (0,1) 69.6 1.735 ARIST 38 (4,5) 67.9 1.451 Information Technology & Libraries 38 (4,5) 67.9 1.716 Journal of Health Communication 38 (0,1) 67.9 1.713 Collection Management 37 (3,4) 66.1 1.358 Information Outlook 36 (3,4) 64.3 1.569 Information Systems Journal 35 (0,1) 62.5 1.842 Knowledge Organization 35 (0,1) 62.5 1.790 Library Collections, Acquisitions & Technical Services 35 (4,5) 62.5 1.605 Telecommunications Policy 35 (0,1) 62.5 1.711 The Information Society 34 (0,1) 60.7 1.827 Information Systems Research 34 (0,1) 60.7 1.935 MIS Quarterly 34 (0,1) 60.7 1.724 School Library Media Research 34 (0,1) 60.7 1.496 Social Science Computer Review 34 (0,1) 60.1 1.836 Electronic Library 33 (0,1) 58.9 1.768 International Journal of Geographical Information Science 33 (0,1) 58.9 1.945 Public Libraries 33 (0,1) 58.9 1.489 International Information & Library Review 32 (0,1) 57.1 1.915 Perception of Library and Information Science Journals 357 TABLE 5 Degree of Internal Consensus by ARL Library Directors in Journal Ratings: “Not familiar” and blank responses are counted as 0. Journal Title International Journal of Legal Information Top Adjacent Totals1 32 (0,1) Percentage of Total 57.1 Standard Deviation 1.799 Internet Research 32 (0,1) 57.1 1.723 Reference Services Review 32 (4,5) 57.1 1.804 First Monday 31 (0,1) 55.4 1.836 Information Research 31 (0,1) 55.4 1.877 Journal of the American Medical Informatics Association 31 (0,1) 55.4 2.033 Journal of the Medical Library Association 31 (4,5) 55.4 2.036 Library Journal 31 (4,5) 55.4 1.216 Scientist 31 (0,1) 55.4 1.925 Social Science Information 31 (0,1) 55.4 1.809 Aslib Proceedings 30 (3,4) 53.6 1.676 Online 30 (3,4) 53.6 1.705 American Libraries 29 (2,3) 51.8 1.252 Information Processing & Management 29 (0,1) 51.8 1.991 Library & Information Science Research 29 (4,5) 51.8 1.968 Government Information Quarterly 28 (3,4) 50.0 1.676 Interlending & Document Supply 28 (3,4) 50.0 1.759 Journal of Information Ethics 28 (0,1) 50.0 1.921 Journal of Documentation 27 (3,4) 48.2 1.921 Journal of Government Information 27 (3,4) 48.2 1.849 Journal of Management Information Systems 27 (0,1) 48.2 1.832 Information & Management 27 (3,4) 48.2 1.833 Library & Information Science 27 (3,4) 48.2 1.888 Microform & Imaging Review 27 (3,4) 48.2 1.577 Online Information Review 27 (0,1) 48.2 1.678 School Library Journal 27 (0,1) 48.2 1.390 Econtent 26 (3,4) 46.4 1.732 Harvard Library Bulletin 26 (0,1) 46.4 1.667 International Journal of Information Management 26 (0,1) 46.4 1.843 Journal of Information Technology 26 (0,1) 46.4 1.977 Journal of Scholarly Publishing 26 (4,5) 46.4 2.068 Libraries & Culture 26 (3,4) 46.6 1.980 Journal of Education for Library & Information Science 25 (3,4) 44.6 1.852 358 College & Research Libraries July 2005 TABLE 5 Degree of Internal Consensus by ARL Library Directors in Journal Ratings: “Not familiar” and blank responses are counted as 0. Journal Title Journal of Information Science Top Adjacent Totals1 25 (3,4) Percentage of Total 44.6 Standard Deviation 1.978 Libri 25 (3,4) 44.6 1.837 Canadian Journal of Information & Library Science 24(0,1) 42.9 1.804 D-Lib Magazine 24 (3,4) 42.9 1.755 Journal of Librarianship & Information Science 24 (3,4) 42.9 1.966 Law Library Journal 23 (0,1) 41.1 1.974 ASIST Proceedings 23 (0,1) 41.1 1.836 1The number of responses in the two adjacent rating categories (0 through 5) receiving the highest number of ratings with the two categories indicated in parentheses rather than standard deviation was used in Kohl and Davis as a measure of consen- sus because of its “intuitive clarity” and the fact that it is not skewed by extreme scores. They considered internal consen- sus to exist for any title with a score of 50.0 percent or higher.44 The internal consensus among the ARL directors is presented in table 5, calculated when blank responses count 0, and in table 6, calculated with blank responses disregarded. Both tables list the journals in descending order beginning with the highest degree of consensus. Tables 7 and 8 follow the same format for analyzing internal consensus in the deans’ ratings. To further illustrate the calculation of the internal consensus scores (and the different calculation methods for tables 5 and 6 contrasted with tables 7 and 8), let us examine the directors’ ratings for Cybermetrics: Rated as 0 (i.e., did not rate) = 43 Rated as 1 = 1 Rated as 2 = 0 Rated as 3 = 7 Rated as 4 = 3 Rated as 5 = 2 For table 5, which considers nonrating responses as 0, the two adjacent categories with the largest number of ratings are 0 (43 responses) and 1 (1 response) for a total of forty-four out of the fi y-six directors, the internal consensus is 78.6 percent. In table 6, the forty-three nonrat- ing responses are disregarded. Therefore, the two adjacent categories with the most frequent number of responses are 3 (7 responses) and 4 (3) responses for a total of 10. Cybermetrics’ internal consensus in table 6 is thus calculated as 76.9 percent (10 of the 13 responses in the analysis). Using 50.0 percent as the threshold and counting nonresponses as 0, as done by Kohl and Davis, the directors achieved consensus on forty-nine titles (69.0% of the 71) and the deans displayed consensus on fi y-four titles (76.1%). The degree of consensus among directors is somewhat higher than the 64.5 percent figure (20 of 31) in the original study, but consensus among deans is lower than the 87.1 percent level (27 of 31) from the first investigation.45 When blank responses are disregarded, a method not used by Kohl and Davis, the directors’ consensus increases to 98.6 percent (70 of 71) and the deans’ consensus rises to 100 percent. Note that higher consensus levels are al- most inevitable with the second method because there are only five possibilities rather than six. http:investigation.45 http:higher.44 Perception of Library and Information Science Journals 359 TABLE 6 Degree of Internal Consensus by ARL Library Directors in Journal Ratings: “Not familiar” and blank responses are not considered. Journal Title Most Frequent Adjacent Totals1 Percentage of Total Standard Deviation College & Research Libraries 49 (4,5) 92.5 0.662 Journal of the American Society for Info. Science & Technology 44 (4,5) 91.7 0.849 Library Trends 50 (4,5) 90.9 0.836 Journal of Academic Librarianship 50 (4,5) 89.3 0.846 Library Quarterly 48 (4,5) 87.3 0.963 Library & Information Science 27 (3,4) 87.1 0.715 Reference & User Services Quarterly 43 (4,5) 82.7 0.764 Information & Management 27 (3,4) 81.8 0.795 Knowledge Organization 17 (3,4) 81.0 0.814 Information Technology & Libraries 38 (4,5) 80.9 0.798 Information Outlook 36 (3,4) 80.0 0.815 International Journal of Information Management 24 (3,4) 80.0 0.819 Journal of Information Technology 24 (3,4) 80.0 0.774 Cybermetrics 10 (3,4) 76.9 1.044 Library Resources & Technical Services 40 (4,5) 76.9 0.916 Online 30 (3,4) 76.9 0.893 Journal of Scholarly Publishing 26 (4,5) 76.5 0.965 Interlending & Document Supply 28 (3,4) 75.7 0.968 Journal of the Medical Library Association 31 (4,5) 75.6 0.989 ARIST 38 (4,5) 74.5 0.904 Library & Information Science Research 29 (4,5) 74.4 0.900 Econtent 26 (3,4) 74.3 0.910 Information Systems Research 16 (3,4) 72.7 0.869 Social Science Computer Review 16 (3,4) 72.7 0.912 Collection Management 37 (3,4) 72.5 0.874 Social Science Information 18 (3,4) 72.0 0.913 Journal of Information Science 25 (3,4) 71.4 0.868 Reference Services Review 32 (4,5) 71.1 0.863 Internet Research 17 (3,4) 70.8 0.897 Libraries & Culture 26 (3,4) 70.3 0.843 Library Collections, Acquisitions & Technical Services 35 (4,5) 70.0 1.069 The Information Society 16 (3,4) 69.6 1.037 MIS Quarterly 16 (2,3) 69.6 1.180 360 College & Research Libraries July 2005 TABLE 6 Degree of Internal Consensus by ARL Library Directors in Journal Ratings: “Not familiar” and blank responses are not considered. Journal Title Journal of Government Information Most Frequent Adjacent Totals1 27 (3,4) Percentage of Total 69.2 Standard Deviation 0.935 Journal of Management Information Systems 20 (3,4) 69.0 0.942 Journal of Librarianship & Information Science 24 (3,4) 68.6 0.919 Telecommunications Policy 15 (3,4) 68.2 1.020 Information Research 17 (3,4) 68.0 0.963 International Journal of Legal Information 17 (3,4) 68.0 1.030 Journal of the American Medical Informatics Association 17 (4,5) 68.0 0.987 Journal of Documentation 27 (3,4) 67.5 0.958 Microform & Imaging Review 27 (3,4) 67.5 0.934 Information Systems Journal 14 (3,4) 66.7 0.978 Journal of Health Communication 12 (3,4) 66.7 0.984 Law Library Journal 22 (3,4) 66.7 0.951 Canadian Journal of Information & Library Science 23 (3,4) 65.7 1.114 Aslib Proceedings 30 (3,4) 65.2 1.069 Program: Electronic Library & Information Systems 11 (2,3) 64.7 1.029 Scientometrics 10 (3,4) 62.5 1.223 Online Information Review 19 (2,3) 61.3 1.048 ASIST Proceedings 22 (3,4) 61.1 1.162 International Journal of Geographical Information Science 14 (3,4) 60.9 1.020 International Information & Library Review 15 (3,4) 60.0 1.122 Government Information Quarterly 28 (3,4) 59.6 1.065 Journal of Education for Library & Information Science 25 (3,4) 59.5 1.173 School Library Journal 24 (1,2) 58.5 1.101 Libri 25 (3,4) 58.1 1.081 Electronic Library 15 (3,4) 57.7 1.211 School Library Media Research 19 (1,2) 57.6 1.242 First Monday 16 (3,4) 57.1 1.228 Public Libraries 21 (1,2) 56.8 1.262 Information Processing & Management 15 (3,4) 55.6 1.074 Perception of Library and Information Science Journals 361 TABLE 6 Degree of Internal Consensus by ARL Library Directors in Journal Ratings: “Not familiar” and blank responses are not considered. Journal Title Library Journal Most Frequent Adjacent Totals1 31 (4,5) Percentage of Total 55.4 Standard Deviation 1.216 NFD Information—Wissenschaft und Praxis 6 (1,2) 54.5 1.191 Harvard Library Bulletin 20 (3,4) 54.1 1.221 D-Lib Magazine 24 (3,4) 53.3 1.248 Restaurator 9 (4,5) 52.9 1.551 American Libraries 29 (2,3) 52.7 1.203 Zeitschrift fur Bibliothekswesen und Bibliographie 11 (4,5) 52.4 1.411 Journal of Information Ethics 14 (3,4) 50.0 1.138 Scientist 14 (4,5) 45.2 1.159 1The number of responses in the two adjacent rating categories (1 through 5) receiving the highest number of ratings with the two categories indicated in parentheses Comparison of Ratings and Rankings with Original Kohl–Davis Study For the purpose of this analysis, a changed title is still considered the same journal. Thus, Journal of the American Society for Information Science and Technology is the equivalent of Journal of the American So- ciety for Information Science and Reference & Users Services Quarterly is the same title as RQ. Less obviously, Information Outlook is considered the successor to Special Libraries. With nonresponses rated 0, there is a strong .864 correlation between the direc- tors’ ratings here and in Kohl–Davis for the set of twenty-five titles covered in both; the correlation for the deans’ rat- ings is .781. These correlations suggest considerable stability in both groups’ perception of journals over the past twenty years. It is questionable whether one could make a valid comparison be- tween the ratings in table 2, calculated by disregarding nonrating responses, and the Kohl–Davis study because they did not use that method. Table 9 lists the ten most highly rated journals by directors and then by deans in Kohl–Davis and their positions in this study.46 There is a remarkable consis- tency in the directors’ perceptions of the top journals over the past two decades. College & Research Libraries, first in 1985, continues in first place and eight of the top ten in 1985 remain in the top ten ac- cording to both methods of calculating the mean rating. Compared to the directors, there is less overlap in the deans’ perceptions of the top ten journals between 1985 and now. Although Library Quarterly and JASIST continue to occupy the top two positions (but in a different order), only two of their 3rd through 10th choices in 1985 (Library Trends and Library & Information Science Research) remain in the top ten. However, changes in the composition of the list to be rated may be an explanatory factor. Drexel Library Quarterly, ranked 7th in 1985, has ceased publication and several titles in the deans’ top ten in this study were, for a variety of reasons, not included in Kohl–Davis (e.g., ARIST, ASIST Proceed- ings, and Journal of Documentation in table 1 and Journal of Documentation, ARIST, Scientometrics, Journal of the American Medical Informatics Association, and MIS Quarterly in table 2). http:study.46 362 College & Research Libraries July 2005 TABLE 7 Degree of Internal Consensus by LIS Deans in Journal Ratings: “Not familiar” and blank responses are counted as 0. Journal Title Journal of the American Society for Info. Science & Technology Top Adjacent Totals1 33 (4,5) Percentage of Total 89.2 Standard Deviation 1.423 Library Quarterly 33 (4,5) 89.2 1.166 NFD Information—Wissenschaft und Praxis 32 (0,1) 86.5 0.956 Cybermetrics 31 (0,1) 83.8 1.301 American Libraries 29 (1,2) 78.4 0.947 International Journal of Geographical Information Science 29 (0,1) 78.4 1.377 Library & Information Science Research 28 (4,5) 75.7 1.500 Restaurator 28 (0,1) 75.7 1.114 ARIST 27 (4,5) 73.0 1.258 Journal of Documentation 27 (4,5) 73.0 1.488 Reference & User Services Quarterly 27 (3,4) 73.0 1.214 Scientist 27 (0,1) 73.0 1.430 Social Science Computer Review 27 (0,1) 73.0 1.475 Zeitschrift fur Bibliothekswesen und Bibliographie 26 (0,1) 70.3 1.813 ASIST Proceedings 25 (3,4) 67.6 1.095 Online Information Review 25 (2,3) 67.6 1.234 Telecommunications Policy 25 (0,1) 67.6 1.924 Information Processing & Management 24 (4,5) 64.9 1.880 Journal of Academic Librarianship 24 (3,4) 64.9 1.325 Library Trends 24 (4,5) 64.9 1.552 Online 24 (2,3) 64.9 1.373 Aslib Proceedings 23 (3,4) 62.2 1.357 Information Technology & Libraries 23 (2,3) 62.2 1.334 Journal of Information Science 23 (3,4) 62.2 1.615 Journal of Scholarly Publishing 23 (3,4) 62.2 1.742 Social Science Information 23 (0,1) 62.2 1.647 Econtent 22 (0,1) 59.5 1.367 Information Outlook 22 (2,3) 59.5 1.238 Interlending & Document Supply 22 (0,1) 59.5 1.351 International Journal of Legal Information 22 (0,1) 59.5 1.701 Journal of Education for Library & Information Science 22 (3,4) 59.5 1.288 Journal of the Medical Library Association 22 (3,4) 59.5 1.525 Perception of Library and Information Science Journals 363 TABLE 7 Degree of Internal Consensus by LIS Deans in Journal Ratings: “Not familiar” and blank responses are counted as 0. Journal Title Libraries & Culture Top Adjacent Totals1 22 (4,5) Percentage of Total 59.5 Standard Deviation 1.706 Library Journal 22 (1,2) 59.5 1.244 Library Resources & Technical Services 22 (4,5) 59.5 1.646 Public Libraries 22 (2,3) 59.5 1.351 Reference Services Review 22 (2,3) 59.5 1.441 Canadian Journal of Information & Library Science 21 (3,4) 56.8 1.367 Information Systems Journal 21 (0,1) 56.8 1.592 Libri 21 (3,4) 56.8 1.413 Program: Electronic Library & Information Systems 21 (0,1) 56.8 1.726 School Library Journal 21 (3,4) 56.8 1.330 School Library Media Research 21 (3,4) 56.8 1.656 College & Research Libraries 20 (4,5) 54.1 1.481 D-Lib Magazine 20 (0,1) 54.1 1.573 Electronic Library 20 (0,1) 54.1 1.415 Government Information Quarterly 20 (3,4) 54.1 1.492 Microform & Imaging Review 20 (2,3) 54.1 1.324 MIS Quarterly 20 (0,1) 54.1 2.070 Scientometrics 20 (0,1) 54.1 2.192 Information Systems Research 19 (0,1) 51.4 1.951 Journal of Government Information 19 (3,4) 51.4 1.552 Journal of Health Communication 19 (0,1) 51.4 1.722 Harvard Library Bulletin 19 (0,1) 51.4 1.216 First Monday 18 (0,1) 48.6 1.605 Journal of Information Technology 18 (0,1) 48.6 1.813 Journal of the American Medical Informatics Association 18 (4,5) 48.6 1.906 Knowledge Organization 18 (0,1) 48.6 1.888 Law Library Journal 18 (3,4) 48.6 1.622 Library & Information Science 18 (3,4) 48.6 1.747 Library Collections, Acquisitions & Technical Services 18 (3,4) 48.6 1.722 Collection Management 17 (2,3) 45.9 1.573 Information Research 17 (4,5) 45.9 1.942 International Information & Library Review 17 (0,1) 45.9 1.614 364 College & Research Libraries July 2005 TABLE 7 Degree of Internal Consensus by LIS Deans in Journal Ratings: “Not familiar” and blank responses are counted as 0. Journal Title International Journal of Information Management Top Adjacent Totals1 17 (3,4) Percentage of Total 45.9 Standard Deviation 1.745 Journal of Information Ethics 17 (2,3) 45.9 1.702 Journal of Librarianship & Information Science 17 (3,4) 45.9 1.739 The Information Society 16 (3,4) 43.2 1.946 Internet Research 16 (3,4) 43.2 1.669 Journal of Management Information Systems 16 (0,1) 43.2 1.900 Information & Management 15 (0,1) 40.5 1.712 1The number of responses in the two adjacent rating categories (0 through 5) receiving the highest number of ratings with the two categories indicated in parentheses The Top Five Method Following White’s methodology for evaluating LIS education programs,47 Kohl and Davis noted that the unordered “top five” technique forces respondents into “extreme choices,” which, unlike an ordinal scale, are not influenced by sec- ondary or tertiary ratings.48 They believed this approach might be unreliable when consensus falls below 40 percent, but that it “works well when there is a strong con- sensus.” Table 10 lists the directors’ top five choices and table 11 the titles named by deans as among the top five. Thirty-nine titles were listed by direc- tors, topped by College & Research Librar- ies, which ranked first in both tables 1 and 2. The directors’ top four picks in table 10 are the top four in table 1 (but in different order) and account for three of the top four in table 2. Portal: Libraries and the Academy, which tied for fi h in table 10, was not on the original list. The deans listed thirty-one different titles, headed by the JASIST and then Li- brary Quarterly. These two journals tied for first in table 1 and ranked first and second in table 2. Indeed, of the deans’ top five choices in table 11, four ranked among the top five in both tables 1 and 2; none placed lower than 7th in either table. Although more titles are listed here as top five candidates than in the original Kohl–Davis study (where deans listed 21 journals and directors 18), there is remarkable continuity in the directors’ choices. College & Research Libraries was also their first choice in 1985, and their four most frequently mentioned titles were the same as in this study, although the precise order differs. There is less continuity in the deans’ selections. In 1985, Library Quarterly was their most fre- quent choice, followed by JASIS, an order that is reversed here. (For this study’s purposes, JASIST is the equivalent of JASIS.) The deans’ next three top five selections from 1985 (College & Research Libraries, Library Trends, and Journal of Education for Librarianship) were 8th, 7th, and 9th (tied), respectively, in this investigation.49 In summary, the overlap in the most prestigious journals according to both the mean rating and “top five” approaches in this study and the notable continuity over twenty years in the top five choices reinforce the perception of an exceed- ingly high-prestige status for some elite journals. As with the mean ratings, there is greater continuity in the directors’ per- ceptions than in the deans’. http:investigation.49 http:ratings.48 Perception of Library and Information Science Journals 365 TABLE 8 Degree of Internal Consensus by LIS Deans in Journal Ratings: “Not familiar” and blank responses are not considered. Journal Title Top Adjacent Totals1 Percentage of Total Standard Deviation Journal of the American Society for Info. Science & Technology 33 (4,5) 94.3 0.968 Library Quarterly 33 (4,5) 91.7 0.910 Journal of Scholarly Publishing 23 (3,4) 88.5 0.706 Online 24 (2,3) 85.7 0.810 Microform & Imaging Review 20 (2,3) 83.3 0.776 American Libraries 29 (1,2) 82.9 0.867 Journal of Information Technology 17 (3,4) 81.0 1.007 Online Information Review 25 (2,3) 80.6 0.791 Information Processing & Management 24 (4,5) 80.0 0.877 Library & Information Science Research 28 (4,5) 80.0 1.183 Restaurator 8 (2,3) 80.0 0.823 Journal of Documentation 27 (4,5) 79.4 0.989 Reference Services Review 23 (3,4) 79.3 0.778 Library & Information Science 18 (3,4) 78.3 0.822 Reference & User Services Quarterly 27 (3,4) 77.1 0.910 Program: Electronic Library & Information Systems 13 (3,4) 76.5 0.951 ARIST 27 (4,5) 75.0 1.079 Information Systems Journal 12(2,3) 75.0 0.929 Knowledge Organization 15 (3,4) 75.0 0.999 Library Collections, Acquisitions & Technical Services 18 (3,4) 75.0 0.932 Journal of Information Science 23 (3,4) 74.2 1.029 Information Systems Research 14 (3,4) 73.7 1.017 The Information Society 16 (3,4) 72.7 1.008 Internet Research 16 (3,4) 72.7 0.976 Scientometrics 13 (4,5) 72.2 1.110 Information Technology & Libraries 23 (2,3) 71.9 0.856 Journal of Management Information Systems 15 (2,3) 71.4 0.928 NFD Information—Wissenschaft und Praxis 5 (1,2) 71.4 1.069 Library Resources & Technical Services 22 (4,5) 71.0 0.973 International Journal of Legal Information 12 (3,4) 70.6 1.111 Library Trends 24 (4,5) 70.6 1.153 Electronic Library 14 (2,3) 70.0 0.945 Aslib Proceedings 23 (3,4) 69.7 0.992 366 College & Research Libraries July 2005 TABLE 8 Degree of Internal Consensus by LIS Deans in Journal Ratings: “Not familiar” and blank responses are not considered. Journal Title ASIST Proceedings Top Adjacent Totals1 25 (3,4) Percentage of Total 69.4 Standard Deviation 0.939 Libraries & Culture 22 (4,5) 68.8 1.191 Journal of Academic Librarianship 24 (3,4) 68.6 1.051 Interlending & Document Supply 13 (2,3) 68.4 0.946 International Journal of Information Management 17 (3,4) 68.0 1.130 School Library Media Research 21 (3,4) 67.7 1.013 Information Outlook 22 (2,3) 66.7 0.936 Journal of the Medical Library Association 22 (3,4) 66.7 1.091 MIS Quarterly 12 (4,5) 66.7 1.043 Public Libraries 22 (2,3) 66.7 1.015 Scientist 8 (3,4) 66.7 1.055 Journal of Government Information 19 (3,4) 65.5 1.017 Harvard Library Bulletin 17 (2,3) 65.4 0.891 Journal of Information Ethics 17 (2,3) 65.4 1.148 International Information & Library Review 15 (2,3) 65.2 1.137 Journal of the American Medical Informatics Association 18 (4,5) 64.3 1.044 Law Library Journal 18 (3,4) 64.3 1.008 Social Science Information 9 (3,4) 64.3 0.949 Canadian Journal of Information & Library Science 21 (3,4) 63.6 1.029 Social Science Computer Review 7 (2,3) 63.6 1.136 D-Lib Magazine 12 (3,4) 63.2 0.976 Information Research 17 (4,5) 63.0 1.171 Journal of Librarianship & Information Science 17 (3,4) 63.0 1.068 Library Journal 22 (1,2) 62.9 1.157 Government Information Quarterly 20 (3,4) 62.5 1.118 Libri 21 (3,4) 61.8 1.149 Journal of Education for Library & Information Science 22 (3,4) 61.1 1.178 Information & Management 14 (2,3) 60.9 1.065 Collection Management 17 (2,3) 60.7 1.113 Journal of Health Communication 12 (2,3) 60.0 1.217 School Library Journal 21 (3,4) 60.0 1.175 Perception of Library and Information Science Journals 367 TABLE 8 Degree of Internal Consensus by LIS Deans in Journal Ratings: “Not familiar” and blank responses are not considered. Journal Title First Monday Top Adjacent Totals1 13 (2,3) Percentage of Total 59.1 Standard Deviation 1.120 College & Research Libraries 20 (4,5) 58.8 1.115 International Journal of Geographical Information Science 7 (1,2) 58.3 1.379 Zeitschrift fur Bibliothekswesen und Bibliographie 7 (4,5) 58.3 1.314 Econtent 12 (2,3) 57.1 1.044 Telecommunications Policy 7 (4,5) 53.8 1.387 Cybermetrics 4 (3,4) 50.0 1.389 1The number of responses in the two adjacent rating categories (1 through 5) receiving the highest number of ratings with the two categories indicated in parentheses Additionally Suggested Journals Additional titles suggested by respon- dents that were not on the original list of seventy-one to be rated are tabulated in tables 12 (by directors) and 13 (by deans). The titles were verified through Ulrich’s Periodicals Directory, Google, or contacting the Indiana University Libraries Reference Department, because a few respondents listed them incorrectly. The ARL directors proposed forty-two additional titles, but their list was highly skewed toward two titles: Portal: Libraries & the Academy, men- tioned by twenty-two directors, and Libres, named by eighteen, suggesting these two journals should have been considered for inclusion on the original list. Seven direc- tors named the Journal of Library Adminis- tration, and six directors named Educause Review and Library Hi-Tech. The table does not include two titles mentioned by directors, Journal of the American Society for Information Science and Technology and The Information Society, which actually did appear on the original list. In contrast to the directors, there is a remarkably even distribution among the thirty-nine additional titles suggested by the deans as only four were named more than twice. American Archivist was men- tioned by four deans, and Archival Science, Archivaria, and Information Retrieval were each named by three deans. It is note- worthy that most of these titles focus on archival science. Ten titles were proposed by two different deans, and twenty-five were listed only once. In the Kohl–Davis study, LIS deans suggested fifteen additional titles and ARL directors suggested only ten, lead- ing them to conclude, “our choice of core library journals was confirmed.”50 The larger number of additional titles sug- gested here, even though the titles on the list for rating more than doubled in size from thirty-one to seventy-one, calls into question whether a single core list for the LIS field exists. Open-ended Responses Fifteen deans and thirteen directors wrote comments in response to the questionnaire’s open-ended section. The most prevalent theme concerned the importance of non-LIS journals. Specific comments from various deans included: “Titles from other disciplines are impor- tant for faculty from those disciplines”; “The nature of LIS research is such that any listing of LIS journals will not catch the outlyers [sic]”; “Because our faculty is multidisciplinary, we also look favorably 368 College & Research Libraries July 2005 on major research journals from related fields such as communication and policy studies”; and “‘Peer-reviewed’ journals are important in the T & P process wheth- er they are directly in LIS or in a cognate field.” A number of directors made similar observations, such as: “Our librarians publish in nonlibrary journals”; “Tenure criteria are broader than LIS-type publi- cations”; and “P & T is not limited these days to LIS journals.” Education, higher education, instructional technology, TABLE 9 Top Ten Journals in Kohl-Davis and Their Current Ranking ARL Directors Rank and Title Table 1 (NFs & blank responses = 0) Table 2 (NFs & blank responses disregarded) 1. College & Research Libraries 1 1 2. Library Quarterly 4 6 3. Journal of Academic Librarianship 3 4 4. Library Resources & Technical Services 6 8* 5. Library Trends 2 2 6. Information Technology & Libraries 10 7 7. JASIS 7** 3** 8. Library Journal 11* 45 9. American Libraries 18 66 10. RQ 5*** 5*** *Tied position ** Under present title, Journal of the American Society for Information Science & Technology ***Under present title, Reference & User Services Quarterly LIS Deans Rank and Title Table 1 (NFs & blank responses = 0) Table 2 (NFs & blank responses disregarded) 1. Library Quarterly 1* 2 2. JASIS 1* ** 1** 3. College & Research Libraries 11 12* 4. Library Trends 6 8 5. Journal of Education for Librarianship 12*** 28*** 6. Library Resources & Technical Services 15 12* 7. Drexel Library Quarterly NR NR 8. Special Libraries 28**** 55**** 9. Information Technology & Libraries 25* 40* 10. Library & Info. Science Research 3* 4 NR: Not included in rating because title ceased publication *Tied position **Under present title, Journal of the American Society for Information Science & Technology ***Under present title, Journal of Education for Library & Information Science ****Under present title, Information Outlook Perception of Library and Information Science Journals 369 TABLE 10 Journals Listed among the Top Five Most Prestigious by ARL Directors Journal Title College & Research Libraries Number of Times Listed 40 Percentage of the 56 Directors Who Listed It 71.4 Journal of Academic Librarianship 27 48.2 Library Trends 26 46.4 Library Quarterly 17 30.4 Portal: Libraries and the Academy 14 25.0 Journal of the American Society for Info. Science & Technology 14 25.0 Reference & User Services Quarterly 13 23.2 Library Resources & Technical Services 10 17.9 D-Lib Magazine 8 14.3 Annual Review of Information Science & Technology 7 12.5 Information Technology & Libraries 7 12.5 Library Collections, Acquisitions & Technical Services 7 12.5 Library Journal 6 10.7 Educause Review 3 5.4 Journal of the Medical Library Association 3 5.4 Reference Services Review 3 5.4 Collection Management 2 3.6 First Monday 2 3.6 Journal of Documentation 2 3.6 Journal of Information Science 2 3.6 MIS Quarterly1 2 3.6 ASIST Proceedings 2 3.6 American Libraries 1 1.8 Canadian Journal of Information and Library Science 1 1.8 Chronicle of Higher Education 1 1.8 College & Research Libraries News 1 1.8 Cybermetrics 1 1.8 Government Information Quarterly 1 1.8 Harvard Library Bulletin 1 1.8 Information Processing & Management 1 1.8 International Journal of Geographical Information Science 1 1.8 Journal of Scholarly Publishing 1 1.8 370 College & Research Libraries July 2005 TABLE 10 Journals Listed among the Top Five Most Prestigious by ARL Directors Journal Title Number of Times Listed Percentage of the 56 Directors Who Listed It Journal of the American Medical Informatics Association 1 1.8 Journal of Education for Library & Information Science 1 1.8 Journal of Information Technology 1 1.8 Journal of Librarianship & Information Science 1 1.8 Library & Information Science Research 1 1.8 Libri 1 1.8 Serials Librarian 1 1.8 1Listed as Management Information Quarterly computer science, bioinformatics, opera- tions research, communications studies, sociology, and history were specifically mentioned as fields whose journals could be important. One dean wrote, “An essential prob- lem here is the diversity of our field… the fractured nature of the discourse functionally means that there are few reputable journals publishing across the field” and a director stated “[It is] very hard to rate such different sorts of journals on one common scale.” Other points included the significance of peer- reviewed journals, the influence of a candidate’s specialty area on the relative importance of the journals in his or her case, and the fact that factors other than journal quality are considered. Some respondents mentioned various journal categories they believed were omi ed from or underrepresented on the initial list (e.g., archival science, Haworth Press publications, and all-electronic journals). A few noted that their institutions did not have a “stated policy” or “prescribed list” regarding journals in the promotion and tenure process. Thirteen directors responded but declined to complete the questionnaire, citing reasons such as institutional policy against completing surveys, their library does not collect LIS literature, lack of knowledge about the journals, the con- cept of prestige is “ambiguous,” and (the most frequently stated reason) their institution does not have a promotion and tenure policy. One nonresponding director wrote that the same list could not be used for both promotion and tenure and journal collection manage- ment decisions—a potentially debatable point. One director stated, “I want many of these journals [on our list] to cease pub- lication. They simply should not exist.” And a dean exclaimed, “No stamp? You got my time and my 37 cents—you lucked out” (return postage was not included because institutional support for mailing was assumed), thus demonstrating there are some curmudgeons in both groups. Correspondence between Perception Ratings and Citation Data To explore the relationship between the perceptions of journals and citation data, this study’s mean ratings were correlated with the journals’ citation scores from the 2001 Journal Citations Reports (the most current version when the project was designed during the first half of 2003). Table 14, displaying the set of eight cor- relations, shows moderate positive cor- Perception of Library and Information Science Journals 371 TABLE 11 Journals Listed among the Top Five Most Prestigious by LIS Deans Journal Title Number of Times Listed Percentage of the 37 Deans Who Listed It Journal of the American Society for Information Science & Technology 29 78.4 Library Quarterly 25 67.6 Library & Information Science Research 14 37.8 Annual Review of Information Science & Technology 12 32.4 Information Processing & Management 11 29.7 Journal of Documentation 9 24.3 Library Trends 8 21.6 College & Research Libraries 6 16.2 Journal of Education for Library & Information Science 5 13.5 Libraries & Culture 5 13.5 ACM Transactions on Information Systems 3 8.1 Reference & User Services Quarterly 3 8.1 American Archivist 2 5.4 Canadian Journal of Information & Library Science 2 5.4 Journal of Management Information Systems 2 5.4 Journal of the Medical Library Association 2 5.4 Management Information Quarterly 2 5.4 ASIST Proceedings 2 5.4 Archival Science 1 2.7 Government Information Quarterly 1 2.7 Information Research 1 2.7 Information Retrieval 1 2.7 Journal of Academic Librarianship 1 2.7 Journal of American Libraries1 1 2.7 Journal of the American Medical Informatics Association 1 2.7 Journal of Health Communication 1 2.7 Journal of Information Science 1 2.7 Journal of Librarianship & Information Science 1 2.7 School Library Media Research 1 2.7 Scientometrics 1 2.7 Telecommunications Policy 1 2.7 1Unverified title 372 College & Research Libraries July 2005 TABLE 12 Additional Titles Suggested by ARL Directors Journal Title Number of Times Suggested Portal: Libraries & the Academy 22 Libres 18 Journal of Library Administration 7 Educause Review 6 Library Hi-Tech 6 Cataloging & Classification Quarterly 5 Serials Librarian 4 Against the Grain 3 Charlotte Advisor 3 International Fed. of Library Assoc. & Institutions Proceedings 3 Research Strategies 3 American Archivist 2 Acquisitions Librarian 2 Chronicle of Higher Education 2 Educause Quarterly 2 Journal of Internet Cataloging 2 Library Administration & Management Journal 2 Library Technology Reports 2 Reference Librarian 2 Serials Review 2 Advances in Library Administration & Organization 1 Association of Research Libraries Newsletter 1 Bottom Line, The 1 Choice 1 College & Research Libraries News 1 Colorado Libraries 1 Computers in Libraries 1 Cybernetrics & Human Knowing 1 Data Mining & Knowledge Discovery 1 Human Computer Interaction 1 Institute of E & E Engineers Transactions on Info Theory 1 International Journal of Educational Technology 1 International Journal of Human-Computer Studies 1 International Journal of Medical Informatics 1 Inspel1 1 Issues in Science & Technology Librarianship 1 Perception of Library and Information Science Journals 373 TABLE 12 Additional Titles Suggested by ARL Directors Journal Title Journal of Access Services Number of Times Suggested 1 Journal of Behavioral & Social Sciences Librarianship 1 Journal of Data Mining & Knowledge Discovery2 1 Library Acquisitions, Practices & Theory3 1 Science & Technology Libraries 1 Technical Services Quarterly 1 1Listed as International Journal of Special Libraries 2Listed as Data Mining & Knowledge Discovery 3The current title, Library Collections, Acquisitions, & Technical Services, is on the list of 71 for rating. relations of the deans’ mean ratings with impact factor (.528) and of their ratings with total citations (.479), when blank responses are disregarded. In contrast, the correlations drop precipitously when blank responses are counted as 0 to .304 for impact factor and .254 for total cita- tions. It is striking that the correlations between the directors’ ratings and citation data range from weak (.267) to practically nonexistent (.038). The correlations in table 14 are higher when blank responses are disregarded rather than counted as 0, suggesting the former may be a more valid measure of calculating the mean rating score. There may be a variety of reasons why the deans’ ratings have a moderate association with the JCR citation measures and the ARL library directors’ do not. It is probable that JCR citation data measure a journal’s contribution to research to a greater ex- tent than its usefulness for professional practice, whereas deans placed a greater emphasis in their ratings on the former and directors the la er. Deans may be familiar with a broader range of journals and may have more accurately estimated the quality of information science and nonacademic library science journals. Indeed, the fact that deans tended to give higher ratings to information science journals that have high citation scores is undoubtedly an important factor. A com- plete explanation for this phenomenon is not readily apparent and requires further research beyond this article’s scope. Although most journal rankings do not analyze the correlation between percep- tion and citation data, a few such reports for other disciplines are available in the literature. Christenson and Sigelman51 found that JCR impact factor scores dis- played a .526 correlation with Glenn’s52 perception rating of sociology journals and a .572 correlation with Giles and Wright’s53 perception study of political science journals. Nisonger ’s54 ranking of political science journals based on manipulation of JCR impact factor had a .71 correlation with Giles, Mizell, and Pa erson’s55 perception ranking and a .59 correlation with Garand’s56 ranking, which combined mean perception ratings with the proportion of respondents rating the journal. Conclusions Although a new citation ranking of LIS journals is available each year through the Journal Citation Reports, this replica- tion of the Kohl–Davis57 study offers a current perception-based ranking, updat- ing Blake’s58 1996 ranking, which used data gathered in 1992. This study, like its predecessors, has demonstrated that a hierarchy of prestige among LIS journals does indeed exist, but the hierarchical order differs somewhat between the two constituencies. There is notable continu- 374 College & Research Libraries July 2005 TABLE 13 Additional Titles Suggested by LIS Deans Journal Title Number of Times Suggested American Archivist 4 Archival Science 3 Archivaria 3 Information Retrieval 3 ACM Transactions on Information Systems 2 Behaviour & Information Technology 2 Bookbird 2 Education for Information 2 International Journal of Human–Computer Studies 2 Journal of Library Administration 2 Journal of Youth Services1 2 Portal: Libraries & the Academy 2 School Libraries Worldwide 2 World Libraries2 2 ACM Communications 1 ACM Transactions on Computer–Human Interaction 1 Archival Issues 1 Arges 1 Bulletin de bibliothèques de France 1 Documentaliste-Sciences de l’Information3 1 Documentation et bibliothèques 1 Harvard Business Review 1 Human Factors 1 Information et Documentation 1 Inspel 1 Institute of Electrical and Electronic Engineers Network 1 KM World 1 Knowledge Management Review4 1 Knowledge Quest 1 Library, The 1 MIT Sloan Management Review5 1 Papers of the Bibliographical Society of America 1 Papers of the Bibliographical Society of Canada 1 RBM: A Journal of Rare Books, Manuscripts & Cultural Heritage6 1 RLG DigiNews7 1 Serials Review 1 Perception of Library and Information Science Journals 375 TABLE 13 Additional Titles Suggested by LIS Deans Journal Title Signal Number of Times Suggested 1 Sloan Management 1 Voice of Young Advocates 1 (Vague or illegible responses) 3 1Listed once as Journal of Youth Services 2Listed once under previous title Third World Libraries 3Listed as Documentaliste 4Listed as KM Review 5Listed as Sloan Management 6Listed as Rare Books and Manuscripts 7Listed as DigiNews ity in the perception of LIS journals over a twenty-year period, but more so in the directors’ perceptions than in the deans’. The most elite journals in the 1980s main- tain the highest status positions (e.g., JASIS (now JASIST), Library Quarterly, and College & Research Libraries). The authors’ findings suggest (but do not prove) that the composition of LIS as a discipline is changing. As noted above, several respondents questioned whether a single list of journals could represent the LIS field, given its increasingly di- verse, interdisciplinary, and even mul- tidisciplinary nature. This contention is supported by the fact that the number of journals listed among the “top five” and as additions to the list for rating is much higher now than twenty years ago. Some caveats regarding the use and interpretation of these findings are in order. Journal value is multifaceted, so that a low-ranking journal in this study may still be important for sup- porting teaching, professional practice, a specialty area, or some other purpose. Although a journal rating has potential use for journal collection management decisions in libraries and university promotion and tenure decisions, the perception of a journal’s prestige is sim- ply one bit of information that should be used cautiously in conjunction with other indicators, such as the publisher’s reputation, rejection rate, indexing coverage, editorial board membership, status of authors contributing to the jour- nal, and so on. The journal’s relevance TABLE 14 Pearson Correlations between Perception Ratings and 2001 JCR Citation Data Variables Correlation Deans’ rating (NFs & blanks disregarded) and total citations .479 Deans’ rating (NFs & blanks disregarded) and impact factor .528 Deans’ rating (NFs & blanks counted as 0) and total citations .254 Deans’ rating (NFs & blanks counted as 0) and impact factor .304 Directors’ rating (NFs & blanks disregarded) and total citations .208 Directors’ rating (NFs & blanks disregarded) and impact factor .267 Directors’ rating (NFs & blanks counted as 0) and total citations .038 Directors’ rating (NFs & blanks counted as 0) and impact factor .086 376 College & Research Libraries to the candidate’s specialty would be important in promotion and tenure deci- sions, whereas cost, usage, and collecting priorities would be additional factors in libraries’ journal decision-making. This caveat would, of course, apply to citation data as well. As stated by Kohl and Da- vis, “the prestige of a journal is only an July 2005 indication, not a guarantee of the quality of its articles.”59 Finally it is unclear how recent trends or developments, such as the move- ment toward open-access or full-text databases, are currently affecting journal perceptions or might affect them in the future. Notes 1. David F. Kohl and Charles H. Davis, “Ratings of Journals by ARL Library Directors and Deans of Library and Information Science Schools,” College & Research Libraries 46 (Jan. 1985): 40–47. 2. Virgil L. P. Blake, “The Perceived Prestige of Professional Journals, 1995: A Replication of the Kohl–Davis Study,” Education for Information 14 (Oct. 1996): 157–79. 3. Thomas E. Nisonger, Management of Serials in Libraries (Englewood, Colo.: Libraries Un- limited, 1998), 201. 4. ———, “JASIS and Library and Information Science Journal Rankings: A Review and Analysis of the Last Half Century,” Journal of the American Society for Information Science 50 (Sept. 1999): 1004–19. 5. Alice I. Bryan, The Public Librarian (New York: Columbia University Pr., 1952). 6. Mary Lee Bundy, “Public Library Administrators View Their Professional Periodicals,” Illinois Libraries 43 (June 1996): 397–420. 7. C. W. Hanson and Patricia Tilbury, “Library Literature Read by ASLIB Conference At- tenders,” Journal of Documentation 19 (June 1963): 63–71. 8. Rudolf Jacob Penner, “Measuring a Library’s Capability … ,” Journal of Education for Li- brarianship 13 (summer 1972): 17–30. 9. Charles T. Meadow and Mary Ann Zaborowski, “Some Statistical Aspects of JASIS Publica- tions,” Journal of the American Society for Information Science 30 (Nov. 1979): 368–71. 10. Robert M. Hayes, “Citation Statistics as a Measure of Faculty Research Productivity,” Journal of Education for Librarianship 23 (winter 1983): 151–72. 11 Herbert S. White, “Perceptions by Educators and Administrators of the Ranking of Library School Programs,” College & Research Libraries 42 (May 1981): 191–202. 12. ———, “Perceptions by Educators and Administrators of the Ranking of Library School Programs: An Update and Analysis,” Library Quarterly 57 (July 1987): 252–68. 13. ———, “Rankings of Library and Information Science Faculty and Programs: The Third in a Series of Studies Undertaken at Six-year Intervals,” Library Quarterly 63 (Apr. 1993): 166–88. 14. Kohl and Davis, “Ratings of Journals by ARL Library Directors and Deans,” 41. 15. Jesse H. Shera, Introduction to Library Science (Li leton, Colo.: Libraries Unlimited, 1976). 16. Kohl and Davis, “Ratings of Journals by ARL Library Directors and Deans,” 45. 17. Renee Tjoumas, “Professional Journal Utilization by Public Library Directors,” Serials Librarian 20, nos. 2/3 (1991): 1–16. 18. Blake, “In the Eyes of the Beholder: Perceptions of Professional Journals by Library/In- formation Science Educators and District School Library Media Center Coordinators,” Collection Management 14, nos. 3/4 (1991): 101–48. 19. Renee Tjoumas and Virgil L. P. Blake, “Faculty Perceptions of the Professional Journal Literature: Quo Vadis?” Journal of Education for Library & Information Science 33 (summer 1992): 173–94. 20. Blake, “The Perceived Prestige of Professional Journals.” 21. E. E. Nkereuwem, “Accrediting Knowledge: The Ranking of Library and Information Science Journals,” Library Review 46, no. 2 (1997): 99–104 and Asian Libraries 6, nos. 1/2 (1997): 71–76. 22. Michael W. Giles, Francie Mizell, and David Pa erson, “Political Scientists’ Journal Evalu- ations Revisited,” PS: Political Science & Politics 22 (Sept. 1989): 613–17. 23. Norval D. Glenn, “American Sociologists’ Evaluations of Sixty-three Journals,” American Sociologist 6 (Nov. 1971): 298–303. 24. David Lee and Arthur Evans, “American Geographers’ Rankings of American Geography Journals,“ Professional Geographer 36 (Aug. 1984): 292–300. Perception of Library and Information Science Journals 377 25. David Shichor, Robert M. O’Brien, and David L. Decker, “Prestige of Journals in Criminol- ogy and Criminal Justice,” Criminology 19 (Nov. 1981): 461–69. 26. J. Craig Peery and Gerald R. Adams, “Qualitative Ratings of Human Development Jour- nals,” Human Development 24, no. 5 (1981): 312–19. 27. Jean-Louis Malouin and J.-Francois Outreville, “The Relative Impact of Economics Jour- nals—A Cross Country Survey and Comparison,” Journal of Economics and Business 39 (Aug. 1987): 267–77. 28. Marian M. Extejt and Jonathan E. Smith, ”The Behavioral Sciences and Management: An Evaluation of Relevant Journals,” Journal of Management 16 (Sept. 1990): 539–51. 29. Robert H. Luke and E. Reed Doke, “Marketing Journal Hierarchies: Faculty Perceptions, 1986–87,” Journal of the Academy of Marketing Science 15 (spring 1987): 74–78. 30. Joe Albert and P. R. Chandy, “Research and Publishing in Real Estate,” Akron Business & Economics Review 17 (winter 1986): 46–54. 31. Andrew C. Wicks and Robbin Derry, “An Evaluation of Journal Quality: The Perspective of Business Ethics Researchers,” Business Ethics Quarterly 6 (July 1996): 359–72. 32. Lawrence D. Brown and Ronald J. Huefner, “The Familiarity with and Perceived Quality of Accounting Journals: Views of Senior Accounting Faculty in Leading U.S. MBA Programs,” Contemporary Accounting Research 11 (summer 1994): 223–50. 33. Ram A. Cnaan, Richard K. Caputo, and Yochi Shmuely, “Senior Faculty Perceptions of Social Work Journals,” Journal of Social Work Education 30 (spring/summer 1994): 185–99. 34. In order to protect respondent confidentiality, as required by the Indiana University Human Subjects Commi ee, returned questionnaires were removed from the envelopes and filed so that it was impossible to associate a completed questionnaire with a particular individual. However, for the second mailing it was possible to estimate nonrespondents to the initial mailing based on postmarks on the returning envelopes. 35. Kohl and Davis, “Ratings of Journals by ARL Library Directors and Deans,” 41. 36. Blake, “The Perceived Prestige of Professional Journals,” 160–61. 37. Bundy, “Public Library Administrators View Their Professional Periodicals,” 398. 38. Hanson and Tilbury, “Library Literature Read by ASLIB Conference A enders,” 64. 39. Cnaan, Caputo, and Shmuely, “Senior Faculty Perceptions of Social Work Journals,” 188. 40. Brown and Huefner, “The Familiarity with and Perceived Quality of Accounting Journals,” 227. 41. Wicks and Derry, “An Evaluation of Journal Quality,” 363. 42. The authors are aware that statistical purists maintain that t-tests should not be used for ordinal data and for descriptive rather than inferential statistics. In reality, t-tests are commonly applied to these types of data, and we have used them here for the sake of continuity with the original Kohl–Davis study and its replications. 43. Kohl and Davis, “Ratings of Journals by ARL Library Directors and Deans,” 43; percentage calculated by Nisonger and Davis. 44. Ibid., 42. 45. Ibid., 43—44; percentages calculated by Nisonger and Davis. 46. Ibid., 42. 47. White, “Perceptions by Educators and Administrators of the Ranking of Library School Programs,” 194. 48. Kohl and Davis, “Ratings of Journals by ARL Library Directors and Deans,” 45. 49. Ibid., 46. 50. Ibid., 45. 51. James A. Christenson and Lee Sigelman, “Accrediting Knowledge: Journal Stature and Citation Impact in Social Science,” Social Science Quarterly 66 (Dec. 1985): 964–75. 52. Glenn, “American Sociologists’ Evaluations of Sixty-Three Journals.” 53. Michael W. Giles and Gerald C. Wright Jr., “Political Scientists’ Evaluations of Sixty-Three Journals,” PS: Political Science & Politics 8 (summer 1975): 254–56. 54. Thomas E. Nisonger, “A Ranking of Political Science Journals Based on Citation Data,” Serials Review 19, no. 4 (1993): 7–14. 55. Giles, Mizell, and Pa erson, “Political Scientists’ Journal Evaluations Revisited.” 56. James C. Garand, “An Alternative Interpretation of Recent Political Science Journal Evalu- ations,” PS: Political Science & Politics 23 (Sept. 1990): 448–51. 57. Kohl and Davis, “Ratings of Journals by ARL Library Directors and Deans.” 58. Blake, “The Perceived Prestige of Professional Journals.” 59. Kohl and Davis, “Ratings of Journals by ARL Library Directors and Deans,” 47.