Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals Previous Contents Next Issues in Science and Technology Librarianship Summer 2008 DOI:10.5062/F4FF3Q9G Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals Edward Gray Public Services Librarian teddy.gray@duke.edu Sarah Z. Hodkinson Library Assistant sarah.hodkinson@duke.edu Biological and Environmental Sciences Library Duke University Durham, North Carolina Copyright 2008, Edward Gray and Sarah Z. Hodkinson. Used with permission. Abstract Impact factors for journals listed under the subject categories "ecology" and "environmental sciences" in the Journal Citation Reports database were calculated using citation data from the Scopus database. The journals were then ranked by their Scopus impact factor and compared to the ranked lists of the same journals derived from Journal Citations Reports. Although several titles varied significantly in impact factor and rank, the Journal Citation Reports and Scopus lists had a high degree of statistical similarity. Introduction The Thomson Scientific impact factor is a popular tool that is used to measure the influence of scholarly journals. Nearly fifty years ago, Eugene Garfield developed this method as a way to use the Thomson Scientific (formerly Institute for Scientific Information, ISI) citation indexes to analyze the frequency with which the average journal article is cited (Garfield & Sher 1963; Garfield 1972). The impact factor of a journal is calculated by dividing the references cited in one year by the number of citable articles published in the same journal over the previous two years. This ratio is published annually in Thomson Scientific's Journal Citation Reports (JCR), along with a number of other quantitative tools for comparing and evaluating scholarly journals. For years, the impact factor has been used to "clarify the significance of absolute citation frequencies" (Thomson Scientific 1994), and as a result its applications have gained great significance in the world of academia (Monastersky 2005). Academic librarians frequently use impact factors to help them decide which journals are important enough to subscribe to and which subscriptions may be canceled (Barschall 1988; Coleman 2007). While impact factor data were originally used extensively in journal marketing to rank and compare scholarly journals for prospective subscribers, it has become a way of ranking the scientists who publish articles in these journals. Since the importance of scientific research is notoriously difficult to evaluate quantitatively, many universities now make funding and tenure decisions based on the average impact factor value, or prestige, of the journals that a scientist has published in. This may seem to be a reliable tool to assess the importance of a particular scientist's research, but there have been numerous reports that show the dangers of overestimating the value of the impact factor (Seglen 1997; Opthof 1997; Coleman 1999). The impact factor of a journal may be artificially inflated when it frequently includes review articles and letters, and there is also a bias against articles that are not published in English. Journals that publish articles on botanical or zoological taxonomy may also have lower impact factors because they often rely on older references, and references to taxonomic names are not included in the references list at the ends of these papers (Werner 2006). Since the widely used JCR impact factors are all based on data from journals indexed in Web of Science, the researchers of the present study were interested in comparing the impact factors calculated from citation data from another database. Elsevier's Scopus, which became available in 2004, has been considered a competitor to Web of Science because of its citation tracking capabilities (Bakkalbasi et al. 2006). Scopus does not rank journals according to "impact factor," using instead a measure called the h-index that was developed in 2005 by Hirsch to evaluate the impact of journals. However, the data required to calculate an impact factor using Garfield's ratio are all available in the Scopus database (Pislyakov 2007). If impact factor is the most widely used tool for evaluating the importance of a journal and thus the worth of its writers, it would be wise to determine whether or not calculated impact factors vary significantly between JCR and Scopus. Methods Impact factors were obtained for the 116 ecology titles and the 160 environmental sciences titles listed in the 2007 edition of JCR. In this latest edition of JCR, cites to recent articles (the numerator of the impact factor equation) were from 2007, while citable articles (the denominator) were from 2005 and 2006. JCR does not state what constitutes a citable article. Generally, the number of citable articles in JCR corresponds to the number of research and review articles indexed in Web of Science for any particular journal, however, the two numbers don't always correspond. Impact factors were generated in Scopus by individually searching in the source title field each journal from the JCR ecology and environmental sciences lists. Dates were limited to 2005 through 2006. If Scopus didn't index a journal from the JCR list during this time period, the title was eliminated from further consideration. Once results were obtained for each journal, all the citations were selected and the citation tracker feature was used to determine the number of times these articles were cited in 2007. These numbers were placed in an Excel spreadsheet and were divided against the number of citable articles from JCR to come up with Scopus impact factors. Using the rank formula in Excel, the journals were ranked by both their JCR Scopus impact factors. For each journal, the Scopus rank was subtracted from the JCR rank to determine a change in rank. Finally, the rankings were entered into an online statistical application to determine their Spearman rank-order correlation coefficients, which determine statistical similarity between two lists (Lowry 2008). Results and Discussion Tables 1 and 2 show the JCR and Scopus impact factors, the JCR and Scopus ranks, and the change in rank from JCR to Scopus for the top 50 ecology and environmental sciences titles, respectively. Although several of the titles changed greatly in rank (e.g., Wildlife Monographs dropped 38 places from 17 in JCR to 55 in Scopus for the ecology list), the JCR and Scopus ranks had a high degree of statistical similarity. The Spearman rank-order correlation coefficient was .9818 for the ecology titles and .9823 for the environmental science titles. Several journals warrant closer examination because of their changes in impact factor and rank. Although Bulletin of the American Museum of Natural History only dropped one spot from number one in JCR ecology rank to number two in Scopus ecology rank, its Scopus impact factor (16.385) was nearly 4.7 points less than its JCR impact factor (11.692). Several factors could account for this steep drop in impact factor. JCR listed 13 citable articles that determined its impact factor. Scopus, which did not match Web of Science in its indexing of this title, only returned 11 articles for this journal. Because we relied on the number of citable articles from JCR to create our Scopus impact factor, any title that isn't completely indexed by Scopus will have an underinflated impact factor. Had we used the 11 articles indexed in Scopus as our number of citable articles, the impact factor would have increased by over two points to 13.818. The high JCR impact factor for Bulletin of the American Museum of Natural History might also result from the data in JCR not matching the data in Web of Science. When we performed a publication search in Web of Science for Bulletin of the American Museum of Natural History for 2005 and 2006 and selected the "create citation report" feature, the database showed that the journal received only 153 cites in 2007; JCR (which is supposedly using Web of Science data) returned 213 cites. Using 153 cites instead of 213 cites creates a JCR impact factor of 11.769, which is more in line with the Scopus number. In case JCR is using incorrect citations in Web of Science to come up with a cite count, we also performed a search for the journal in the cited reference search. Articles from 2005 and 2006 were cited 203 times in 2007. If we use 203 cites, we get a JCR impact factor of 15.615. Because JCR doesn't indicate what cites are used in their data, we can't be certain if the discrepancy between the JCR and Scopus impacts is the result of incomplete indexing by Scopus, poor data in JCR, our methodology of relying on JCR for the number of citable articles, or some combination of all three. Wildlife Monographs was another journal that shows a major change between the two different sets of impact factors as that title dropped from 17th in the JCR rank to 55th in the Scopus rank for ecology titles. Both Web of Science and Scopus indexed 10 articles for the journal from 2005 to 2006, so incomplete indexing by Scopus can't be the cause of the difference. Using the "create citation report" feature in Web of Science, we got only 10 cites for Wildlife Monographs compared to the 39 listed in JCR. Using the cited reference search, we got 33 cites for the journal. If we rely on the Web of Science numbers instead of the JCR numbers, the impact factor for Wildlife Monographs is either 1.000 or 3.300, which places the Scopus impact factor of 1.700 nearly in the middle. Table 1: Impact Factors, Ranks, and Change in Ranks for the Top 50 Ecology Journals Journal Title JCR Impact Factor Scopus Impact Factor JCR Rank Scopus Rank Change in Rank Bulletin of the American Museum of Natural History 16.385 11.692 1 2 -1 Trends in Ecology & Evolution 14.797 14.993 2 1 1 Annual Review of Ecology Evolution and Systematics 10.340 10.440 3 3 0 Ecology Letters 8.204 8.292 4 4 0 Ecological Monographs 8.117 8.117 5 5 0 Molecular Ecology 5.169 5.151 6 6 0 Ecology 4.822 4.775 7 8 -1 Global Change Biology 4.786 5.000 8 7 1 American Naturalist 4.543 4.457 9 9 0 Evolution 4.502 4.441 10 10 0 Global Ecology and Biogeography 4.435 4.259 11 12 -1 Journal of Ecology 4.422 4.422 12 11 1 Frontiers in Ecology and the Environment 4.269 4.241 13 13 0 Journal of Applied Ecology 4.220 4.174 14 14 0 Conservation Biology 3.934 3.926 15 15 0 Journal of Evolutionary Biology 3.920 3.911 16 16 0 Wildlife Monographs 3.900 1.700 17 55 -38 Journal of Animal Ecology 3.747 3.720 18 17 1 Ecological Applications 3.571 3.562 19 18 1 Journal of Biogeography 3.539 3.503 20 19 1 Biological Conservation 3.296 3.354 21 20 1 Paleobiology 3.225 3.270 22 21 1 Functional Ecology 3.157 3.112 23 24 -1 Oikos 3.136 3.094 24 25 -1 Ecography 3.066 3.132 25 23 2 Perspectives in Plant Ecology Evolution and Systematics 3.043 3.261 26 22 4 Behavioral Ecology 3.018 3.075 27 26 1 Oecologia 2.973 3.022 28 28 0 Diversity and Distributions 2.965 3.035 29 27 2 Evolutionary Ecology 2.905 2.068 30 45 -15 Biogeosciences 2.813 2.520 31 34 -3 Behavioral Ecology and Sociobiology 2.754 2.699 32 29 3 Ecosystems 2.684 2.637 33 30 3 Microbial Ecology 2.558 2.522 34 33 1 Marine Ecology-Progress Series 2.546 2.533 35 32 3 Animal Conservation 2.495 2.392 36 36 0 Ecotoxicology 2.405 1.954 37 49 -12 Aquatic Microbial Ecology 2.385 2.366 38 37 1 Agriculture Ecosystems & Environment 2.308 2.505 39 35 4 Journal of Vegetation Science 2.251 2.152 40 43 -3 Basic and Applied Ecology 2.247 2.351 41 38 3 Journal of the North American Benthological Society 2.217 2.224 42 40 2 Ecology and Society 2.215 2.049 43 46 -3 Ecological Engineering 2.175 2.330 44 39 5 Biological Invasions 2.125 2.165 45 42 3 Ecological Modelling 2.077 2.196 46 41 5 Landscape Ecology 2.061 2.624 47 31 16 Theoretical Population Biology 1.950 1.917 48 50 -2 Journal of Chemical Ecology 1.941 1.959 49 48 1 Restoration Ecology 1.928 1.993 50 47 3 Table 2: Impact Factors, Ranks, and Change in Ranks for the Top 50 Environmental Sciences Journals Journal Title JCR Impact Factor Scopus Impact Factor JCR Rank Scopus Rank Change in Rank Environmental Health Perspectives 5.636 5.804 1 1 0 Global Change Biology 4.786 5.000 2 2 0 Critical Reviews in Environmental Science and Technology 4.615 4.615 3 3 0 Environmental Science & Technology 4.363 4.285 4 6 -2 Global Biogeochemical Cycles 4.335 3.947 5 8 -3 Frontiers in Ecology and the Environment 4.269 4.241 6 7 -1 Annual Review of Environment and Resources 4.036 4.357 7 5 2 Conservation Biology 3.934 3.926 8 9 -1 Global Environmental Change-Human and Policy Dimensions 3.915 4.559 9 4 5 Environmental Science and Pollution Research 3.894 3.173 10 17 -7 Ecological Applications 3.571 3.562 11 10 1 Water Research 3.427 3.223 12 15 -3 Biological Conservation 3.296 3.354 13 12 1 Applied Catalysis A-General 3.166 3.290 14 13 1 Environmental Pollution 3.135 3.231 15 14 1 Geobiology 3.114 2.864 16 21 -5 Remote Sensing of Environment 3.013 3.207 17 16 1 Journal of Environmental Science and Health Part C- Environmental Carcinogenesis & Ecotoxicology Reviews 3.000 3.429 18 11 7 Environmental Research 2.962 2.996 19 18 1 Climatic Change 2.890 2.826 20 22 -2 Environmental Chemistry 2.809 2.351 21 30 -9 Environment International 2.797 2.977 22 19 3 Chemosphere 2.739 2.820 23 23 0 International Journal of Hydrogen Energy 2.725 2.893 24 20 4 Atmospheric Environment 2.549 2.729 25 24 1 Biogeochemistry 2.534 2.592 26 26 0 Environmental Reviews 2.467 2.600 27 25 2 Ecotoxicology 2.405 1.954 28 46 -18 Environmental and Molecular Mutagenesis 2.361 2.340 29 31 -2 Aerosol Science and Technology 2.350 2.070 30 42 -12 Journal of Hazardous Materials 2.337 2.442 31 28 3 Marine Pollution Bulletin 2.334 2.368 32 29 3 Journal of Environmental Quality 2.331 2.218 33 36 -3 Environmental Toxicology and Chemistry 2.309 2.218 34 37 -3 Agriculture Ecosystems & Environment 2.308 2.505 35 27 8 Journal of Paleolimnology 2.287 2.182 36 39 -3 Science of the Total Environment 2.182 2.298 37 34 3 Ecological Engineering 2.175 2.330 38 32 6 Water Resources Research 2.154 1.914 39 48 -9 Estuaries 2.133 1.867 40 51 -11 Environmental Modelling & Software 2.099 2.293 41 35 6 Ecotoxicology and Environmental Safety 2.014 2.118 42 41 1 Journal of Industrial Ecology 1.962 2.308 43 33 10 Boreal Environment Research 1.951 2.122 44 40 4 Marine Environmental Research 1.930 1.930 45 47 -2 Journal of Aerosol Science 1.902 2.019 46 43 3 Energy Policy 1.901 2.185 47 38 9 Journal of Contaminant Hydrology 1.852 1.955 48 45 3 Journal of Environmental Monitoring 1.833 1.792 49 54 -5 Reviews of Environmental Contamination and Toxicology 1.818 1.818 50 52 -2 Conclusion Impact factors are useful for ranking and comparing scholarly journals, and are one of the most widely used indicators of journals' significance to the scholarly community. With all the weight that is given to the impact factor, it is important that these values do not significantly vary from one database's calculation to another. This study's comparison between JCR and Scopus impact factors in the fields of ecology and environmental sciences showed no statistically significant difference, which indicates that one or the other may be used for evaluative purposes. While most of the impact factors and journal ranks for Web of Science and Scopus were fairly similar, there were a few journals that had significantly different impact factors. In some instances, these differences could be accounted for by incomplete indexing in Scopus. Because we relied upon the number of citable articles in JCR to calculate our Scopus impact factors, our methodology may have underinflated the Scopus impact factor and rank for some titles. Our quick examination of the data in Web of Science also shows that number of cites used in JCR might be incorrect for some titles. Finally, the collection of titles indexed by the two citation databases can affect the differences in rank and impact factors seen in JCR and Scopus. Since JCR impact factors are the standard used by most journal advertisers and university administrators, we may now be confident that these values are close enough to Scopus impact factors to justify the use of a single database's impact factor. This is valuable information because it proves that it is not necessary to calculate Scopus impact factors in addition to the JCR data. It is, however, still important to use the JCR impact factor carefully. There are many variables that can affect the impact factor of a journal that must be considered when using Garfield's ratio to evaluate the importance of a journal. References Bakkalbasi, N., Bauer, K., Glover, J. & Wang, L. 2006. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries 3(1):7. Barschall, H.H. 1988. The cost-effectiveness of physics journals. Physics Today. 41(7):56-59. Coleman, A. 2007. Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology 58(8):1148-1161. Coleman, R. 1999. Impact factors: Use and abuse in biomedical research. The Anatomical Record 257:54-57. Garfield, W. 1972. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science 178(4060):471-479. Garfield, E. & Sher, I.H. 1963. New factors in the evaluation of scientific literature through citation indexing. American Documentation 14(3):195-201. Hirsch, J.E. 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America. 102(46):16569-16572. Lowry, R. 2008. VassarStats: Web Site for Statistical Computation. [Online]. Available: http://faculty.vassar.edu/lowry/VassarStats.html [July 8, 2008]. Monastersky, R. 2005. The number that's devouring science: The impact factor, once a simple way to rank scientific journals, has become an unyielding yardstick for hiring, tenure, and grants. The Chronicle of Higher Education 52(8):A12. Opthof, T. 1997. Sense and nonsense about the impact factor. Cardiovascular Research 33:1-7. Pislyakov, V. 2007. Comparing two "thermometers": Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. E-LIS. [Online]. Available: http://eprints.rclis.org/archive/00011865 [July 2, 2008]. Seglen, P.O. 1997. Why the impact factor should not be used for evaluating research. BMJ 314:498-502. Thomson Scientific. 1994. The Thomson Scientific impact factor. [Online]. Available: http://scientific.thomsonreuters.com/free/essays/journalcitationreports/impactfactor [June 27, 2008]. Werner, Y.L. 2006. The case of impact factor versus taxonomy: a proposal. Journal of Natural History 40(21-22):1285-1286. Previous Contents Next