Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals
Previous Contents Next
Issues in Science and Technology Librarianship
Summer 2008
DOI:10.5062/F4FF3Q9G
Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals
Edward Gray
Public Services Librarian
teddy.gray@duke.edu Sarah Z. Hodkinson
Library Assistant
sarah.hodkinson@duke.edu
Biological and Environmental Sciences Library
Duke University
Durham, North Carolina
Copyright 2008, Edward Gray and Sarah Z. Hodkinson. Used with permission.
Abstract
Impact factors for journals listed under the subject categories "ecology" and "environmental sciences" in the Journal Citation Reports database were calculated using citation data from the Scopus database. The journals were then ranked by their Scopus impact factor and compared to the ranked lists of the same journals derived from Journal Citations Reports. Although several titles varied significantly in impact factor and rank, the Journal Citation Reports and Scopus lists had a high degree of statistical similarity.
Introduction
The Thomson Scientific impact factor is a popular tool that is used to measure the influence of scholarly journals. Nearly fifty years ago, Eugene Garfield developed this method as a way to use the Thomson Scientific (formerly Institute for Scientific Information, ISI) citation indexes to analyze the frequency with which the average journal article is cited (Garfield & Sher 1963; Garfield 1972). The impact factor of a journal is calculated by dividing the references cited in one year by the number of citable articles published in the same journal over the previous two years. This ratio is published annually in Thomson Scientific's Journal Citation Reports (JCR), along with a number of other quantitative tools for comparing and evaluating scholarly journals. For years, the impact factor has been used to "clarify the significance of absolute citation frequencies" (Thomson Scientific 1994), and as a result its applications have gained great significance in the world of academia (Monastersky 2005). Academic librarians frequently use impact factors to help them decide which journals are important enough to subscribe to and which subscriptions may be canceled (Barschall 1988; Coleman 2007). While impact factor data were originally used extensively in journal marketing to rank and compare scholarly journals for prospective subscribers, it has become a way of ranking the scientists who publish articles in these journals. Since the importance of scientific research is notoriously difficult to evaluate quantitatively, many universities now make funding and tenure decisions based on the average impact factor value, or prestige, of the journals that a scientist has published in. This may seem to be a reliable tool to assess the importance of a particular scientist's research, but there have been numerous reports that show the dangers of overestimating the value of the impact factor (Seglen 1997; Opthof 1997; Coleman 1999). The impact factor of a journal may be artificially inflated when it frequently includes review articles and letters, and there is also a bias against articles that are not published in English. Journals that publish articles on botanical or zoological taxonomy may also have lower impact factors because they often rely on older references, and references to taxonomic names are not included in the references list at the ends of these papers (Werner 2006).
Since the widely used JCR impact factors are all based on data from journals indexed in Web of Science, the researchers of the present study were interested in comparing the impact factors calculated from citation data from another database. Elsevier's Scopus, which became available in 2004, has been considered a competitor to Web of Science because of its citation tracking capabilities (Bakkalbasi et al. 2006). Scopus does not rank journals according to "impact factor," using instead a measure called the h-index that was developed in 2005 by Hirsch to evaluate the impact of journals. However, the data required to calculate an impact factor using Garfield's ratio are all available in the Scopus database (Pislyakov 2007). If impact factor is the most widely used tool for evaluating the importance of a journal and thus the worth of its writers, it would be wise to determine whether or not calculated impact factors vary significantly between JCR and Scopus.
Methods
Impact factors were obtained for the 116 ecology titles and the 160 environmental sciences titles listed in the 2007 edition of JCR. In this latest edition of JCR, cites to recent articles (the numerator of the impact factor equation) were from 2007, while citable articles (the denominator) were from 2005 and 2006. JCR does not state what constitutes a citable article. Generally, the number of citable articles in JCR corresponds to the number of research and review articles indexed in Web of Science for any particular journal, however, the two numbers don't always correspond. Impact factors were generated in Scopus by individually searching in the source title field each journal from the JCR ecology and environmental sciences lists. Dates were limited to 2005 through 2006. If Scopus didn't index a journal from the JCR list during this time period, the title was eliminated from further consideration. Once results were obtained for each journal, all the citations were selected and the citation tracker feature was used to determine the number of times these articles were cited in 2007. These numbers were placed in an Excel spreadsheet and were divided against the number of citable articles from JCR to come up with Scopus impact factors.
Using the rank formula in Excel, the journals were ranked by both their JCR Scopus impact factors. For each journal, the Scopus rank was subtracted from the JCR rank to determine a change in rank. Finally, the rankings were entered into an online statistical application to determine their Spearman rank-order correlation coefficients, which determine statistical similarity between two lists (Lowry 2008).
Results and Discussion
Tables 1 and 2 show the JCR and Scopus impact factors, the JCR and Scopus ranks, and the change in rank from JCR to Scopus for the top 50 ecology and environmental sciences titles, respectively. Although several of the titles changed greatly in rank (e.g., Wildlife Monographs dropped 38 places from 17 in JCR to 55 in Scopus for the ecology list), the JCR and Scopus ranks had a high degree of statistical similarity. The Spearman rank-order correlation coefficient was .9818 for the ecology titles and .9823 for the environmental science titles. Several journals warrant closer examination because of their changes in impact factor and rank. Although Bulletin of the American Museum of Natural History only dropped one spot from number one in JCR ecology rank to number two in Scopus ecology rank, its Scopus impact factor (16.385) was nearly 4.7 points less than its JCR impact factor (11.692). Several factors could account for this steep drop in impact factor. JCR listed 13 citable articles that determined its impact factor. Scopus, which did not match Web of Science in its indexing of this title, only returned 11 articles for this journal. Because we relied on the number of citable articles from JCR to create our Scopus impact factor, any title that isn't completely indexed by Scopus will have an underinflated impact factor. Had we used the 11 articles indexed in Scopus as our number of citable articles, the impact factor would have increased by over two points to 13.818.
The high JCR impact factor for Bulletin of the American Museum of Natural History might also result from the data in JCR not matching the data in Web of Science. When we performed a publication search in Web of Science for Bulletin of the American Museum of Natural History for 2005 and 2006 and selected the "create citation report" feature, the database showed that the journal received only 153 cites in 2007; JCR (which is supposedly using Web of Science data) returned 213 cites. Using 153 cites instead of 213 cites creates a JCR impact factor of 11.769, which is more in line with the Scopus number. In case JCR is using incorrect citations in Web of Science to come up with a cite count, we also performed a search for the journal in the cited reference search. Articles from 2005 and 2006 were cited 203 times in 2007. If we use 203 cites, we get a JCR impact factor of 15.615. Because JCR doesn't indicate what cites are used in their data, we can't be certain if the discrepancy between the JCR and Scopus impacts is the result of incomplete indexing by Scopus, poor data in JCR, our methodology of relying on JCR for the number of citable articles, or some combination of all three.
Wildlife Monographs was another journal that shows a major change between the two different sets of impact factors as that title dropped from 17th in the JCR rank to 55th in the Scopus rank for ecology titles. Both Web of Science and Scopus indexed 10 articles for the journal from 2005 to 2006, so incomplete indexing by Scopus can't be the cause of the difference. Using the "create citation report" feature in Web of Science, we got only 10 cites for Wildlife Monographs compared to the 39 listed in JCR. Using the cited reference search, we got 33 cites for the journal. If we rely on the Web of Science numbers instead of the JCR numbers, the impact factor for Wildlife Monographs is either 1.000 or 3.300, which places the Scopus impact factor of 1.700 nearly in the middle.
Table 1: Impact Factors, Ranks, and Change in Ranks for the Top 50 Ecology Journals
Journal Title
JCR Impact Factor
Scopus Impact Factor
JCR Rank
Scopus Rank
Change in Rank
Bulletin of the American Museum of Natural History
16.385
11.692
1
2
-1
Trends in Ecology & Evolution
14.797
14.993
2
1
1
Annual Review of Ecology Evolution and Systematics
10.340
10.440
3
3
0
Ecology Letters
8.204
8.292
4
4
0
Ecological Monographs
8.117
8.117
5
5
0
Molecular Ecology
5.169
5.151
6
6
0
Ecology
4.822
4.775
7
8
-1
Global Change Biology
4.786
5.000
8
7
1
American Naturalist
4.543
4.457
9
9
0
Evolution
4.502
4.441
10
10
0
Global Ecology and Biogeography
4.435
4.259
11
12
-1
Journal of Ecology
4.422
4.422
12
11
1
Frontiers in Ecology and the Environment
4.269
4.241
13
13
0
Journal of Applied Ecology
4.220
4.174
14
14
0
Conservation Biology
3.934
3.926
15
15
0
Journal of Evolutionary Biology
3.920
3.911
16
16
0
Wildlife Monographs
3.900
1.700
17
55
-38
Journal of Animal Ecology
3.747
3.720
18
17
1
Ecological Applications
3.571
3.562
19
18
1
Journal of Biogeography
3.539
3.503
20
19
1
Biological Conservation
3.296
3.354
21
20
1
Paleobiology
3.225
3.270
22
21
1
Functional Ecology
3.157
3.112
23
24
-1
Oikos
3.136
3.094
24
25
-1
Ecography
3.066
3.132
25
23
2
Perspectives in Plant Ecology Evolution and Systematics
3.043
3.261
26
22
4
Behavioral Ecology
3.018
3.075
27
26
1
Oecologia
2.973
3.022
28
28
0
Diversity and Distributions
2.965
3.035
29
27
2
Evolutionary Ecology
2.905
2.068
30
45
-15
Biogeosciences
2.813
2.520
31
34
-3
Behavioral Ecology and Sociobiology
2.754
2.699
32
29
3
Ecosystems
2.684
2.637
33
30
3
Microbial Ecology
2.558
2.522
34
33
1
Marine Ecology-Progress Series
2.546
2.533
35
32
3
Animal Conservation
2.495
2.392
36
36
0
Ecotoxicology
2.405
1.954
37
49
-12
Aquatic Microbial Ecology
2.385
2.366
38
37
1
Agriculture Ecosystems & Environment
2.308
2.505
39
35
4
Journal of Vegetation Science
2.251
2.152
40
43
-3
Basic and Applied Ecology
2.247
2.351
41
38
3
Journal of the North American Benthological Society
2.217
2.224
42
40
2
Ecology and Society
2.215
2.049
43
46
-3
Ecological Engineering
2.175
2.330
44
39
5
Biological Invasions
2.125
2.165
45
42
3
Ecological Modelling
2.077
2.196
46
41
5
Landscape Ecology
2.061
2.624
47
31
16
Theoretical Population Biology
1.950
1.917
48
50
-2
Journal of Chemical Ecology
1.941
1.959
49
48
1
Restoration Ecology
1.928
1.993
50
47
3
Table 2: Impact Factors, Ranks, and Change in Ranks for the Top 50 Environmental Sciences Journals
Journal Title
JCR Impact Factor
Scopus Impact Factor
JCR Rank
Scopus Rank
Change in Rank
Environmental Health Perspectives
5.636
5.804
1
1
0
Global Change Biology
4.786
5.000
2
2
0
Critical Reviews in Environmental Science and Technology
4.615
4.615
3
3
0
Environmental Science & Technology
4.363
4.285
4
6
-2
Global Biogeochemical Cycles
4.335
3.947
5
8
-3
Frontiers in Ecology and the Environment
4.269
4.241
6
7
-1
Annual Review of Environment and Resources
4.036
4.357
7
5
2
Conservation Biology
3.934
3.926
8
9
-1
Global Environmental Change-Human and Policy Dimensions
3.915
4.559
9
4
5
Environmental Science and Pollution Research
3.894
3.173
10
17
-7
Ecological Applications
3.571
3.562
11
10
1
Water Research
3.427
3.223
12
15
-3
Biological Conservation
3.296
3.354
13
12
1
Applied Catalysis A-General
3.166
3.290
14
13
1
Environmental Pollution
3.135
3.231
15
14
1
Geobiology
3.114
2.864
16
21
-5
Remote Sensing of Environment
3.013
3.207
17
16
1
Journal of Environmental Science and Health Part C- Environmental Carcinogenesis & Ecotoxicology Reviews
3.000
3.429
18
11
7
Environmental Research
2.962
2.996
19
18
1
Climatic Change
2.890
2.826
20
22
-2
Environmental Chemistry
2.809
2.351
21
30
-9
Environment International
2.797
2.977
22
19
3
Chemosphere
2.739
2.820
23
23
0
International Journal of Hydrogen Energy
2.725
2.893
24
20
4
Atmospheric Environment
2.549
2.729
25
24
1
Biogeochemistry
2.534
2.592
26
26
0
Environmental Reviews
2.467
2.600
27
25
2
Ecotoxicology
2.405
1.954
28
46
-18
Environmental and Molecular Mutagenesis
2.361
2.340
29
31
-2
Aerosol Science and Technology
2.350
2.070
30
42
-12
Journal of Hazardous Materials
2.337
2.442
31
28
3
Marine Pollution Bulletin
2.334
2.368
32
29
3
Journal of Environmental Quality
2.331
2.218
33
36
-3
Environmental Toxicology and Chemistry
2.309
2.218
34
37
-3
Agriculture Ecosystems & Environment
2.308
2.505
35
27
8
Journal of Paleolimnology
2.287
2.182
36
39
-3
Science of the Total Environment
2.182
2.298
37
34
3
Ecological Engineering
2.175
2.330
38
32
6
Water Resources Research
2.154
1.914
39
48
-9
Estuaries
2.133
1.867
40
51
-11
Environmental Modelling & Software
2.099
2.293
41
35
6
Ecotoxicology and Environmental Safety
2.014
2.118
42
41
1
Journal of Industrial Ecology
1.962
2.308
43
33
10
Boreal Environment Research
1.951
2.122
44
40
4
Marine Environmental Research
1.930
1.930
45
47
-2
Journal of Aerosol Science
1.902
2.019
46
43
3
Energy Policy
1.901
2.185
47
38
9
Journal of Contaminant Hydrology
1.852
1.955
48
45
3
Journal of Environmental Monitoring
1.833
1.792
49
54
-5
Reviews of Environmental Contamination and Toxicology
1.818
1.818
50
52
-2
Conclusion
Impact factors are useful for ranking and comparing scholarly journals, and are one of the most widely used indicators of journals' significance to the scholarly community. With all the weight that is given to the impact factor, it is important that these values do not significantly vary from one database's calculation to another. This study's comparison between JCR and Scopus impact factors in the fields of ecology and environmental sciences showed no statistically significant difference, which indicates that one or the other may be used for evaluative purposes. While most of the impact factors and journal ranks for Web of Science and Scopus were fairly similar, there were a few journals that had significantly different impact factors. In some instances, these differences could be accounted for by incomplete indexing in Scopus. Because we relied upon the number of citable articles in JCR to calculate our Scopus impact factors, our methodology may have underinflated the Scopus impact factor and rank for some titles. Our quick examination of the data in Web of Science also shows that number of cites used in JCR might be incorrect for some titles. Finally, the collection of titles indexed by the two citation databases can affect the differences in rank and impact factors seen in JCR and Scopus.
Since JCR impact factors are the standard used by most journal advertisers and university administrators, we may now be confident that these values are close enough to Scopus impact factors to justify the use of a single database's impact factor. This is valuable information because it proves that it is not necessary to calculate Scopus impact factors in addition to the JCR data. It is, however, still important to use the JCR impact factor carefully. There are many variables that can affect the impact factor of a journal that must be considered when using Garfield's ratio to evaluate the importance of a journal.
References
Bakkalbasi, N., Bauer, K., Glover, J. & Wang, L. 2006. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries 3(1):7. Barschall, H.H. 1988. The cost-effectiveness of physics journals. Physics Today. 41(7):56-59.
Coleman, A. 2007. Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology 58(8):1148-1161.
Coleman, R. 1999. Impact factors: Use and abuse in biomedical research. The Anatomical Record 257:54-57.
Garfield, W. 1972. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science 178(4060):471-479.
Garfield, E. & Sher, I.H. 1963. New factors in the evaluation of scientific literature through citation indexing. American Documentation 14(3):195-201.
Hirsch, J.E. 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America. 102(46):16569-16572.
Lowry, R. 2008. VassarStats: Web Site for Statistical Computation. [Online]. Available: http://faculty.vassar.edu/lowry/VassarStats.html [July 8, 2008].
Monastersky, R. 2005. The number that's devouring science: The impact factor, once a simple way to rank scientific journals, has become an unyielding yardstick for hiring, tenure, and grants. The Chronicle of Higher Education 52(8):A12.
Opthof, T. 1997. Sense and nonsense about the impact factor. Cardiovascular Research 33:1-7.
Pislyakov, V. 2007. Comparing two "thermometers": Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. E-LIS. [Online]. Available: http://eprints.rclis.org/archive/00011865 [July 2, 2008].
Seglen, P.O. 1997. Why the impact factor should not be used for evaluating research. BMJ 314:498-502.
Thomson Scientific. 1994. The Thomson Scientific impact factor. [Online]. Available: http://scientific.thomsonreuters.com/free/essays/journalcitationreports/impactfactor [June 27, 2008].
Werner, Y.L. 2006. The case of impact factor versus taxonomy: a proposal. Journal of Natural History 40(21-22):1285-1286.
Previous Contents Next