Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals

      	Previous		Contents		Next
	Issues in Science and Technology Librarianship
	Summer 2008

	DOI:10.5062/F4FF3Q9G



      

      Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals
    
Edward Gray
  Public Services Librarian
teddy.gray@duke.edu    Sarah Z. Hodkinson
  Library Assistant
sarah.hodkinson@duke.edu

    Biological and Environmental Sciences Library
  Duke University
  Durham, North Carolina  


    


  

Copyright 2008, Edward Gray and Sarah Z. Hodkinson. Used with permission.  

Abstract
  	Impact factors for journals listed under the subject categories "ecology" and "environmental sciences" in the Journal Citation Reports database were calculated using citation data from the Scopus database. The journals were then ranked by their Scopus impact factor and compared to the ranked lists of the same journals derived from Journal Citations Reports. Although several titles varied significantly in impact factor and rank, the Journal Citation Reports and Scopus lists had a high degree of statistical similarity.
      

Introduction
    The Thomson Scientific impact factor is a popular tool that is used to measure the influence of scholarly journals. Nearly fifty years ago, Eugene Garfield developed this method as a way to use the Thomson Scientific (formerly Institute for Scientific Information, ISI) citation indexes to analyze the frequency with which the average journal article is cited (Garfield & Sher 1963; Garfield 1972). The impact factor of a journal is calculated by dividing the references cited in one year by the number of citable articles published in the same journal over the previous two years. This ratio is published annually in Thomson Scientific's Journal Citation Reports (JCR), along with a number of other quantitative tools for comparing and evaluating scholarly journals. For years, the impact factor has been used to "clarify the significance of absolute citation frequencies" (Thomson Scientific 1994), and as a result its applications have gained great significance in the world of academia (Monastersky 2005). Academic librarians frequently use impact factors to help them decide which journals are important enough to subscribe to and which subscriptions may be canceled (Barschall 1988; Coleman 2007).     While impact factor data were originally used extensively in journal marketing to rank and compare scholarly journals for prospective subscribers, it has become a way of ranking the scientists who publish articles in these journals. Since the importance of scientific research is notoriously difficult to evaluate quantitatively, many universities now make funding and tenure decisions based on the average impact factor value, or prestige, of the journals that a scientist has published in. This may seem to be a reliable tool to assess the importance of a particular scientist's research, but there have been numerous reports that show the dangers of overestimating the value of the impact factor (Seglen 1997; Opthof 1997; Coleman 1999). The impact factor of a journal may be artificially inflated when it frequently includes review articles and letters, and there is also a bias against articles that are not published in English. Journals that publish articles on botanical or zoological taxonomy may also have lower impact factors because they often rely on older references, and references to taxonomic names are not included in the references list at the ends of these papers (Werner 2006).   

    Since the widely used JCR impact factors are all based on data from journals indexed in Web of Science, the researchers of the present study were interested in comparing the impact factors calculated from citation data from another database. Elsevier's Scopus, which became available in 2004, has been considered a competitor to Web of Science because of its citation tracking capabilities (Bakkalbasi et al. 2006). Scopus does not rank journals according to "impact factor," using instead a measure called the h-index that was developed in 2005 by Hirsch to evaluate the impact of journals. However, the data required to calculate an impact factor using Garfield's ratio are all available in the Scopus database (Pislyakov 2007). If impact factor is the most widely used tool for evaluating the importance of a journal and thus the worth of its writers, it would be wise to determine whether or not calculated impact factors vary significantly between JCR and Scopus.

    

Methods
    Impact factors were obtained for the 116 ecology titles and the 160 environmental sciences titles listed in the 2007 edition of JCR. In this latest edition of JCR, cites to recent articles (the numerator of the impact factor equation) were from 2007, while citable articles (the denominator) were from 2005 and 2006. JCR does not state what constitutes a citable article. Generally, the number of citable articles in JCR corresponds to the number of research and review articles indexed in Web of Science for any particular journal, however, the two numbers don't always correspond.    Impact factors were generated in Scopus by individually searching in the source title field each journal from the JCR ecology and environmental sciences lists. Dates were limited to 2005 through 2006. If Scopus didn't index a journal from the JCR list during this time period, the title was eliminated from further consideration. Once results were obtained for each journal, all the citations were selected and the citation tracker feature was used to determine the number of times these articles were cited in 2007. These numbers were placed in an Excel spreadsheet and were divided against the number of citable articles from JCR to come up with Scopus impact factors.

    Using the rank formula in Excel, the journals were ranked by both their JCR Scopus impact factors. For each journal, the Scopus rank was subtracted from the JCR rank to determine a change in rank. Finally, the rankings were entered into an online statistical application to determine their Spearman rank-order correlation coefficients, which determine statistical similarity between two lists (Lowry 2008).

    

Results and Discussion
    Tables 1 and 2 show the JCR and Scopus impact factors, the JCR and Scopus ranks, and the change in rank from JCR to Scopus for the top 50 ecology and environmental sciences titles, respectively. Although several of the titles changed greatly in rank (e.g., Wildlife Monographs dropped 38 places from 17 in JCR to 55 in Scopus for the ecology list), the JCR and Scopus ranks had a high degree of statistical similarity. The Spearman rank-order correlation coefficient was .9818 for the ecology titles and .9823 for the environmental science titles.    Several journals warrant closer examination because of their changes in impact factor and rank. Although Bulletin of the American Museum of Natural History only dropped one spot from number one in JCR ecology rank to number two in Scopus ecology rank, its Scopus impact factor (16.385) was nearly 4.7 points less than its JCR impact factor (11.692). Several factors could account for this steep drop in impact factor. JCR listed 13 citable articles that determined its impact factor. Scopus, which did not match Web of Science in its indexing of this title, only returned 11 articles for this journal. Because we relied on the number of citable articles from JCR to create our Scopus impact factor, any title that isn't completely indexed by Scopus will have an underinflated impact factor. Had we used the 11 articles indexed in Scopus as our number of citable articles, the impact factor would have increased by over two points to 13.818.

    The high JCR impact factor for Bulletin of the American Museum of Natural History might also result from the data in JCR not matching the data in Web of Science. When we performed a publication search in Web of Science for Bulletin of the American Museum of Natural History for 2005 and 2006 and selected the "create citation report" feature, the database showed that the journal received only 153 cites in 2007; JCR (which is supposedly using Web of Science data) returned 213 cites. Using 153 cites instead of 213 cites creates a JCR impact factor of 11.769, which is more in line with the Scopus number. In case JCR is using incorrect citations in Web of Science to come up with a cite count, we also performed a search for the journal in the cited reference search. Articles from 2005 and 2006 were cited 203 times in 2007. If we use 203 cites, we get a JCR impact factor of 15.615. Because JCR doesn't indicate what cites are used in their data, we can't be certain if the discrepancy between the JCR and Scopus impacts is the result of incomplete indexing by Scopus, poor data in JCR, our methodology of relying on JCR for the number of citable articles, or some combination of all three.  

    Wildlife Monographs was another journal that shows a major change between the two different sets of impact factors as that title dropped from 17th in the JCR rank to 55th in the Scopus rank for ecology titles. Both Web of Science and Scopus indexed 10 articles for the journal from 2005 to 2006, so incomplete indexing by Scopus can't be the cause of the difference. Using the "create citation report" feature in Web of Science, we got only 10 cites for Wildlife Monographs compared to the 39 listed in JCR. Using the cited reference search, we got 33 cites for the journal. If we rely on the Web of Science numbers instead of the JCR numbers, the impact factor for Wildlife Monographs is either 1.000 or 3.300, which places the Scopus impact factor of 1.700 nearly in the middle.    

Table 1: Impact Factors, Ranks, and Change in Ranks for the Top 50 Ecology Journals
  	Journal Title
	JCR Impact Factor
	Scopus Impact Factor
	JCR Rank
	Scopus Rank
	Change in Rank

	Bulletin of the American Museum of Natural History
	16.385
	11.692
	1
	2
	-1

	
Trends in Ecology & Evolution 
        	14.797
	14.993
	2
	1
	1

	Annual Review of Ecology Evolution and Systematics 
	10.340
	10.440
	3
	3
	0

	Ecology Letters 
	8.204
	8.292
	4
	4
	0

	Ecological Monographs
	8.117
	8.117
	5
	5
	0

	Molecular Ecology
	5.169
	5.151
	6
	6
	0

	Ecology
	4.822
	4.775
	7
	8
	-1

	Global Change Biology
	4.786
	5.000
	8
	7
	1

	American Naturalist
	4.543
	4.457
	9
	9
	0

	Evolution 
	4.502
	4.441
	10
	10
	0

	Global Ecology and Biogeography
	4.435
	4.259
	11
	12
	-1

	Journal of Ecology
	4.422
	4.422
	12
	11
	1

	Frontiers in Ecology and the Environment
	4.269
	4.241
	13
	13
	0

	Journal of Applied Ecology
	4.220
	4.174
	14
	14
	0

	Conservation Biology
	3.934
	3.926
	15
	15
	0

	Journal of Evolutionary Biology
	3.920
	3.911
	16
	16
	0

	Wildlife Monographs
	3.900
	1.700
	17
	55
	-38

	Journal of Animal Ecology
	3.747
	3.720
	18
	17
	1

	Ecological Applications
	3.571
	3.562
	19
	18
	1

	Journal of Biogeography
	3.539
	3.503
	20
	19
	1

	Biological Conservation
	3.296
	3.354
	21
	20
	1

	Paleobiology
	3.225
	3.270
	22
	21
	1

	Functional Ecology
	3.157
	3.112
	23
	24
	-1

	Oikos
	3.136
	3.094
	24
	25
	-1

	Ecography
	3.066
	3.132
	25
	23
	2

	Perspectives in Plant Ecology Evolution and Systematics
	3.043
	3.261
	26
	22
	4

	Behavioral Ecology
	3.018
	3.075
	27
	26
	1

	Oecologia
	2.973
	3.022
	28
	28
	0

	Diversity and Distributions
	2.965
	3.035
	29
	27
	2

	Evolutionary Ecology
	2.905
	2.068
	30
	45
	-15

	Biogeosciences
	2.813
	2.520
	31
	34
	-3

	Behavioral Ecology and Sociobiology
	2.754
	2.699
	32
	29
	3

	Ecosystems
	2.684
	2.637
	33
	30
	3

	Microbial Ecology
	2.558
	2.522
	34
	33
	1

	Marine Ecology-Progress Series
	2.546
	2.533
	35
	32
	3

	Animal Conservation
	2.495
	2.392
	36
	36
	0

	Ecotoxicology
	2.405
	1.954
	37
	49
	-12

	Aquatic Microbial Ecology
	2.385
	2.366
	38
	37
	1

	Agriculture Ecosystems & Environment
	2.308
	2.505
	39
	35
	4

	Journal of Vegetation Science
	2.251
	2.152
	40
	43
	-3

	Basic and Applied Ecology
	2.247
	2.351
	41
	38
	3

	Journal of the North American Benthological Society 
	2.217
	2.224
	42
	40
	2

	Ecology and Society
	2.215
	2.049
	43
	46
	-3

	Ecological Engineering
	2.175
	2.330
	44
	39
	5

	Biological Invasions
	2.125
	2.165
	45
	42
	3

	Ecological Modelling
	2.077
	2.196
	46
	41
	5

	Landscape Ecology
	2.061
	2.624
	47
	31
	16

	Theoretical Population Biology
	1.950
	1.917
	48
	50
	-2

	Journal of Chemical Ecology
	1.941
	1.959
	49
	48
	1

	Restoration Ecology
	1.928
	1.993
	50
	47
	3



   Table 2: Impact Factors, Ranks, and Change in Ranks for the Top 50 Environmental Sciences Journals
  	Journal Title
	JCR Impact Factor
	Scopus Impact Factor
	JCR Rank
	Scopus Rank
	Change in Rank

	Environmental Health Perspectives
	5.636
	5.804
	1
	1
	0

	Global Change Biology
	4.786
	5.000
	2
	2
	0

	Critical Reviews in Environmental Science and Technology
	4.615
	4.615
	3
	3
	0

	Environmental Science & Technology 
	4.363
	4.285
	4
	6
	-2

	Global Biogeochemical Cycles 
	4.335
	3.947
	5
	8
	-3

	Frontiers in Ecology and the Environment
	4.269
	4.241
	6
	7
	-1

	Annual Review of Environment and Resources
	4.036
	4.357
	7
	5
	2

	Conservation Biology
	3.934
	3.926
	8
	9
	-1

	Global Environmental Change-Human and Policy Dimensions
	3.915
	4.559
	9
	4
	5

	Environmental Science and Pollution Research
	3.894
	3.173
	10
	17
	-7

	Ecological Applications
	3.571
	3.562
	11
	10
	1

	Water Research
	3.427
	3.223
	12
	15
	-3

	Biological Conservation
	3.296
	3.354
	13
	12
	1

	Applied Catalysis A-General
	3.166
	3.290
	14
	13
	1

	Environmental Pollution
	3.135
	3.231
	15
	14
	1

	Geobiology
	3.114
	2.864
	16
	21
	-5

	Remote Sensing of Environment
	3.013
	3.207
	17
	16
	1

	Journal of Environmental Science and Health Part C- Environmental Carcinogenesis & Ecotoxicology Reviews
	3.000
	3.429
	18
	11
	7

	Environmental Research
	2.962
	2.996
	19
	18
	1

	Climatic Change
	2.890
	2.826
	20
	22
	-2

	Environmental Chemistry
	2.809
	2.351
	21
	30
	-9

	Environment International
	2.797
	2.977
	22
	19
	3

	Chemosphere
	2.739
	2.820
	23
	23
	0

	International Journal of Hydrogen Energy
	2.725
	2.893
	24
	20
	4

	Atmospheric Environment
	2.549
	2.729
	25
	24
	1

	Biogeochemistry
	2.534
	2.592
	26
	26
	0

	Environmental Reviews
	2.467
	2.600
	27
	25
	2

	Ecotoxicology
	2.405
	1.954
	28
	46
	-18

	Environmental and Molecular Mutagenesis
	2.361
	2.340
	29
	31
	-2

	Aerosol Science and Technology
	2.350
	2.070
	30
	42
	-12

	Journal of Hazardous Materials 
	2.337
	2.442
	31
	28
	3

	Marine Pollution Bulletin
	2.334
	2.368
	32
	29
	3

	Journal of Environmental Quality
	2.331
	2.218
	33
	36
	-3

	Environmental Toxicology and Chemistry
	2.309
	2.218
	34
	37
	-3

	Agriculture Ecosystems & Environment
	2.308
	2.505
	35
	27
	8

	Journal of Paleolimnology
	2.287
	2.182
	36
	39
	-3

	Science of the Total Environment
	2.182
	2.298
	37
	34
	3

	Ecological Engineering
	2.175
	2.330
	38
	32
	6

	Water Resources Research
	2.154
	1.914
	39
	48
	-9

	Estuaries
	2.133
	1.867
	40
	51
	-11

	Environmental Modelling & Software
	2.099
	2.293
	41
	35
	6

	Ecotoxicology and Environmental Safety 
	2.014
	2.118
	42
	41
	1

	Journal of Industrial Ecology
	1.962
	2.308
	43
	33
	10

	Boreal Environment Research
	1.951
	2.122
	44
	40
	4

	Marine Environmental Research
	1.930
	1.930
	45
	47
	-2

	Journal of Aerosol Science
	1.902
	2.019
	46
	43
	3

	Energy Policy
	1.901
	2.185
	47
	38
	9

	Journal of Contaminant Hydrology
	1.852
	1.955
	48
	45
	3

	Journal of Environmental Monitoring
	1.833
	1.792
	49
	54
	-5

	Reviews of Environmental Contamination and Toxicology
	1.818
	1.818
	50
	52
	-2



  

Conclusion
    Impact factors are useful for ranking and comparing scholarly journals, and are one of the most widely used indicators of journals' significance to the scholarly community. With all the weight that is given to the impact factor, it is important that these values do not significantly vary from one database's calculation to another. This study's comparison between JCR and Scopus impact factors in the fields of ecology and environmental sciences showed no statistically significant difference, which indicates that one or the other may be used for evaluative purposes.     	While most of the impact factors and journal ranks for Web of Science and Scopus were fairly similar, there were a few journals that had significantly different impact factors. In some instances, these differences could be accounted for by incomplete indexing in Scopus. Because we relied upon the number of citable articles in JCR to calculate our Scopus impact factors, our methodology may have underinflated the Scopus impact factor and rank for some titles. Our quick examination of the data in Web of Science also shows that number of cites used in JCR might be incorrect for some titles. Finally, the collection of titles indexed by the two citation databases can affect the differences in rank and impact factors seen in JCR and Scopus.  

  	  	Since JCR impact factors are the standard used by most journal advertisers and university administrators, we may now be confident that these values are close enough to Scopus impact factors to justify the use of a single database's impact factor. This is valuable information because it proves that it is not necessary to calculate Scopus impact factors in addition to the JCR data. It is, however, still important to use the JCR impact factor carefully. There are many variables that can affect the impact factor of a journal that must be considered when using Garfield's ratio to evaluate the importance of a journal.

  	  

References
  	  Bakkalbasi, N., Bauer, K., Glover, J. & Wang, L. 2006. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries 3(1):7.    Barschall, H.H. 1988. The cost-effectiveness of physics journals. Physics Today. 41(7):56-59.

    Coleman, A. 2007. Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology 58(8):1148-1161.

    Coleman, R. 1999. Impact factors: Use and abuse in biomedical research. The Anatomical Record 257:54-57.

    Garfield, W. 1972. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science 178(4060):471-479.

    Garfield, E. & Sher, I.H. 1963. New factors in the evaluation of scientific literature through citation indexing. American Documentation 14(3):195-201. 

    Hirsch, J.E. 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America. 102(46):16569-16572.

    Lowry, R. 2008. VassarStats: Web Site for Statistical Computation. [Online]. Available: http://faculty.vassar.edu/lowry/VassarStats.html [July 8, 2008].

    Monastersky, R. 2005. The number that's devouring science: The impact factor, once a simple way to rank scientific journals, has become an unyielding yardstick for hiring, tenure, and grants. The Chronicle of Higher Education 52(8):A12.

    Opthof, T. 1997. Sense and nonsense about the impact factor. Cardiovascular Research 33:1-7.

    Pislyakov, V. 2007. Comparing two "thermometers": Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. E-LIS. [Online]. Available: http://eprints.rclis.org/archive/00011865 [July 2, 2008].

    Seglen, P.O. 1997. Why the impact factor should not be used for evaluating research. BMJ 314:498-502.

    Thomson Scientific. 1994. The Thomson Scientific impact factor. [Online]. Available: http://scientific.thomsonreuters.com/free/essays/journalcitationreports/impactfactor [June 27, 2008].

    Werner, Y.L. 2006. The case of impact factor versus taxonomy: a proposal. Journal of Natural History 40(21-22):1285-1286.

    

  	Previous		Contents		Next