key: cord-0005413-s9gtcbq1 authors: Groneberg, David A.; Klingelhöfer, Doris; Brüggmann, Dörthe; Scutaru, Cristian; Fischer, Axel; Quarcoo, David title: New quality and quantity indices in science (NewQIS): results of the first decade—project progress review date: 2019-07-13 journal: Scientometrics DOI: 10.1007/s11192-019-03188-8 sha: 34b608e247d160f121e265962ec2368be035bd79 doc_id: 5413 cord_uid: s9gtcbq1 Strategies employing information science and scientometric approaches were introduced to science policy and management over the past decades. As a rapidly evolving field, new bibliometric parameters are proposed and discussed continuously and the fields also benefits from the introduction of novel visualization techniques. The present article summarizes the experiences with a platform that combines geographical mapping with scientometrics. It was established between 2005 and 2008 at the Charité in Berlin and termed “New Quality and Quantity Indices in Science” (NewQIS), consisting of the integration of common scientometric parameters such as the h-index and novel visualization techniques including density equalizing mapping. NewQIS was used to assess socio-economic important fields of medicine and sciences. Within NewQIS studies, research activities, citation patterns and their relation to socio-economic figures were analyzed with regard to time periods, countries, continents or even single cities. Within the decade after its establishment, more than 80 NewQIS articles were peer-reviewed and published. Being a non-funded low budget project, it was used by many medical students to conduct their MD thesis. The narrow technical frame led to the chance of a comparison of research output between different fields of science. This article summarizes NewQIS 1.0 activities, discusses its limits and gives a look into the future of NewQIS 2.0 with a target of 200 evaluated entities of the biomedical field of sciences. Academic science is big business and big money. Billions of US-dollars (USD), Euros and other currencies are channeled into academic science every year. As a matter of fact, the decision makers-politicians and career officials-want to know about the fate of funding: Did it work? What was done? How much was done? Who did it? Who did the most? In order to answer these questions, the field of scientometrics and bibliometric offer convenient but debated benchmarking parameters including the impact factor of a scientific journal. Since the journal impact factor is a very superficial measure with no direct relation to the quality of a single scientific work (Carey 2016; Casadevall and Fang 2014) but only providing information about the performance of a specific journal over a relatively short term, other more sophisticated parameters were developed such as the Hirsch (H) index (Hirsch 2005 (Hirsch , 2007 . They also include a count of the individual citations that a scientific article receives. However, also the H-index is debatable and should not to be used for any purpose (Bertoli-Barsotti and Lando 2017a, b; Bornmann and Leydesdorff 2018) . In this respect, experts in the field have coined the expression of amateur bibliometricians describing the uncritical use of bibliometric tools (Bornmann and Leydesdorff 2014) . With these benchmarking options at hand, other questions arise for decision makers: Can we allocate the funding towards a direction that those who did most-get more funding in order to increase their productivity? To which extend can we do so? Is there a ceiling effect? I.e. by which extend is the total (not relative) productivity increased, if we allocate 2 staff positions to a research group which consist of 2 scientists (making a total of 4 scientists then) in comparison to the allocation of 2 staff positions to a group consisting of 20 scientists or of 50 scientists (making it 22 or 52, respectively). Measured in citations? Or in accumulated impact factors or whatsoever? These questions are linked to the so called Matthew effect (Merton 1968 ): Those who have most get even more. When counting, measuring and benchmarking are done by the use of superficial parameters in an uncritical way by amateur bibliometricians, and (intramural and extramural) funding is allocated on the basis of who performs best in those superficial counts (i.e. total accumulated impact factor count) these questions critically target freedom of research: Scientists or fields who do not produce measurable amounts of superficial parameters such as accumulated journal impact factors will suffer (Lowy 1997) . Further to the question of funding allocation, also career opportunities are critically dependent on bibliometric benchmarking processes and it is common (but critically debated) law: publish or perish (in high impact factor journals) (Jokstad 2016; Publish or perish 2015; Bergquist et al. 2018) . Taking these aspects into account, it is obvious that scientometric markers need to be used only with great caution. They should not be easily used to compare scientists of different ages and different fields, institutions or areas with the purpose to cut off funding since research should only be interpreted for quality on the individual level of a published piece of work. Still, bibliometric parameters can be used to assess gross information contents and evolution of scientific fields over longer periods of time. It was exactly this purpose when in the years 2005-2009 a new project was started at the Charité in Berlin (Borger et al. 2008; Groneberg-Kloft et al. 2008a , b, 2009a : A platform termed NewQIS (1.0) was constructed to establish a new approach to visualize research quantity and quality indices (Groneberg-Kloft et al. 2009b, c) . NewQIS 1.0 should be used to assess research activities for (1) distinct areas of science, for (2) single institutions, for (3) single countries, or for (4) single time periods (Fig. 1) . The platform was intended to be a sound basis for future NewQIS studies in all areas of medicine and science. In the following, we (a) briefly summarize the technical basis, and (b) present an overview of the studies and MD theses which were performed on the basis of NewQIS. One important aspect of NewQIS 1.0 was to establish a unified technical platform that enables researchers from different fields of science to be able to assess their area of interest. Therefore, a study panel was formed that decided upon the feasibility of the proposed area. The usual applicants were medical students who-in their duty to conduct an MD thesis-submitted search topics to the study panel. After review and affirmation, the NewQIS analysis were performed and raw data was transferred to the applicants for their purpose. In NewQIS studies, data is usually retrieved from the Web of Science (WoS) database, i.e. (Kusma et al. 2009 ). The reason to choose WoS was the ability to perform a citation analysis. This was not possible with PubMed data files. Depending on the topic of the NewQIS study, the search terms that are entered in the search field consist of various terms which are linked together with Boolean operators such as "AND", "OR", "NOT", i.e. (Glynn et al. 2010) . Depending on the date of the research, the amount of publications and the focus of the research, the evaluation time span covers periods from 1900 until today. Usually, the year in which the NewQIS project is performed, is left out because of incomplete data acquisitions for that given year, i.e. (Al-Mutawakel et al. 2010 ). The large majority of NewQIS projects focus on a single field of medicine such as a disease and put a focus on the global landscape of research on this particular disease. Thus, the following parameters are usually analyzed (Fig. 1 ). Quantity parameters: Productivity • Total number of published items (i.e. Scutaru et al. 2010b) • Country specific number of publishes items (i.e. Vitzthum et al. 2010a) (Semi-)qualitative parameters: Usually, high quality research is characterized by a high number of citations. Therefore, the following citation parameters were also analyzed in the NewQIS projects: • Total number of citations • Total number of citations per country • Country-specific h index • Country-specific average citation rate per article Cooperation parameters: A key instrument of NewQIS is to visualize different levels of collaboration. This includes either collaborations between single scientists, countries or institutions. The field of RSV (respiratory syncytial virus) research can give an example how this is achieved: After having identified all relevant RSV-associated publications, the collaborative studies were related to their countries of origin. Publications with two or more authors affiliated to the same country were counted only once for the total number of collaborations of this particular country (Bruggmann et al. 2017c ). If an author had two affiliations, these were counted for every country mentioned in the affiliations. Connecting vectors visualized these co-operations; their width and shade of grey reflected the number of joint publications (Bruggmann et al. 2017c) . Figure 2 illustrates international collaborations for RSV research. The above listed parameters can also be found in other publications using other approaches (Burak Atci et al. 2019; Ekundayo and Okoh 2018) . A specific purpose of NewQIS was to combine these bibliometric parameters with visualization techniques in order to provide a picture of the global landscape of different research aspects. Among different available techniques, density-equalizing map projections (DEMP) were chosen. As elegantly described by Gastner and Newman, map makers searched for a long a way to generate cartograms, in which the sizes of countries appear in proportion to a chosen parameter such as their population (Gastner and Newman 2004) . For the purpose of NewQIS, these maps could be used to visualize research activities. As stated by Gastner and Newman, in order to scale countries and still have them properly fit together, they need to be distorted, causing difficulties to read them. In 2004, a new method was proposed which was integrated to the NewQIS platform. With DEMPs being a part of NewQIS, the territories of countries were re-sized according to a particular variable, i.e. in proportion to the countries´ total number of published items regarding to a specific disease. Figure 3 shows examples of DEMPs published within NewQIS studies over the past decade. The distorted global landscape is usually characterized by a dominating USA and an enlarged European area as depicted in Fig. 3a for pulmonary hypertension research output (Gotting et al. 2017 ). However, there are also research areas in which also countries from other continental regions appear enlarged. This can be seen for snakebite envenoming research for Brazil as shown in Fig. 3b ). China-a rising star in many areas of science-does also appear in some NewQIS assessments prominently as shown for ovarian cancer research in Fig. 3c (Bruggmann et al. 2017d ). Concerning Asian countries, a previous assessment of 5527,558 articles has indicated that Asian countries have largely different research focuses in comparison to Western countries (Groneberg-Kloft et al. 2008b) . In order to assess changes over the time, spatiotemporal analyses can also be performed by merging to a video consisting of different density-equalizing mapping (Groneberg-Kloft et al. 2009e ). The original concept of NewQIS was a low budget intramural platform which was established without major external funding. In order to be able to assess numerous fields of medicine, medical students were enabled to conduct their MD thesis within the NewQIS platform. The highly structured boundaries of the platform also served as a quality control Gotting et al. (2017) . b DEMP for snakebite envenoming research output. Data from Groneberg et al. (2016c) . c DEMP exemplifying prominent Chinese research activities in ovarian cancer research. Data from Bruggmann et al. (2017d) for the results of the thesis projects making scientific misconduct very difficult (since there was no possibility for the students to manipulate the algorithms applied by the platform). Since 2009, nearly 80 theses were completed using the methodology of the platform making NewQIS one of the most successful structured thesis programs in Germany. As tutors/mentors of the theses, seven associate/full professors served so far. Also, two technical tutors were present to oversee calculations and data management. Table 1 lists the medical thesis topics. Since 2008, more than 80 studies using the NewQIS platform were published after peer review. The majority of them based on medical thesis projects with the MD students being first authors in case of writing the manuscripts or co-authors of the scientific studies. The topics ranged from infectious diseases, infectious agents, to cancers, neurological or psychiatric disorders, lung diseases or other diseases. Apart from diseases, they also encompassed i.e. public health issues including tobacco control, medical procedures or techniques. In total, more than 1.6 million published articles related to specific search terms were analyzed for the above listed parameters. Table 2 provides an overview of the different NewQIS articles. There are numerous limitations present in every NewQIS-based study: (1) As with every other bibliometric approach, also NewQIS is limited to the data base it uses. Although producing global landscapes of research, it should never be forgotten that these pictures only delineate the research output which can be found in a specific data base (i.e. Web of Science) with a specific search term. Thus, all research not listed in the WoS and all research excluded by the search term (no search term can be absolutely perfect) is not included in the global landscape. This needs to be taken into account carefully when NewQIS results are interpreted. Especially the language bias constitutes an important problem: journals published in English have a higher chance of getting included to the data bases (Nieminen and Isohanni 1999) . Thus, non-English speaking countries are underrepresented concerning their research activities and important but regional data such as regional epidemiologic data is not identified (Pleger et al. 2014 ). (2) A further limitation that needs to be addressed is the above Matthew effect mentioned above: Communication systems in science are directed towards a reward of highly productive and renowned scientists and institutions. This leads to a pyramidal citation scheme (Merton 1968; Pleger et al. 2014) . 3) The so-called (semi-)qualitative indicators that are used in NewQIS are parameters such as the total citations, citation rate, country-specific h-index. They need to be interpreted very carefully. As already earlier critically discussed, they are not real measures for the quality of individual research (Pleger et al. 2014) . In this respect, a recent study addressed the question if methodological quality and completeness of reporting are associated with citation-based measures of publication impact (Mackinnon et al. 2018) . The authors performed a secondary analysis of a systematic review of dementia biomarker studies. They reported that citation rates and 5-year journal impact factors Table 1 Medical thesis projects that applied the structured NewQIS program No. Year of exam Trost (2018) 3. Brich (2017) 6. Löhlein (2018) 12. Kröber (2018) 13. Weber (2018) 14. Lammer (2018) 16. Richter (2018) 17. Krempel (2017) 18. Mäule (2017) 19. Kayser (2017) 20. Ofosu (2015) 26. Depression, suicide, cannabis, bipolar disorder Vogelzang (2015) 27. Mund (2013) 29. No. Year Groneberg-Kloft et al. (2009a) 4. Groneberg-Kloft et al. 2009 Inter Groneberg (2019) appear to measure different dimensions. While citation rates were weakly associated with completeness of reporting, none of these metrics was related to methodological rigor. They suggested that high publication usage and journal outlet is not a guarantee of quality and readers should critically appraise all papers regardless of presumed impact (Mackinnon et al. 2018) . Therefore, qualitative aspects are better addressed by advanced meta-analysis approaches using i.e. Cochrane systematics (Stovold et al. 2014 ). Scientometrics as research area is a niche within science. Funding is difficult to acquire for scientometric projects. However, it is the long term aim of NewQIS to analyze about 200 different areas within the next decade and to repeat assessments in 5-to 10-year intervals of important areas in order to assess changes in global research activities. When counting the raw data analyzed in the first 100 projects, we approximately invested about 50,000 work hours. Without extramural funding, this was only achievable by the workforce of medical students who performed their MD projects within NewQIS. In contrast to peer reviewed scientific reports which have been published for different NewQIS studies, a German medical thesis usually encompasses a much longer manuscript with 80-100 pages. This has been achieved by the medical students by writing comprehensive introductions about the field of research they analyze within their thesis. Thereby, they demonstrate that they possess an extensive knowledge about their thesis project. This is a prerequisite to obtain an MD degree. Also, the thesis students have to write detailed descriptions of their methodological approach (the NewQIS techniques) in the methods sections of the thesis and they have to discuss limits of the methodology in the discussion sections of their thesis. This leads to two potential pitfalls: (1) In the case of the methods sections, the thesis students have to follow strictly the above described protocols of NewQIS. This technical overlap is important and a strength of the platform in order to facilitate the comparison of results between the different diseases studied. However, it can be anticipated, that the use of these stringent protocols in nearly 80 different thesis projects-all with different target areas, i.e. ranging from burnout syndrome (Fröhlich 2009 ) to bronchial asthma (Puk 2009 )-brings the same problem as rewriting a passage on the methodology of other highly structured techniques such as RT-PCR (reverse transcriptase-polymerase chain reaction) which has now been published more than 250,000 times according to the PubMed. As with nearly identical descriptions of PCR and other molecular biology methods which can be found in peer reviewed scientific papers, an overlapping wording does not represent an act of plagiarism but rather exemplifies the impossibility to reword a similar methods section for more than 80 times without overlapping sentences. This does also apply for the part of the discussion in which the methodology and its limitations are discussed. Addressing these issues, the international Committee on Publication Ethics (COPE) points to a guideline of BioMed Central editors which outlines the following: "Use of similar or identical phrases in methods sections where there are limited ways to describe a common method, (…), is not uncommon. In such cases, an element of text recycling is likely to be unavoidable in further publications using the same method. Editors should use their discretion when deciding how much overlap of methods text is acceptable, considering factors such as whether authors have been transparent and stated that the methods have already been described in detail elsewhere and provided a citation" (COPE) (https ://publi catio nethi cs.org/text-recyc ling-guide lines ). Therefore, to overcome this pitfall, peer-reviewed NewQIS studies cite previous studies because of the methodological similarities-which are a strength of the platform. Also, thesis students are urged to cite every other NewQIS thesis which used the platform and to declare that the used methodology is part of NewQIS and therefore similar (apart from i.e. the different search terms). (2) The introductions of the respective thesis usually follow the guidelines of up-to-date reviews i.e. on the disease which is analyzed for the thesis. In this respect, numerous introductions from NewQIS related thesis projects were also published as CME (continuing medical education) articles or as narrative reviews. Unfortunately, a recent analysis showed that within one thesis project, almost all parts of the introduction were copied by the student from the Wikipedia-a case of severe plagiarism that led to the deprivation of the Dr. med. degree (MD Thesis) of the student (Sudik 2011) . In order to prevent future cases of plagiarism, all medical thesis now need to be analyzed within a plagiarism check prior to the official submission of the thesis to the medical school. The NewQIS platform will be used as NewQIS 2.0 in a next decade of further scientometric studies. There will be the following issues: Project of 200 As stated earlier, NewQIS 2.0 is intended to encompass about 200 different search projects with all areas of medicine, life sciences and also other areas of science in the next 10 years. Also, projects carried out 10 years ago and reported worldwide research activities (in the Web of Science) until 2005, should now be repeated in order to investigate the development of scientific activities. New focuses Originally conceived as a tool to investigate publication activities in single areas of medicine, i.e. in different infectious diseases, NewQIS has also proven to be a valuable tool for other purposes, i.e. to analyze journals Groneberg et al. 2018) . Also, it could be used for the analysis of cities with regard to research activities of affiliations in these cities. A recent example was the so-called NewQIS-Wroclaw project that assessed scientific activities in the Central European Polish city in three different areas: biomedical research, chemical research and social sciences (Groneberg 2018a (Groneberg , b, 2019 and demonstrated a strong increase over the past decades. New parameters As introduced in the past years, NewQIS studies may also focus upon socio-economic features. In this respect, various economic key figures were used. I.e. two quotients were calculated to assess the scientific output of a specific country for RSV research (Bruggmann et al. 2017c ): (1) in relation to the number of inhabitants (Q1) (2) in relation to its economic power (as measured by the gross domestic product, GDP, Q2) (Bruggmann et al. 2017c ). Data regarding the population and GDP of investigated countries was obtained from 2012 The quotients were calculated as follows: 1. Articles/population index (Q1) = number of articles/population in million inhabitants 2. Articles/GDP index (Q2) = number of articles/GDP in 1000 billion US-Dollars Within the RSV research NewQIS study, also, all countries were classified into highincome, upper-middle-income, lower-middle-income and low-income groups according to World Bank definitions (Bruggmann et al. 2017c) . Then, the total number of RSV articles was related to the gross domestic expenditure on Research and Development (R&D in % of GDP) as well as to the number of researchers (per million inhabitants) affiliated to the investigated countries. For over 10 years, the NewQIS platform has been used as a tool for peer reviewed scientific studies and for medical thesis in order to study numerous fields of science. As NewQIS 2.0 the project now heads into the next decade with a variety of new aspects in focus such as detailed socio-economic analysis or gender aspects. Using density equalizing mapping projections thousands of new pictures of global research landscapes will be generated. With numerous novel aspects that have been introduced to NewQIS within the past years, the platform will be a helpful tool for different aspects of scientometrics in the future. Vergleich psychiatrischer Fachzeitschriften-Eine szientometrische Analyse. Thesis Glioblastoma multiforme: Eine szientometrische Analyse. Thesis Methicillin-resistenter Staphylococcus aureus (MRSA): Eine szientometrische Analyse. Thesis MRSA: A density-equalizing mapping analysis of the global research architecture Dental Implants-A scientometric analysis. Thesis: Goethe Universität Frankfurt Scientometric analysis of the world-wide research efforts concerning Leishmaniasis Publish or perish: Need for bibliometric and access reform Infektiöse Endokarditis: Eine szientometrische Analyse. Thesis The h-index as an almost-exact function of some basic statistics A theoretical model of the relationship between the h-index and other simple citation indicators Streptococcus: Eine szientometrische Analyse. Thesis Der "Body Mass Index Scientometric analysis of the BMI Models of asthma: Density-equalizing mapping and output benchmarking Scientometrics in a changing research landscape: Bibliometrics has become an integral part of research quality evaluation and has been changing the practice of research Count highly-cited papers instead of papers with h citations: Use normalized citation counts and compare "like with like"! Arbeitsunfälle mit Infektionsgefahr im Gesundheitswesen unter besonderer Berücksichtigung der Nadelstichverletzung-Eine szientometrische Analyse Lärm in der medizinischen Forschung-Eine szientometrische Studie. Thesis World-wide research architecture of vitamin D research: Density-equalizing mapping studies and socio-economic analysis Polycystic ovary syndrome: Analysis of the global research architecture using density equalizing mapping Endometriosis and its global research architecture: An in-depth density-equalizing mapping analysis Congenital toxoplasmosis: An in-depth density-equalizing mapping analysis to explore its global research architecture. Parasites Vectors, 8, 646 Human papilloma virus: Global research architecture assessed by density-equalizing mapping Ectopic pregnancy: Exploration of its global research architecture using density-equalising mapping and socioeconomic benchmarks Respiratory syncytial virus: A systematic scientometric analysis of the global publication output and the gender distribution of publishing authors Caesarean section-A density-equalizing mapping study to depict its global research architecture The uterine fibroid/myoma tumour: Analysis of the global research architecture using density-equalizing mapping World-wide architecture of osteoporosis research: Density-equalizing mapping studies and gender analysis Ovarian cancer: Density equalizing mapping of the global research architecture Global architecture of gestational diabetes research: Density-equalizing mapping studies and gender analysis Maternal depression research: Socioeconomic analysis and density-equalizing mapping of the global research architecture Yellow fever disease: Density equalizing mapping and gender analysis of international research output The top 50 most-cited articles on low-grade glioma: A bibliometric analysis Varizella-Zoster-Virus: Windpocken und Herpes Zoster: Eine szientometrische Analyse. Thesis Quantifying scientific merit: Is it time to transform the impact factor? Curare-a curative poison: A scientometric analysis Causes for the persistence of impact factor mania Blasenkrebs: Eine szientometrische Analyse. Thesis A global bibliometric analysis of Plesiomonas-related research Zystische fibrose: Szientometrische analyse einer genetischen Erkrankung. Thesis Adipositas-Bibliometrische Analyse einer globalen Epidemie. Thesis Bedeutung der Influenza für das Gesundheitswesen-Eine szientometrische Analyse. Thesis Influenza: A scientometric and densityequalizing analysis Karpaltunnelsyndrom: Eine szientometrische Analyse einer Erkrankung des muskuloskeletalen Systems. Thesis Biomedizinische Forschung in Tunesien-Eine szientometrische Analyse. Thesis Diffusion-based method for producing density-equalizing maps Type 2 diabetes research yield Giftschlangenbisse: Eine Untersuchung der Literatur von Gout: A critical analysis of scientific development Antineutrophil cytoplasmic antibody-associated vasculitides: A scientometric approach visualizing worldwide research activity A scientometric analysis of global research activity during the last 35 years Silicosis: Geographic changes in research: An analysis employing density-equalizing mapping Laryngeal cancer: Quantitative and qualitative assessment of research output Breast cancer research output, 1945-2008: A bibliometric and density-equalizing analysis Pulmonale Hypertonie: Eine szientometrische Analyse. Thesis Pulmonary hypertension: Scientometric analysis and density-equalizing mapping Eine szientometrische Analyse anhand zweier epithelialer Präkursorläsionen. Thesis A scientometric analysis of leukoplakia and erythroplakia Biomedical research in Wroclaw: A combined density-equalizing mapping and scientometric analysis Social sciences research in the Central European city of Wroclaw: A density-equalizing mapping analysis Academic chemistry and related fields in Wroclaw: Density-equalizing mapping studies over the past decades Glioblastoma research: US and international networking achievements Pancreatitis: Global research activities and gender imbalances: A scientometric approach using density-equalizing mapping The story behind Oncotarget? A bibliometric analysis Density equalizing mapping of obesity: Analysis of a global epidemic Analysis of the research architecture on the burnout syndrome Snakebite envenoming-A combined density equalizing mapping and scientometric analysis of the publication history Telemedicine-A scientometric and density equalizing analysis Drowning-a scientometric analysis and data acquisition of a constant global problem employing density equalizing mapping and scientometric benchmarking procedures Density equalizing mapping of the global tuberculosis research architecture Cough as a symptom and a disease entity: Scientometric analysis and density-equalizing calculations New quality and quantity indices in science (NewQIS): The study protocol of an international project Analysis of research architecture in the field of psychiatric rehabilitation. Zentralbl Arbeitsmed Arbeitsschutz Ergonomie Traffic medicine-related research: A scientometric analysis Interfield dysbalances in research input and output benchmarking: Visualisation by density equalizing procedures Quality and quantity indices in science: Use of visualization tools Interdisease comparison of research quantity and quality: Bronchial asthma and chronic obstructive pulmonary disease Analysis of research output parameters: Density equalizing mapping and citation trend analysis Institutional operating figures in basic and applied sciences: Scientometric analysis of quantitative output benchmarking Die kongenitale Toxoplasmose und ihr Erreger Toxoplasma gondii: Eine szientometrische Analyse. Thesis The h index and the identification of global benchmarks for breast cancer research output An index to quantify an individual's scientific research output Does the H index have predictive power Multiple Sklerose: Eine szientometrische Analyse. Thesis Public Health-Eine szientometrische Analyse verschiedener Publikationsformen und -medien. Thesis Quantitative und qualitative Analyse der biomedizinischen Forschung im Bereich der neurologischen und psychiatrischen Rehabilitation. Thesis Has the pressure to publish or perish in academia been overtaken by a need to also generate a prominent h-index? Fifteen years after September 11: Where is the medical research heading? A scientometric analysis Aflatoxin-Publication analysis of a global health threat Myasthenia gravis: Analyse der Forschungsentwicklung mit szientometrischen Methoden. Thesis Rotavirus: The architecture of the global research output and gender distribution. Thesis Rotavirus-Global research density equalizing mapping and gender analysis Die Bedeutung von SARS für das Gesundheitswesen: Eine szientometrische Analyse. Thesis Karies-Eine szientometrische Analyse der wissenschaftlichen Publikationen von 1900 bis 2012. Thesis Asbest und seine gesundheitlichen Effekte: Eine szientometrische Untersuchung. Thesis Tobacco control: Visualisation of research activity using density-equalizing mapping and scientometric benchmarking procedures Schizophrenie-Publikationsanalyse einer psychischen Erkrankung mittels szientometrischer Methoden. Thesis Development of the global schizophrenia research under epidemiological and socioeconomic influences Impact factor limits funding Alzheimer's disease-A scientometric analysis and data acquisition Are methodological quality and completeness of reporting associated with citation-based measures of publication impact? A secondary analysis of a systematic review of dementia biomarker studies Osteoporose: Eine szientometrische Analyse. Thesis Arthrose: Eine Analyse nach qualitativen und quantitativen szientometrischen Blickpunkten. Thesis The Matthew Effect in Science: The reward and communication systems of science are considered Goethe Universität Frankfurt Smoking and pregnancy: A scientometric analysis. Thesis Global research on smoking and pregnancy-a scientometric and gender analysis Humanes Immundefizienz-Virus (HIV): Eine szientometrische Analyse. Thesis Bias against European journals in medical publication Databases Arthrosis: A scientometric analysis Magnetic resonance imaging. Density equalizing mapping analysis of global research architecture Bakterielle Meningitis: Eine szientometrische Analyse. Thesis Bacterial meningitis: A density-equalizing mapping analysis of the global research architecture Publish or perish Ebola and its global research architecture-need for an improvement Telemedizin: Szientometrische Analysen und Evaluation der Bedeutung für das Gesundheitswesen. Thesis Vergleich gastroenterologischer Fachmagazine-Eine szientometrische Analyse. Thesis Ertrinkungsunfälle: Eine szientometrische Analyse und Datenerhebung zu einem globalen Problem. Thesis Hepatitis B-eine szientometrische Analyse. Thesis Hepatitis B: Global scientific development from a critical point of view Erythropoietin: Qualitative und quantitative Analyse nach szientometrischen Gesichtspunkten. Thesis Ulcerative colitis: A scientometric approach to the global research output and network Crohn's disease: A critical approach to publication procedures and citation behavior of the global research network A critical perspective on the global research activity in the field of bladder cancer Hirschsprung disease: Critical evaluation of the global research architecture employing scientometrics and density-equalizing mapping Sarcoidosis: A descriptive approach to the global research network and recent scientific developments Pancreatic cancer-critical examination of the global research architecture and recent scientific developments Rheumatoid arthritis: Scientific development from a critical point of view Arthroplasty: Critical scientometric analysis of current benchmarking and evaluation procedures Critical analysis of publication procedures and evaluation regarding ankylosing spondylitis by densityequalizing mapping and scientometeric methods The role of endocarditis, myocarditis and pericarditis in qualitative and quantitative data analysis Evaluation of the global research architecture regarding diabetic retinopathy Borrelia burgdorferi: Eine szientometrische Analyse. Thesis Patient safety: The landscape of the global research output and gender distribution Die Magnetresonanz Tomographie (MRT): Eine szientometrische Analyse. Thesis Densityequalizing mapping and scientometric benchmarking of European allergy research Density-equalizing mapping and scientometric benchmarking in Industrial Health Die altersabhängige Makuladegeneration: Eine szientometrische Analyse. Thesis Study flow diagrams in Cochrane systematic review updates: An adapted PRISMA flow diagram Verbrennungen und ihre Bedeutung in der Biomedizin Szientometrische Analyse der Bedeutung des Herpes simplex Virus. Thesis Morbus Alzheimer: Eine szientometrische Analyse der wissenschaftlichen Publikationen von Immigration: eine szientometrische Analyse der biomedizinischen Forschung; Immigration: Analysis, trends and outlook on the global research activity. Thesis Immigration: Analysis, trends and outlook on the global research activity Clostridium botulinum und seine Toxine: Szientometrische Analyse zur Relevanz als Bakterium Shift-and Nightwork-A scientometric analysis Eine szientometrische Analyse arbeitsmedizinisch relevanter orthopädischer Entitäten. Thesis Scoliosis: Density-equalizing mapping and scientometric analysis. Scoliosis, 4, 15 Scientometric analysis and combined density-equalizing mapping of environmental tobacco smoke (ETS) research Cardiac insufficiency: A critical analysis of the current publication procedures under quantitative and qualitative aspects Publikationsanalysen zu den Themen 'Depression und Suizidalität', 'Cannabis'und ‚Bipolare Störung' unter Verwendung von Output Benchmarking und Density Equalizing Mapping. Thesis A bibliometric analysis of bipolar affective disorders using density-equalizing mapping and output benchmarking Depression and suicide publication analysis, using density equalizing mapping and output benchmarking Diabetische Retinopathie: Eine szientometrische Analyse. Thesis Die exogen-allergische Alveolitis: Eine szientometrische Analyse. Thesis, Goethe Universität Frankfurt Wiederbelebung: Eine szientometrische Analyse. Thesis Allergische Rhinitis: Analyse der Forschungstätigkeit (Bibliometrische Untersuchung) Luftverschmutzung, Feinstaub und Schwefeldioxid: Szientometrische Analyse. Thesis Air pollution research: Visualization of research activity using density-equalizing mapping and scientometric benchmarking procedures We thank Beatrix Kloft for her pioneering work concerning NewQIS. Without her input, this platform would not exist. We also thank all involved scientists, among them many MD students and also senior scientists, for investing thousands of hours of academic work for the purpose of NewQIS. Authors' contributions DAG, DK, DB, CS, AF and DQ conceived the review, and participated in the process of drafting the manuscript. All authors read and approved the final manuscript. The authors declare that they have no competing interests.