key: cord-1006469-t881fp3d authors: Pomare, Chiara; Mahmoud, Zeyad; Vedovi, Alex; Ellis, Louise A.; Knaggs, Gilbert; Smith, Carolynn L.; Zurynski, Yvonne; Braithwaite, Jeffrey title: Learning health systems: A review of key topic areas and bibliometric trends date: 2021-03-18 journal: Learn Health Syst DOI: 10.1002/lrh2.10265 sha: 09b6f017156d7ea446c53a43c40f2dccc578065e doc_id: 1006469 cord_uid: t881fp3d INTRODUCTION: The emergent field of learning health systems (LHSs) has been rapidly evolving as the concept continues to be embraced by researchers, managers, and clinicians. This paper reports on a scoping review and bibliometric analysis of the LHS literature to identify key topic areas and examine the influence and spread of recent research. METHODS: We conducted a scoping review of LHS literature published between January 2016 and May 2020. The authors extracted publication data (eg, journal, country, authors, citation count, keywords) and reviewed full‐texts to identify: type of study (empirical, non‐empirical, or review), degree of focus (general or specific), and the reference used when defining LHSs. RESULTS: A total of 272 publications were included in this review. Almost two thirds (65.1%) of the included articles were non‐empirical and over two‐thirds (68.4%) were from authors in the United States. More than half of the publications focused on specific areas, for example: oncology, cardiovascular care, and genomic medicine. Other key topic areas included: ethics, research, quality improvement, and electronic health records. We identified that definitions of the LHS concept are converging; however, many papers focused on data platforms and analytical processes rather than organisational and behavioural factors to support change and learning activities. CONCLUSIONS: The literature on LHSs remains largely theoretical with definitions of LHSs focusing on technical processes to reuse data collected during the clinical process and embedding analysed data back into the system. A shift in the literature to empirical LHS studies with consideration of organisational and human factors is warranted. We carried out a scoping review of LHS literature published between January 2016 and May 2020. A scoping review synthesises and maps the research on a particular topic to identify key concepts and gaps in the literature. 14 Searches were carried out in two online databases (PubMed and Scopus) using the term "learning health* system*". Publications were downloaded into EndNote and duplicates were removed. The review team (CP, ZM, AV, LAE, GK, CLS) screened retrieved publications in full to determine their inclusion. Publications were included if they were (a) in the English-language, (b) peer-reviewed publications (journal articles, review articles, journal commentaries, editorials, books, or book chapters), and (c) had a key focus on LHS. Publications containing no substantive discussion of the LHS concept (eg, only superficially used the term in the abstract, conclusion, or among the keywords) were excluded. Five percent of retrieved publications were independently screened by the entire review team to ensure consistent inclusion. Disagreements about the eligibility of publications for inclusion were resolved through review team consultation, with YZ and JB resolving any outstanding disagreements. Publications were reviewed in full to identify publication data (the journal in which the paper was published, keywords, author names, and the country of residence of the corresponding author), the type of study (empirical, non-empirical, or review), degree of focus (general or specific), and, if a definition was included. This information was extracted and stored in Microsoft Excel 365. The number of citations for each publication was also recorded, as reported by Web of Science in September 2020, and was used as a measure of impact or influence, rather than as an indicator of quality. Publication data were collected to determine the most active journals, authors, and countries publishing in the LHS field. For degree of focus, a publication was coded as "general" when LHS concepts were discussed without discussing a specific program, health condition, or topic area. Publications coded as having a specific degree of focus were later aggregated to create counts per key topic area (ie, to delineate how many included publications focused on ethics). Key topic areas were also examined through an analysis of keywords. Keywords of publications were identified in EndNote X9, then cleaned and checked for consistency. Derivatives (eg, health system, healthcare systems, health systems) were amalgamated. The keyword data were analysed for frequency counts and co-occurrence using VOSviewer v.16.14 (https://www.vosviewer.com/). The citation average for the field was calculated by citation count per paper divided by the number of years elapsed since publication. Where a definition of LHS was used, we extracted the definition and referenced citation. Definitions were analysed using text analysis to determine word use frequency (ie, similar terms were grouped together, such as improvement, improve, improving). LHS definition references were graphically presented using Gephi, version 0.9.2. The nodes in the network were the publications in the review, as well as any output that was cited as a definition. Ties were LHS definition citations (eg, publication X cited publication Y to define an LHS). The most influential publications in the network were assessed using in-degree calculation 15 (ie, the number of times a reference was cited by different publications when defining an LHS). The search yielded 430 publications. After the removal of duplicates, there were 420 publications, 148 of which did not meet the inclusion criteria. This resulted in 272 publications on LHSs included in our analysis. Of the 272 publications, most were non-empirical (n = 177, 65.1%), one quarter were empirical (n = 68, 25.0%), less than a tenth were reviews of the literature (n = 25, 9.2%), and two were books (0.7%). Most publications (n = 193, 70.4%) provided a definition of an LHS. Ninety-five different citations were used when providing a definition of an LHS. More than half of the definition references were used more than once (n = 56, 58.9%). Seven definitions did not have citations and the wording could not be identified as belonging to a previous paper; these were classified as definitions developed by the author(s). Common terms used when defining LHSs are listed in Table 3 . See Table 4 for the most-cited sources of LHS definitions. Among the 272 included publications, 1067 unique keywords were identified. Of these, 69 were used at least five times. Co-occurrence of keywords (ie, keywords used together on a publication) is visually depicted in the network of co-occurring keywords (Figure 2 ). In this network, each circle represents a keyword and each line indicates co- Oncology's CancerLinQ initiative. 22 CancerLinQ is a large database that collects information from electronic health records of cancer patients to foster an oncology learning community across the United States; it has been described as a rapid LHS. 23 This example demonstrates the inherent need for integrated data platforms to support the adoption of LHSs and the importance of rapid implementation and implementation science in this sphere. 24 As to strengths and limitations, this study highlighted key topic reviews. There is a need to capture research written in languages other than English to obtain a more widespread view of the LHS concept and its potential. This is particularly important given the divide between high-, middle-, and low-income countries when it comes to big data and personalised medicine, 25 which are key components of LHSs. Despite ongoing interest in the concept of LHS as demonstrated by the growing body of literature, this study shows that considerations of LHS concepts are largely focused on technical processes rather than the organisational and human factors necessary to facilitate an LHS. The LHS literature warrants a shift in focus to move the field from the conceptual LHS to take-up and adoption, and from technical processes to emphasising the complexity of human factors in LHSs. This study was funded by NHMRC grant nos. 9100002 and APP1176620. The three numbers you need to know about healthcare: the 60-30-10 challenge Changing how we think about healthcare improvement Best Care at Lower Cost: the Path to Continuously Learning Health Care in America Institute of Medicine Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care Use of electronic medical records for health outcomes research: a literature review Electronic health record adoption in US hospitals: progress continues, but challenges persist A rapid-learning health system The learning healthcare system: where are we now? A systematic review An analysis of the learning health system in its first decade in practice: scoping review Toward a science of learning systems: a research agenda for the high-functioning learning health system Telerehabilitation in the age of COVID-19: an opportunity for learning health system research Pandemic as a catalyst for rapid implementation: how our hospital became a learning health system overnight Scoping studies: towards a methodological framework Achieving a nationwide learning health system From learning healthcare systems to learning health systems The Heimdall framework for supporting characterisation of learning health systems Big data, ethics, and regulations: implications for consent in the learning health system Research versus practice: the dilemmas of research ethics in the era of learning health-care systems An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics Using oncology real-world evidence for quality improvement and discovery: the case for ASCO's CancerLinQ Building a rapid learning health care system for oncology: why CancerLinQ collects identifiable health information to achieve its vision The rise of rapid implementation: a worked example of solving an existing problem with a new method by combining concept analysis with a systematic integrative review From big data analysis to personalized medicine for all: challenges and opportunities Learning health systems: A review of key topic areas and bibliometric trends The authors declare that they have no conflicts of interest.