key: cord-295891-c9rfy1w7
authors: Zhao, J.; Grabowska, M. E.; Kerchberger, V. E.; Smith, J. C.; Eken, H. N.; Feng, Q.; Peterson, J. F.; Rosenbloom, S. T.; Johnson, K. B.; Wei, W.-Q.
title: ConceptWAS: a high-throughput method for early identification of COVID-19 presenting symptoms
date: 2020-11-10
journal: nan
DOI: 10.1101/2020.11.06.20227165
sha: 
doc_id: 295891
cord_uid: c9rfy1w7

Objective: Identifying symptoms highly specific to COVID-19 would improve the clinical and public health response to infectious outbreaks. Here, we describe a high-throughput approach - Concept-Wide Association Study (ConceptWAS) that systematically scans a disease's clinical manifestations from clinical notes. We used this method to identify symptoms specific to COVID-19 early in the course of the pandemic. Methods: Using the Vanderbilt University Medical Center (VUMC) EHR, we parsed clinical notes through a natural language processing pipeline to extract clinical concepts. We examined the difference in concepts derived from the notes of COVID-19-positive and COVID-19-negative patients on the PCR testing date. We performed ConceptWAS using the cumulative data every two weeks for early identifying specific COVID-19 symptoms. Results: We processed 87,753 notes 19,692 patients (1,483 COVID-19-positive) subjected to COVID-19 PCR testing between March 8, 2020, and May 27, 2020. We found 68 clinical concepts significantly associated with COVID-19. We identified symptoms associated with increasing risk of COVID-19, including "absent sense of smell" (odds ratio [OR] = 4.97, 95% confidence interval [CI] = 3.21-7.50), "fever" (OR = 1.43, 95% CI = 1.28-1.59), "with cough fever" (OR = 2.29, 95% CI = 1.75-2.96), and "ageusia" (OR = 5.18, 95% CI = 3.02-8.58). Using ConceptWAS, we were able to detect loss sense of smell or taste three weeks prior to their inclusion as symptoms of the disease by the Centers for Disease Control and Prevention (CDC). Conclusion: ConceptWAS is a high-throughput approach for exploring specific symptoms of a disease like COVID-19, with a promise for enabling EHR-powered early disease manifestations identification.

As of October 14, 2020, over 7.7 million people in the United States (U.S.) and 37 million worldwide were infected with coronavirus SARS-CoV-2, the agent responsible for COVID-19 [1] .

The virus's high transmissibility, lack of native immunity, and the dearth of effective treatments make managing COVID-19 uniquely challenging. Hence, early recognition of specific COVID-19 symptoms plays an essential role in the clinical and public health response, enabling rapid symptom screening, diagnostic testing, and contact tracing.

Early in the pandemic, physicians observed fever, cough, and shortness of breath as presenting symptoms of COVID-19; however, these symptoms are common to many viral and bacterial illnesses [2] . Subsequently, as new symptoms were reported, health departments and ministries updated the list of COVID-19 symptoms; for example, the U.S. CDC and the Department of Health and Social Care in the UK added loss of smell or taste, a highly indicative symptom [3] to the list in late April and mid-May, respectively [4, 5] . Therefore, methods of identifying the specific COVID-19 symptoms early during a pandemic are required, which is crucial to inform the public on when to present for testing, and potentially reduce the ultimate size of the outbreak, lowering overall morbidity and mortality.

Recent efforts to track COVID-19 symptoms have used methods such as scanning publications or twitter [6, 7] , deploying questionnaires [8] , or releasing apps to self-report symptoms [9] . However, results from publications and questionnaires may be delayed; data from social media or self-reported apps do not always include proper controls and lack physiological assessments to determine the COVID-19 status. Electronic Health Records (EHR) data has also been used to characterize COVID-19, due to the availability of routinely collected medical data. However, existing studies were limited by structured data (e.g., coded diagnoses, procedures, or lab tests) [10, 11] and lacked a portable and high-throughput approach [12] .

Here, we present a high-throughput approach (ConceptWAS) for early identification of clinical manifestations of COVID-19 using natural language processing (NLP) on EHR clinical notes. By examining the EHR-derived concepts related to patients' signs or symptoms, ConceptWAS assesses whether any of the concepts are associated with the presence or absence of a disease (e.g., . Using this approach, we identified the symptoms that were specific to COVID-19. In particular, we performed ConceptWAS using the cumulative data every two weeks to demonstrate the timeline of emerging symptoms. We also conducted a chart review to validate the significant associations.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint

The study was performed at Vanderbilt University Medical Center (VUMC), one of the largest primary care and referral health systems serving over one million patients annually from middle Tennessee and the Southeast United States. We used data from patients represented in the VUMC EHR aged ≥ 18 years. The study was approved by the VUMC Institutional Review Board (IRB #200512).

We firstly identified patients who received at least one SARS-CoV-2 polymerase chain reaction (PCR) test between March 8 (when the first COVID-19 emerged in VUMC) and May 27, 2020 ( Figure A.1) . The COVID-19 status was determined using the PCR test result. The case group (COVID-19-positive) was defined as patients who had >=1 PCR positive result, and the control group (COVID-19-negative) consisted of patients with only negative PCR tests. We excluded patients who had no clinical notes on the day the PCR test was ordered.

We extracted clinical notes from 24 hours prior to PCR testing date (day 0 ) for COVID-19positive and negative patients (>86% of patients had at least one note within the time window, see Figure B .1). If a patient first tested negative and then subsequently tested positive or if a patient were tested positive more than once, we used the date of the first positive PCR test as day 0 . We also segmented the study period into a two-week interval and used the cumulative data every two weeks to perform a temporal analysis.

We used KnowledgeMap Concept Indexer (KMCI [13] ) to preprocess the clinical notes and extract concepts ( Figure C.1 ). KMCI is a local NLP pipeline developed at VUMC for medical notes processing and entity recognition, which has been used for several clinical and genomic studies [13] [14] [15] . The preprocessing includes sentence boundary detection, tokenization, part-ofspeech tagging, section header identification. The concepts were represented as Unified Medical Language System concept unique identifiers (UMLS CUIs). Since we focused on capturing clinical manifestations of COVID-19, we restricted the concepts to SNOMED Clinical Terms and specific semantic types, e.g., finding, sign or symptom, disease or syndrome, individual behaviors, or mental process (see full list in Table C .1).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint 5 A main challenge of clinical NLP is to accurately detect the clinical entities' assertion modifier such as negated, uncertain, and hypothetical (e.g. describe a future hypothetical or instruction for patients). We took the following steps to post-process the KMCI output to remove CUIs that were uncertain, and hypothetical. We first excluded any concepts that arose from family history sections. Next, we removed any sentences with future sense or subjunctive mood (e.g. "should", "could", or "if") that describe a hypothetical or instruction for patients. We excluded inquiry sentences that served as the template questions without a simple confirmed answer (e.g. "Yes", "No", or "None") as well. For recognition of negated concepts (e.g. "patient denies having any fever"), KMCI implements NegEx, a widely-used algorithm to detect negations. However, NegEx may miss post-negation triggers such as "Cough: No". We added regular expression rules based on our local note template to enhance negation detection.

The extended processing modules was implemented using Python 3.6. After processing, the extracted CUIs served as the input for following ConceptWAS analysis.

Similar like genome-wide association study (GWAS) and phenome-wide association study (PheWAS) that scan genomic and phenomic data for associations with a given disease or a genetic variant [16, 17] , conceptWAS examines the clinical concepts retrieved from clinical notes to determine if any concept is associated with a disease. In this study, we applied a conceptWAS to identify associations between symptoms-related concepts and the presence of COVID-19.

We applied Firth's logistic regression to examine the association for each concept, adjusted by age, gender, and race. We chose Firth's logistic regression because it has become a standard approach for analyzing binary outcomes with small samples [18] . Negated and nonnegated CUIs are treated separately. A CUI with a negation flag (yes/no) is only counted once per patient. Firth's logistic regression was implemented using R version 3.4.3 and the logistf package. As we tested multiple hypotheses, we used a Bonferroni correction for the significance level. We report the odds ratio (OR), p-values, and the prevalence in case and control groups for each CUI. We used a volcano plot to show p-values and the odds ratio for all CUIs. We also used a forest plot to show the significant concepts that were relevant to signs and symptoms.

We performed a manual chart review to evaluate the clinical plausibility of identified signals. We reviewed a CUI if 1) its p-value met Bonferroni-corrected significance, and 2) it was clinically meaningful (e.g., we excluded CUIs such as "finding [CUI C00000243095]" in a . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint 6 sentence like "Findings are nonspecific."). We randomly selected notes from which the CUI was identified. Two authors (M.E.G. and H.N.E.) with clinical background ascertained whether the identified CUI was a true signal or false positive.

We identified 19,692 patients with COVID-19 PCR test results during the study period ( Figure A.1 

We is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint 7 95% CI = 0.28-0.52). Specifically, symptoms concepts associated with COVID-19-positive patients included "absent sense of smell" (OR = 4.97, 95% CI = 3.21-7.50), "fever" (OR = 1.43, 95% CI = 1.28-1.59), "with cough fever" (OR = 2.29, 95% CI = 1.75-2.96), and "ageusia" (OR = 5.18, 95% CI = 3.02-8.58) (Figure 2 ).

Concepts related to smoking status such as "current some day smoker", "former smoker", and "smoking monitoring status" were more frequently reported in the COVID-negative group than in the COVID-positive group (OR < 1, P < 2.55E-06), suggesting more smokers in control group. To ascertain whether this signal was true or false positives due to wrongly assertion detection by NLP pipeline, we performed a chart review of 80 patients' notes that had smoking-related CUIs. We found that 79 of 80 patients confirmed an affirmative smoking status (see below chart review). COVID-19-negative (control) patients. Selected associations include the significant signals related to semantic types of symptoms that met Bonferroni-corrected significance (p-value < 2.55E-06). The odds ratio has been adjusted for age, gender, and race. The concepts are ordered by p-value.

We performed ConceptWAS using the cumulative data every two weeks within the study period ( Figure 3, Figure D CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint (between March 8 and May 3, 2020). The depression and anxiety appeared significantly starting from week 4 (by April 5, 2020).

Temporal ConceptWAS using bi-weekly cumulative data. For significant signals (related to signs, symptoms) using all data (labeled in Figure 2 ), the plot indicates their -log 10 (p-value) for association (y-axis) against using the cumulative data started between March 8, 2020 to n weeks (x-axis). The dashed line indicates a significant association using a Bonferroni correction.

To validate the signals, we reviewed patient's charts for significant concepts with high clinical relevance. We randomly selected 20 notes for each concept to review whether the notes mentioned the symptoms in the expected attribute (e.g. affirmative or negated) ( Table 2 ). The significant concepts from the ConceptWAS that compared COVID-19 positive and negative patients, such as "absent sense of smell", "ageusia", "mental depression", and concepts related to smoking status (e.g. "current some day smoker", "former smoker", and "smoking monitoring status") were consistent with the expected attribute based on chart review.

Although "smoking monitoring status" was generated by an inquiry term used in a template of a chart, after we post-processed the KMCI output to remove irrelevant concepts and refine negation, the smoking monitoring status followed by a negated answer was recognized as . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint a negated attribute. We reviewed 20 notes that mentioned the "smoking monitoring status (affirmative/positive attribute)" and 19 were either current or former smokers.

We also found false positive concepts, mostly due to NLP entity recognition errors. For example, "additional information" was recognized as "adequate knowledge". The concept "fever" with positive attribute has three false positives, mainly due to a few specific chart templates used for denoting the negation, which were not captured by NLP pipeline. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. The false positives were wrongly mapped from a sentence like "He /She has been exposed to covid, family member or friends have tested positive."

Laurin-Sandrow syndrome 20 20 100.00%

Cough nonproductive 20 20 100.00%

In total 260 217 83.46%

Our work describes a high-throughput and reproducible approach (ConceptWAS) that use EHR notes to early identify pandemic disease symptoms and investigate clinical manifestations for further hypothesis-driven study. We applied ConceptWAS to a cohort of patients who underwent COVID-19 testing. We replicated several well-known symptoms of COVID-19, such as fever, loss of smell/taste, and cough with fever [19] [20] [21] . Using ConceptWAS, we were able to detect the signal of loss of smell and taste as early as April 5, 2020, nearly three weeks earlier than the date that they were listed as COVID-19 symptoms by the CDC [3] .

Our results demonstrate the feasibility of using ConceptWAS for an early detection of symptoms of an unknown disease.

We also observed several signals enriched in the COVID-19-negative group. For example, depression and anxiety have a higher prevalence among patients who tested negative.

These signals first became significant starting from April 5, 2020, corresponding to the date when the Governor of Tennessee issued a "safer at home" Executive Order and a "stay at home" order. It reflects the mental health issues that the shutdown and quarantine policies may bring to the people [22, 23] . We also find a higher percentage of smoking status concepts in the COVID-19-negative group. Earlier epidemiological studies found that fewer smokers are among COVID-19 patients or hospitalized COVID-19 patients [21, 24] , which are consistent with our findings of . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint the negative correlation between smoking and COVID-19. One explanation could be the impact of nicotine on ACE-2, as nicotine has been suggested to play a protective role against COVID-19 [25] . It is also possible that smokers are taking greater social precautions because of perceived higher risk for respiratory complications from COVID-19, thus reducing their risk of contracting the virus. Although these findings suggest that smoking may be a protective factor, lack of evidence and known adverse events associated with smoking dissuade continued smoking as a protective measure against ConceptWAS is open-source, portable, and reproducible. Researchers/Users can choose other NLP pipelines (e.g. MetaMap, CLAMP, cTAKES) [26] for concept extraction and use the derived concepts as the input to ConceptWAS.

Through a proof-of-concept study by applying NLP techniques to identify COVID-19 symptoms, we summarize the lessons learned that may help others apply this method.

(1) A high-throughput, lightweight, and reproducible method is important for pandemic disease.

ConceptWAS enables a rapid scan of symptoms using clinical notes. These symptoms provided an initial hypothesis for further investigation and could alert clinicians to pay attention to patients with presenting specific symptoms. Researchers can run the ConceptWAS regularly (e.g., weekly or bi-weekly cumulative data) to track the symptoms changes for a pandemic disease.

(2) Running ConceptWAS needs to be cautious about the distribution of different clinical note types. Different clinical note types are unlike from each other due to their specific clinical usage. They may have variable templates and inconsistent length. Therefore, we recommend that researchers check the distribution of document types between cases and controls to avoid sampling bias.

(3) Although NLP has been used in various medical fields to improve information processing and practice [26] [27] [28] [29] , recognition of negative and uncertain concepts remains a challenge.

We enhanced the detection of uncertain arguments and negated concepts by developing rule-based methods as wrappers for entity-identification generated results. Still, our manual chart review suggest that the outcome is not perfect. For example, some notes mentioned negative concepts like "the following ROS were reviewed and are negative, unless otherwise stated as +positive: Constitutional: Fever; malaise." Such scenarios are difficult for NLP tools to identify. A combination of machine learning and rule-based approaches may improve the detection.

(4) We also learned and recognized that our study has several limitations. First, the study was performed at a single institution with a limited number of COVID-19 patients. As the . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint 13 pandemic crisis evolves and more patients are tested for SARS-CoV-2 in our healthcare system, our ability to detect COVID-19 and clinical concepts' associations will continue to improve. Second, this study used data from a limited time (before May 27, 2020). In the future, we will extract notes prior to the test date to study the progression of the symptoms and analyze them along different periods. Lastly, as the performance of an NLP system may vary across institutions and databases [26, 30] , further studies are necessary to assess the generalizability of our findings.

In this study, we describe a high-throughput approach (ConceptWAS) that systematically scans a disease's clinical manifestations from clinical notes. By applying ConceptWAS on EHR clinical notes from patients subjecting to a COVID-19 test, we detected loss of smell/taste three weeks prior to their inclusion as symptoms of the disease by CDC. The study demonstrates the potential of the EHR-based methods to enable early recognition of specific COVID-19 symptoms, and improve our clinical and public health response to the pandemic.

Up-to-date developments of ConceptWAS are available in GitHub . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint

WHO Coronavirus Disease (COVID-19) Dashboard

Clinical Characteristics of Coronavirus Disease 2019 in China

Seroprevalence of SARS-CoV-2 antibodies in people with an acute loss in their sense of smell and/or taste in a community-based population in London, UK: An observational cohort study

CDC confirms six coronavirus symptoms showing up in patients over and over

Statement from the UK Chief Medical Officers on an update to coronavirus symptoms

CovidNLP: A Web Application for Distilling Systemic Implications of COVID-19 Pandemic with Natural Language Processing

Machine Learning to Detect Self-Reporting of Symptoms, Testing Access, and Recovery Associated With COVID-19 on Twitter: Retrospective Big Data Infoveillance Study

Symptom Profiles of a Convenience Sample of Patients with COVID-19 -United States

Real-time tracking of self-reported symptoms to predict potential COVID-19

Presenting Characteristics, Comorbidities, and Outcomes Among

Patients Hospitalized With COVID-19 in the New York City Area

APHP/Universities/INSERM COVID-19 research collaboration

The Consortium for Clinical . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted

International Electronic Health Record-Derived COVID-19 Clinical Course Profiles: The 4CE Consortium, Infectious Diseases (except HIV/AIDS)

Augmented curation of clinical notes from a massive EHR system reveals symptoms of impending COVID-19 diagnosis

The KnowledgeMap Project: Development of a Concept-Based Medical School Curriculum Database

Extracting timing and status descriptors for colonoscopy testing from electronic medical records

Potential etiologic and functional implications of genome-wide association loci for human diseases and traits

PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene-disease associations

Firth's logistic regression with rare events: accurate effect estimates and predictions? -Puhr -2017 -Statistics in Medicine

International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted

COVID-19 Patients' Clinical Characteristics, Discharge Rate, and Fatality Rate of Meta-Analysis -PubMed

Anosmia and Ageusia: Common Findings in COVID-19 Patients

Smell dysfunction: a biomarker for COVID-19

Mental Health and the Covid-19 Pandemic

Bill Lee issues stay-at-home order through

Prevalence of Underlying Diseases in Hospitalized Patients with COVID-19: a Systematic Review and Meta-Analysis

Nicotine and SARS-CoV-2: COVID-19 may be a disease of the nicotinic cholinergic system, Toxicol Rep

CLAMP -a toolkit for efficiently building customized clinical natural language processing pipelines

Automatic identification of critical followup recommendation sentences in radiology reports

Automated extraction of clinical traits of multiple sclerosis in electronic medical records

Natural Language Processing in Oncology: A Review

Negation's Not Solved: Generalizability Versus Optimizability in Clinical Natural Language

The authors have no competing interests to declare.Processing, (n.d.).. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprintThe copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint 18 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0112774 (accessed August 18, 2020).. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprintThe copyright holder for this this version posted November 10, 2020. ; https://doi.org/10.1101/2020.11.06.20227165 doi: medRxiv preprint