key: cord-0426039-2o9k6h14
authors: Enevoldsen, K. C.; Danielsen, A.; Rohde, C.; Jefsen, O. H.; Nielbo, K.; Ostergaard, S. D.
title: Monitoring of COVID-19 Pandemic-related Psychopathology using Machine Learning
date: 2021-07-16
journal: nan
DOI: 10.1101/2021.07.13.21259962
sha: 1a6e827a38be3b8f2ea0e0acdfc5bd996ccf028d
doc_id: 426039
cord_uid: 2o9k6h14

The COVID-19 pandemic has been shown to have a major negative impact on global mental health and patients with mental illness may be particularly vulnerable. We show that developments in COVID-19 pandemic-related psychopathology among patients with mental illness can be meaningfully monitored using machine learning methods. The COVID-19 pandemic-related psychopathology was found to covary with the pandemic pressure. This correlation was, however, less pronounced during the second wave compared to the first wave of the pandemic - possibly due to habituation.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.13.21259962 doi: medRxiv preprint 3 The COVID-19 pandemic is believed to have a major negative impact on global mental health due to the viral disorder itself as well as the lockdowns, social distancing, isolation, fear, and increased uncertainty associated with the pandemic (Brooks et al., 2020; Sønderskov et al., 2021; Szcześniak et al., 2021) . Individuals with preexisting mental illness are likely to be particularly vulnerable to the psychological stress associated with the COVID-19 pandemic Rohde et al., 2020) . Indeed, based on screening of clinical notes from electronic health records, we recently showed that many patients with mental illness appeared to develop COVID-19 pandemic-related psychopathology -i.e., symptoms of mental illness that seemed to be either directly or indirectly caused by the pandemic -during the first wave of the pandemic in the spring of 2020 . Here, we continue this effort in an attempt to monitor the development of COVID-19 pandemic-related psychopathology over the extended course of the pandemic.

Based on the clinical notes assessed for COVID-19 pandemic-related psychopathology by Rohde et al., (2020) , two types of supervised machine learning models were trained to classify notes as "pandemic-related psychopathology" and "not pandemic-related psychopathology". Rohde et al.

(2020) manually screened a total of 11 072 clinical notes from the electronic health record system of the Psychiatric Services of the Central Denmark Region from the period from February 1 st to March 23 rd , 2020 (the first confirmed case of COVID-19 in Denmark was on February 27 th 2020). The 11 072 clinical notes were extracted from all 412 804 clinical notes from the period in question using COVID-19 related search queries: "corona" (corona), "covid" (covid), "virus" (virus), "epidemi" (epidemic), "pandemi" (pandemic), "smitte" (contaminate/contamination), including compound words. The 11 072 notes were labeled . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.13.21259962 doi: medRxiv preprint 4 manually with regard to COVID-19 pandemic-related psychopathology by CR and OHJ with discrepancies resolved through discussion and consultation with SDØ. 1 357 of the 11 072 clinical notes described COVID-19 pandemic-related psychopathology. For more details, please see Rohde et al. (2020) .

Using the 11 072 labeled clinical notes (1357 describing COVID-19 pandemic-related psychopathology and 9715 without COVID-19 pandemic-related psychopathology), two types of machine learning models (XGBoost and Naïve Bayes) were trained using a 5-fold crossvalidated grid search on the training set. The training set consisted of 70% of the 11 072 clinical notes (randomly drawn). The models were trained on the natural text in the clinical notes, the text field indicator (e.g. 'observations of the patient', 'current mental status', 'plan', 'conclusion') patient's sex and diagnosis, and inpatient or outpatient status. Subsequently, the trained models were validated on a test set consisting of the remaining 30% of the clinical notes. The best performing model was then applied to the clinical notes (selected using the same COVID-19 related search queries as in Rohde et al. (2020) ) from the period from April 1 st , 2020

to March 23 rd , 2021. These notes were labeled dichotomously (pandemic-related psychopathology; yes/no) based on a threshold calculated from the test set to result in 95% specificity. Finally, to validate model performance in this later period, 500 clinical notes from this period were manually labeled by CR and OHJ (similar approach as that used by Rohde et. al. The correlation between the number of clinical notes describing pandemic-related psychopathology and the number of deaths due to COVID-19 in Denmark indicates that the mental health impact of the COVID-19 pandemic among patients with mental illness covaries with the pandemic pressure. This finding is clinically meaningful and compatible with the 6 s . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101/2021.07.13.21259962 doi: medRxiv preprint 7 covariation between the pandemic pressure and psychological well-being observed at the general population level in Denmark (Sønderskov et al., 2021) . The correlation, however, seems to be less pronounced for the second wave compared to the first wave of the pandemic. As there was no systematic decline in model performance over the observation period, the weakened correlation could likely be due to habituation -i.e. that patients are less sensitive to the development in the pandemic as time passes, possibly because the situation appears less insecure compared to the initial phase of the pandemic. Both the correlation in itself and its weakening over time are important observations that may aid psychiatric services in the planning of the management of the psychological consequences of the COVID-19 pandemic. Furthermore, the results are testament to the potential of applying machine learning on structured and natural text data from electronic health records in clinical psychiatry.

The psychological impact of quarantine and how to reduce it: Rapid review of the evidence

COVID-19-related selfharm and suicidality among individuals with mental disorders

Psychiatric symptoms related to the COVID-19 pandemic

Variation in psychological well-being and symptoms of anxiety and depression during the COVID-19 pandemic: Results from a three-wave panel survey

The SARS-CoV-2 and mental health: From biological mechanisms to social consequences

The authors thank Bettina Nørremark from the Business Intelligence Office in the Central Denmark Region for her assistance with extraction of data.

 Figure S1 in the Supplementary Material) as might have been expected if there was a distribution shift, i.e. a substantial change in the content of the clinical notes over time.. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 16, 2021. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 16, 2021. 

CR received the 2020 Lundbeck Foundation Talent Prize. SDØ received the 2020 Lundbeck Foundation Young Investigator Prize. The remaining authors declare no conflicts of interest.

Due to the sensitive nature of the data, they are only available for quality development projects to employees in the Central Denmark Region upon application to, and approval by, the Chief Medical Officer of Psychiatry.. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 16, 2021.https://doi.org/10.1016/j.pnpbp.2020.110046 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)The copyright holder for this preprint this version posted July 16, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021