key: cord-1033441-auwkdf10
authors: Cannon, Julie A.; Strathman, Annabella R.; Warner, Seth; Flignor, Jacob; Beck, Nataly S.
title: Assessing the impact of COVID-19 on psychiatric clinical trials
date: 2022-01-15
journal: J Psychiatr Res
DOI: 10.1016/j.jpsychires.2022.01.017
sha: d4e7427241d0648f28bd4d76c8884747ee16fc48
doc_id: 1033441
cord_uid: auwkdf10

OBJECTIVE: COVID-19 and associated measures to control the spread of the COVID-19 has significantly impacted clinical research. This study aimed to determine the impact COVID-19 has had on psychiatric clinical trials and to assess whether certain trial areas or trial types were differentially affected. METHODS: We used information from ClinicalTrials.gov, the largest online database of clinical trial information, to examine changes in psychiatric clinical trials from January 2010–December 2020. RESULTS: Clinical trial initiation decreased in 2020, with a year-on-year percent change in trial initiation of −5.4% versus an expected percent change based on forecasting observed trends from 2010-2019 of 8.6%. When broken down by disease area, the distribution of trials observed in 2020 was significantly different from the predicted distribution (p < 0.00001). The greatest decrease in trial initiation was seen in Schizophrenia-specific trials, with an observed percent change of −29.2% versus an expected percent change of 3.2%. Conversely, anxiety trials saw a significant increase in trial initiation during 2020, with an observed percent change of 24.6% versus an expected percent change of 16.0%. When assessing interventional versus observational studies, data showed a significant increase in initiation of observational psychiatric clinical trials (p < 0.05), and a significant decrease in initiation of interventional psychiatric clinical trials (p < 0.01). When data was analyzed on a month-by-month time scale, 7/12 months in 2020 showed significant decreases when compared to initiation during matching months over prior years, and a single month, June, showed a significant increase. CONCLUSION: COVID-19 has had significant impacts on the initiation of psychiatric clinical trials over 2020, and this decrease in trial initiation may have long-term impacts on the development and assessment of psychiatric treatments and therapeutics.

COVID-19 and associated measures to control the spread of the COVID-19 has significantly impacted clinical research. This study aimed to determine the impact COVID-19 has had on psychiatric clinical trials and to assess whether certain trial areas or trial types were differentially affected.

We used information from ClinicalTrials.gov, the largest online database of clinical trial information, to examine changes in psychiatric clinical trials from January 2010 -December 2020.

Clinical trial initiation decreased in 2020, with a year-on-year percent change in trial initiation of -5.4%

versus an expected percent change based on forecasting observed trends from 2010-2019 of 8.6%. When broken down by disease area, the distribution of trials observed in 2020 was significantly different from the predicted distribution (p < 0.00001). The greatest decrease in trial initiation was seen in Schizophrenia-specific trials, with an observed percent change of -29.2% versus an expected percent change of 3.2%. Conversely, anxiety trials saw a significant increase in trial initiation during 2020, with an observed percent change of 24.6% versus an expected percent change of 16.0%. When assessing interventional versus observational studies, data showed a significant increase in initiation of observational psychiatric clinical trials (p < 0.05), and a significant decrease in initiation of interventional psychiatric clinical trials (p < 0.01). When data was analyzed on a month-by-month time scale, 7/12 months in 2020 showed significant decreases when compared to initiation during matching months over prior years, and a single month, June, showed a significant increase.

The effects of the COVID-19 pandemic have been far ranging, leading to widespread changes in societal norms, significant mortality rates, and economic strife. The pandemic's effects have been acutely felt by the healthcare system, leading to wide ranging resource scarcity and the reallocation of funding, attention, and personnel to COVID-19, as well as in the rapid transition to telemedicine (Sharma et al., 2020) .

Less visible, but perhaps equally important, is the effect the pandemic has had on the future of clinical research and in particular the impact of the pandemic on the conduct of clinical trials. Analyses have been done across many medical specialties investigating the impact of COVID-19 on clinical trial initiation, continuation, and cancelation (Medidata, 2020; Sathian et al., 2020) . This literature highlighted decreases in enrollment numbers of up to 74% in the first three months of the pandemic compared to the same period in previous years (Medidata, 2020; Sathian et al., 2020) . Furthermore, concerns about the integrity and the lack of standardization of virtual trials have been raised, with many urging for greater standardization in this area (McDermott and Newman, 2020; Nicol et al., 2020) .

Psychiatric clinical trials have been generally overlooked in these analyses despite a preponderance of data showing that the pandemic and efforts to curtail its spread have significantly affected both the mental wellbeing of the public and the provision of psychiatric care (Shah et al., 2020) .

With the increase in mental health diagnoses since the beginning of the pandemic, psychiatric research and the development and validation of psychiatric therapies is of the utmost importance (Shah et al., 2020) . However, both have been greatly impacted by the pandemic. Anecdotal and system specific commentary has highlighted disruptions in clinical trial execution, particularly the impact on psychiatric patients (Tuttle, 2020; Sathian et al., 2020; Brown et al., 2020 , Sharma et al., 2020 . However, the specific systemic impact on psychiatry-related clinical trials remains poorly understood (Türközer and Öngür, 2020) .

Through analysis of clinical trials registered on clinicaltrials.gov, we examined changes in the quantity and characteristics of psychiatric clinical trials initiated in 2020 to assess the impact of COVID-19 on the field in this time period.

We used data from ClinicalTrials.gov, a resource provided by the U.S. National Library of Medicine and the largest online database of clinical trial registrations, to examine changes in psychiatric clinical trial initiation and characteristics over 2010-2020.

To facilitate aggregate analysis, a static clone was downloaded using the Aggregate Analysis of ClinicalTrials.gov (AACT, 2021), courtesy of the U.S. National Library of Medicine (AACT, 2021). This J o u r n a l P r e -p r o o f was used to populate a PostgreSQL relational database that contains all publicly available protocol and results information for registered trials posted prior to 1/1/2021. As all data used in this study is publicly available and contains no identifying information, this work was therefore deemed exempt from Stanford Institutional Review Board review.

Psychiatric clinical trials were filtered using the 2020 Medical Subject Headings (MeSH) terminology from the U.S. National Library of Medicine. Any study that included a diagnosis listed under the MeSH heading "Mental Disorders" in either the study title, or study conditions treated, was included in our analysis (MeSH, 2021) . Furthermore, psychiatric disease areas were analyzed by prescribed MeSH headings, and areas with more than 100 trials initiated in 2020 were further analyzed.

To evaluate changes in study characteristics of trials initiated during the COVID-19 pandemic, trials were filtered according to the above-described criteria, and a chi-square test was used to compute significance between the expected number of trials initiated in 2020 and the observed. Expected numbers of initiated trials were calculated by averaging the percent change between years in the preceding decade (2010) (2011) (2012) (2013) (2014) (2015) (2016) (2017) (2018) (2019) and applying this to the observed value in 2019. A chi-square analysis was performed to investigate whether 2020 varied significantly in the computed yearly rate of change from the expected results, as shown in Figure 1B . To evaluate study design, studies were further filtered through study selfidentification to categorize each as "Interventional" or "Observational", and growth rates for each category were computed, as shown in Figure 1C . From there those categories were further broken down by MeSH headings to additionally analyze by disease area. This analysis included computation of growth rate of both observational and interventional studies by disease type, and overall breakdown of the makeup of observational versus interventional studies per each condition, as shown in Figures 1D and 1E .

Expected values for further analyses were calculated based on the mean values of the 9 years prior to 2020 (2010-2019), and are presented in Table 1 (e.g., expected 2020 phase 1 enrollment was calculated as the average phase 1 enrollment over the preceding decade).

Studies that were canceled and listed "COVID-19" in their rationale for cancellation were further isolated from clinicaltrials.gov. Characteristics and study design details were then recorded from those isolated studies. Finally, trial initiation was broken down by month to analyze changes from the previous year in monthly growth rate through a chi-square test. Figure 1A , fewer clinical trials were initiated in the field of psychiatry in 2020 than in 2019 overall, corresponding to a 5.4% decrease in trials initiated versus a projected increase of 8.6%.

Of note, this was the only year in our analysis that saw a decrease in the number of studies compared to the previous year, highlighting the dramatic disruption in psychiatric research in 2020. When broken J o u r n a l P r e -p r o o f down by disease type, six disease areas had more than 100 studies initiated in 2020, as shown in Figure   1B , and therefore were analyzed further. A chi-square analysis showed a significant deviation from the distribution among disease areas in previous years (p < 0.00001). The most prominent decreases were observed in studies pertaining to neurocognitive disorders and schizophrenia spectrum disorders. A large, but less pronounced decline in study initiation was noted for neurodevelopmental disorders. Of note, a slight increase was also found in the initiation of anxiety disorders in 2020 relative to the expected value (obs. = 24.2%, expected = 16.0%), the only condition with an increase relative to the expected growth rate. Analysis by study design revealed an increase in the growth rate of observational studies relative to the expected value (obs. = 22.2%, expected = 8.4%) and a significant decrease in the growth rate of interventional studies were seen (obs. = 1.2%, expected = 7.9%) (p < 0.01) ( Figure 1C ).

To better assess how trial initiation differed during 2020, a further analysis of study initiation was performed comparing the observed number of trials initiated to the expected number on a month-bymonth basis ( Figure 1D ). The chi-square analysis showed significant deviation from expected values (p < 0.0001). After adjusting the -value for multiple comparisons using the Bonferroni method ( = 0.0021), the results remained significant. Based on the Pearson chi-square values for each individual comparison, April, March, July, and December contributed most significantly to the overall decrease in clinical trial initiation.

A dual analysis was performed to analyze studies by both disease type and study design, interventional or observation. For each disease type, the percentages of observational and interventional studies were computed for all years between 2010 and 2020. Significant changes were noted for anxiety disorders, mood disorders, and trauma related disorders ( Figure 1E ). This was further investigated through analysis of growth rates for each study design by disease type, in which significant increases were noted for observational studies focusing on anxiety disorders, mood disorders, and trauma related disorders (p < 0.05, p < 0.01, p < 0.05, respectively). Significant decreases in observational trial initiation were noted for schizophrenia spectrum disorders and neurocognitive disorders (p < 0.05). Growth rates for interventional studies broken down by disease type were more universal, where significant declines in trial initiations were noted for all disorders apart from anxiety disorders ( Figure 1F ).

To gain insight into the nature of the studies canceled due to COVID-19, studies that had been canceled or withdrawn and that listed COVID-19 as their reason for cancellation, totaling 35 studies, were individually analyzed. Of those 35 studies, all were found to be interventional with years of initiation ranging from 2014-2020.

Further analysis investigated whether there were significant differences in characteristics of study design between studies initiated in 2020 versus those initiated in 2010-2019. A chi-square analysis was conducted on various characteristics, comparing expected and observed numbers for 2020, and corrected J o u r n a l P r e -p r o o f using the Bonferroni method ( = 0.0125). The expected count was calculated by averaging the previous 10 years of data for each characteristic to estimate the proportion of each category, then multiplying by the total number of studies in 2020. Characteristics analyzed included: average enrollment numbers by phase, whether the study accepted healthy volunteers, whether the study enrolled participants under the age of 18. No significant differences were seen in any of the analyzed characteristics when comparing studies initiated in 2020 versus those initiated in 2010-2019 (Table 1) .

These results highlight potential long-term impacts of COVID-19 on psychiatric clinical development. We hypothesize that the differential nature of these impacts' changes may be partially explained by a shift to virtual clinical trials, as suggested by previous literature (Nicol et al, 2020) . This may explain the sharp increase in observational study designs and converse decrease in interventional trials as demonstrated. This was further supported by our analysis of the 35 canceled studies listing COVID-19 as the reason for their cancellation, of which all 35 were interventional in nature. Some literature suggests that increased usage of virtually-conducted trials may persist post-COVID-which would represent a major shift in psychiatric clinical trial design. If this were to prove true, there would likely be an increased need for standardization in the virtual conduct of psychiatric trials (Türközer and Öngür, 2020) .

According to our findings, mood disorders, neurocognitive disorders, and most strikingly schizophrenia spectrum disorders appeared most negatively impacted by changes in the clinical trial environment during 2020. This may reflect the relative difficulty in conducting trials in these subpopulations in a virtual setting or in a socially-distanced fashion (Sharma et al., 2020; Brown et al., 2020) . Conversely, the sharp increase in trials focused on anxiety disorders may reflect a relative ease in transitioning these studies to virtual formats. This increase in studies on anxiety disorders may also be partially explained by socioeconomic stresses related to COVID-19, and may reflect the observed increase in diagnosis rates of anxiety and depression that have been observed during the pandemic (Ettman et al., 2020) .

The re-allocation of funding away from psychiatric clinical trials for COVID-19 research may further explain the analysis by condition. Trials for conditions that may require greater funding-such as schizophrenia spectrum and neurocognitive disorders-were the hardest hit, and also played a major role in the decreases noted in our findings.

This study serves to provide an outlook on how COVID-19 impacted clinical trials in psychiatry, and identifies trends that may prove long lasting after the pandemic. This study is limited, only analyzing studies submitted to clinical-trials.gov, which has been shown in previous studies to lack completeness J o u r n a l P r e -p r o o f compared to other commercial databases (Stergiopoulos et al., 2020) . Further, this study only provides a large-scale analysis of the trends, but opens the door for future work into the changing nature of clinical research in psychiatry. Moreover, this study is unable to demonstrate causality between COVID-19 and observed changes in 2020, with the exception of analysis of trials that explicitly cite COVID-19 as a reason for their cancellation. Despite these limitations, this paper shows that psychiatric clinical trials in 2020 have been significantly impacted, and understanding the nature and quality of these changes may prove invaluable for understanding the long-term impacts COVID-19 may have on psychiatric research. Overall chi-squared p > 0.05 Table 1 . Chi-square analysis comparing observed vs. expected study enrollment characteristics. All statistical results corrected for multiple analysis by the Bonferroni method. All predicted

Improving Public Access to Aggregate Content of ClinicalTrials

Anticipating and mitigating the impact of the COVID-19 pandemic on Alzheimer's disease and related dementias

Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic

Preserving clinical trial integrity during the coronavirus pandemic

COVID-19 and Clinical Trials: The Medidata Perspective

) MeSH. Mental Disorders MeSH Descriptor Data

Repurposing psychiatry during the COVID-19 pandemic. The Journal of clinical psychiatry

Impact of COVID-19 on clinical trials and clinical research: A systematic review

Focus on mental health during the coronavirus (COVID-19) pandemic: applying learnings from the past outbreaks

Evaluating the Completeness of ClinicalTrials. gov. Therapeutic innovation & regulatory science

A projection for psychiatry in the post-COVID-19 era: potential trends, challenges, and directions. Molecular psychiatry

Impact of the COVID-19 pandemic on clinical research

All persons who have made substantial contributions to the work reported in the manuscript (e.g., technical help, writing and editing assistance, general support), but who do not meet the criteria for authorship, are named in the Acknowledgements and have given us their written permission to be named. If we have not included an Acknowledgements, then that indicates that we have not received substantial contributions from non-authors.