key: cord-0988928-blejae3t authors: Myers, Taylor L.; Tarolli, Christopher G.; Adams, Jamie L.; Barbano, Richard; Cristina Gil-Díaz, María; Spear, Kelsey L.; Lowell, Jill; Daeschler, Margaret; Riley, Lindsey; Amondikar, Ninad; Auinger, Peggy; Marras, Connie; Tanner, Caroline M.; Ray Dorsey, E.; Schneider, Ruth B. title: Video-based Parkinson’s disease assessments in a nationwide cohort of Fox Insight participants date: 2021-05-24 journal: Clin Park Relat Disord DOI: 10.1016/j.prdoa.2021.100094 sha: 8213f3de7d044c349a139639b3eca3e86140ad5b doc_id: 988928 cord_uid: blejae3t INTRODUCTION: Parkinson’s disease (PD) research is hampered by slow, inefficient recruitment and burdensome in-person assessments that may be challenging to conduct in a world affected by COVID-19. Fox Insight is an ongoing prospective clinical research study that enables individuals to participate in clinical research from their own homes by completing online questionnaires. To date, over 45,000 participants with and without PD have enrolled. We sought to validate self-reported PD diagnosis in the Fox Insight cohort, assess the validity of other self-reported health information, and evaluate the willingness of participants to participate in video-based research studies. METHODS: Individuals with and without self-reported PD enrolled in Fox Insight were invited to participate in this virtual research study. Participants completed online questionnaires and two virtual visits, during which we conducted standard cognitive and motor assessments. A movement disorder expert determined the most likely diagnosis, which was compared to self-reported diagnosis. RESULTS: A total of 203 participants from 40 U.S. states, 159 with remote clinician-determined PD and 44 without, completed the study (59% male, mean (SD) age 65.7 (9.8)). Level of agreement between self-reported PD diagnosis in Fox Insight and clinician-determined diagnosis was very good ((kappa = 0.85, 95% CI 0.76–0.94). Overall, 97.9% of participants were satisfied with the study, 98.5% were willing to participate in a future observational study with virtual visits, and 76.1% were willing to participate in an interventional trial with virtual visits. CONCLUSION: Among the Fox Insight cohort, self-reported diagnosis is accurate and interest in virtual research studies is high. The success of clinical research is dependent upon patient participation, yet the traditional research model generally limits participation to those residing near a research center and burdens participants with frequent travel. As a result, less than one third of clinical trials successfully meet their recruitment targets [1] . This burden is especially problematic in neurodegenerative diseases such as Parkinson's disease (PD) where driving ability and functional mobility may be impaired, and caregiver burden is high [2] [3] [4] [5] . The COVID-19 pandemic has further heightened concerns about requiring individuals to travel to research centers for study participation [6] . Virtual studies allow individuals to participate in research from their homes, mitigating the risk of infection and reducing disability-and geography-based barriers to participation [7, 8] . Prior studies have demonstrated the feasibility of video-based research visits in a broad range of neurodegenerative disorders and high levels of interest in the model [7, 9, 10] . However, adoption in research studies has been limited to date. Fox Insight (FI) is an ongoing online prospective clinical research study that has enrolled over 45,000 participants with and without selfreported PD to date [11] . On a quarterly basis, participants complete an extensive series of standard assessments pertaining to quality of life, non-motor symptoms, physical activity, and general health [12] . This work is the first effort to validate self-reported diagnosis in FI. Validation of self-reported PD diagnosis is essential for all online-based research studies; a similar effort was undertaken with 23andMe's Parkinson's Research Community [7] . Here, we report the results of a virtual cross-sectional study of a subset of FI participants using video-based visits. Our primary objective was to assess the validity of self-reported PD diagnosis in FI against remote clinician-determined diagnosis. Our hypothesis was that agreement would be very good. Secondary aims were to expand the clinical characterization of participants, assess the validity of self-reported health information, and assess the willingness of FI participants to participate in future video-based research studies. Research team members based at the University of Rochester (UR) conducted all virtual study visits. The study was approved by the UR Institutional Review Board (STUDY0000176). All participants provided informed consent. Eligible participants were adult men and women with and without a self-reported diagnosis of PD (from a healthcare provider) recently enrolled in FI, who had completed their baseline study visit, resided in the United States, and had access to an internet-enabled device that would support study visits. In addition, eligible participants were on a stable regimen of both PD and non-PD medications since their baseline FI study visit and could complete all study related activities within 6 weeks of completing their baseline FI study visit. We used disproportionate stratified random sampling to enroll five approximately equal cohorts: self-reported PD ≤2 years since diagnosis, 3-5 years since diagnosis, 6-9 years since diagnosis, ≥10 years since diagnosis, and without PD. Members of the FI team sent weekly invitation emails to a random sample of individuals who had completed their baseline visit within the last 3 weeks. Interested individuals used a personalized link on their FI page that directed them to a form in the Research Electronic Data Capture (REDCap; Vanderbilt University) system where they provided their contact information and consent to be contacted by a UR coordinator. A coordinator then conducted a pre-screening telephone call to provide an overview of the study, assess interest, and determine eligibility. Individuals reviewed an eConsent document in REDCap and provided their consent to participate. Participants could opt-in to receive an individualized summary of results-a brief description and scores for the motor examination and Schwab and England-and/or have it sent to a provider. We mailed participants without a web camera a suitable camera at no cost. Participants completed a series of questionnaires in REDCap covering demographic information, PD symptom and diagnosis history (Suppl Table 1 ), and fall frequency. Also included were the Movement Disorders Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) parts Ib and II [13] . Subsequently, participants completed one virtual test visit with a coordinator and one virtual research visit with a movement disorders expert (RBS, CGT, RB, JLA) using HIPAA-compliant Zoom video conferencing software (San Jose, CA). Visits were typically completed within an hour. During the virtual test visit, a coordinator completed any troubleshooting necessary related to audio, video, or internet connection. Medications were documented and a Montreal Cognitive Assessment (MoCA) was conducted [14] . Video-based administration of the MoCA has been shown to be feasible in PD [15, 16] and the reliability of videobased administration has been established in non-PD populations [17] . A copy of the visuospatial/executive and naming portions was sent to the participant in advance. During the virtual research visit, the investigator reviewed the medications, collected a health history, and performed the modified MDS-UPDRS motor examination, which excludes tests of rigidity and postural instability that cannot be performed remotely. Video-based administration of the modified MDS-UPDRS motor examination correlates moderately with in-person administration [18] but has not been validated for longitudinal use [19] . Investigators, who were blinded to self-reported PD status, conducted an unscripted history and performed additional examination maneuvers to make a clinical diagnostic determination (PD, no PD, or alternative neurological diagnosis). They also applied three sets of PD diagnostic criteria, modified for remote assessment: the United Kingdom Parkinson's Disease Society Brain Bank Criteria (UK PD Criteria) [20] , NIH Diagnostic Criteria for Parkinson's Disease (NIH PD Criteria) [21] , and the Movement Disorder Society Clinical Diagnostic Criteria for Parkinson's Disease (MDS PD Criteria) [22] . Features that could not be assessed remotely, including rigidity, postural instability, graphesthesia, stereognosis, pyramidal weakness, pathologic hyperreflexia, and Babinski sign, were excluded. Following this visit, participants completed a satisfaction and research interest survey (Suppl Table 2 ). We asked participants to walk with their assistive device as applicable and allowed for gait assessment to be deferred in the presence of any safety concerns. In preparation for possible urgent medical events, we collected contact information for an emergency contact and local emergency services and provided guidance to the study team regarding appropriate actions. We used Cohen's kappa coefficient to determine the level of agreement beyond chance between self-reported diagnosis, cliniciandetermined diagnosis, and diagnostic criteria. We determined that a sample size of 200 participants would provide >80% power to detect a true kappa value of at least 0.8 (indicating very good agreement) assuming the null hypothesis value of kappa to be 0.6, using a significance level of 5%. We explored variation in level of agreement according to self-reported PD duration, temporal proximity from FI baseline to the virtual visit, disease-specific clinical features, and demographic features. We used descriptive statistics to assess willingness to participate, satisfaction with this study, and willingness to participate in future studies as well as to characterize participants. We used observed percent agreement to determine agreement between PD medications and health history (presence/absence) as self-reported in FI and obtained by the study team. We assessed associations between the Geriatric Depression Scale scores from FI and the MDS-UPDRS depression item using Spearman correlation. To explore factors associated with participation, we used logistic regression with enrollment (yes or no) as the outcome variable and age, sex, ethnicity, education, household income, and selfreported disease-duration as predictor variables. The FI team invited 2125 individuals (1559 with self-reported PD, 566 without self-reported PD) to participate (Fig. 1) . A total of 374 (304 (19.5%) with self-reported PD and 70 (12.4%) without self-reported PD) provided consent to contact (17.6% response rate). Examining the 2125 invited, those with self-reported PD duration of 3-10 years were more likely to participate than those without self-reported PD (OR 1.7, 95% CI 1.1-2.7, p-value 0.02) ( Table 1) . Age, sex, ethnicity, level of education, and household income were not associated with participation (all p > 0.05). We successfully contacted 265 (70.9%) of those who provided consent to contact. Twelve declined participation in the study, 23 were ineligible, and 7 were lost to follow-up (Fig. 1) . Ultimately, 223 individuals consented to participate with 98.5% requesting a summary of their results for themselves, 46.8% for their primary care provider, and 54.2% for their neurology provider. Between September 10, 2018 and April 26, 2019, 213 completed the virtual test visit (enrolled). We enrolled an average of 6.5 participants per week and completed study recruitment in approximately 33 weeks. A total of 203 participants from 40 states completed the virtual research visit and were the focus of our analyses (Fig. 2) . Fox Insight participants from all 50 states were invited and ten states (Alaska, Arkansas, Delaware, Idaho, Iowa, Kansas, New Hampshire, New Mexico, North Dakota, and South Dakota) were not represented in this validation study. Virtual research visits were completed a mean (SD) of 37.8 (10.1, range 11-73) days after completion of the FI baseline. The final cohort was 59% male and 96% white with a mean (SD) age of 65.7 (9.8) ( Table 1 ). A total of 38 self-reported no PD; 58 self-reported PD ≤2 years since diagnosis, 41 self-reported 3-5 years since diagnosis, 32 selfreported 6-9 years since diagnosis, and 34 self-reported ≥10 years since diagnosis. Agreement between self-reported PD diagnosis in FI and cliniciandetermined PD diagnosis was very good (kappa = 0.85, 95% CI 0.76-0.94) with stronger agreement (kappa = 0.92 versus 0.75, p = 0.08) when diagnoses were determined in close temporal proximity (≤median of 38 days compared to >38 days apart). Disagreement occurred in 10 (4.9%) of 203 cases (Table 2a ). In four of these cases, selfreported diagnosis in FI differed from self-reported diagnosis provided in this study; the clinician determination agreed with the self-reported diagnosis provided in this study in each of these cases. Agreement between self-reported PD diagnosis in FI and in this validation study was very strong (kappa = 0.89, 95% CI 0.81-0.97). Disagreement occurred in 7 (3.4%) of 203 cases. Six individuals self-reported PD in FI but not in this study and one individual self-reported PD in this study but not in FI. There were no significant differences in level of agreement between selfreported PD diagnosis in FI and clinician-determined PD diagnosis according to self-reported PD duration, age, sex, race, ethnicity, or presence of rest tremor on examination. With modified remote assessment, 42.6% met UK PD Criteria, 60.9% (Table 2b) . We examined cases where clinicians determined PD but the participant did not meet MDS PD Criteria (n = 80), UK PD Criteria (n = 72), or NIH PD Criteria (n = 35); in 68 (85%), 45 (63%), and 10 (29%) of cases respectively the participant had bradykinesia but did not meet core parkinsonism criteria. Results were similar examining cases of disagreement between self-reported diagnosis and the diagnostic criteria. Greater agreement was observed between self-reported PD diagnosis and the NIH PD Criteria with self-reported PD duration ≥10 years compared to 0-2 years (86% vs 67%, p = 0.05), but not for UK PD Criteria or MDS PD Criteria. There was 94% observed agreement between self-reported, No falls occurred during the visits and no urgent medical events occurred that required contacting local emergency services. 97.9% of participants were satisfied or very satisfied with the overall study (Suppl Fig. 1a) . More than 90% of participants felt they were accurately assessed, it was easy to participate, and they would recommend participation in virtual studies (Suppl Fig. 1a) . A small number had concerns regarding confidentiality (6.6%) and communication (2.0%), and a small proportion (12.1%) would have preferred in-person visits. Participants indicated a high level of willingness to participate in future virtual studies. 98.5% were willing to participate in a future observational study with virtual visits, 76.1% in an interventional trial with some virtual visits, and 68.5% in an interventional trial with all virtual visits. More than 50% of participants indicated they were more interested in participating in observational and interventional studies that included virtual visits (Suppl Fig. 1b) . This virtual research study confirmed the accuracy of self-reported diagnosis of PD in FI, established the validity of remotely entered clinical information, and demonstrated great interest in virtual research studies from participants. Self-reported PD status is accurate in the FI cohort. A similarly high level of agreement for PD diagnosis was seen in a study with 23andMe's Parkinson's Research Community and in a study with Fox Trial Finder (an online research interest registry) [7, 9] . Critically, the level of agreement between self-reported and clinician-determined diagnoses was not associated with self-reported PD duration, age, sex, race, ethnicity, or presence of rest tremor on examination. Agreement between self-reported or clinician-determined diagnoses and the diagnostic criteria was only fair to moderate. 29-85% of participants with clinician-determined PD and bradykinesia did not meet core parkinsonism diagnostic criteria. The UK PD Criteria require the presence of bradykinesia plus at least one of the following-rigidity, rest tremor, or postural instability [20] . The NIH PD Criteria for possible PD require the presence of at least two of the following-rest tremor, bradykinesia, rigidity, and asymmetric onset [21] . The MDS PD Criteria require the presence of bradykinesia plus either rigidity or rest tremor [22] . Neither rigidity nor postural instability can be readily assessed remotely forcing a heavier reliance on the presence of rest tremor, which may be absent, suppressed by medications, or missed during remote assessment due to limited direct visualization. Higher levels of agreement were seen with the NIH PD Criteria for those with self-reported PD duration >10 years compared to 0-2 years. These results suggest that the use of these diagnostic criteria as part of the eligibility criteria for a remote study may result in inappropriate exclusion of PD participants. Future work should focus on the development and validation of modified diagnostic criteria for remote PD diagnosis that consider the limitations of a virtual exam. Recruitment was rapid with an average of 6.5 participants enrolled per week at a single site. In comparison, the OBSERVE-PD study (an inperson, cross-sectional observational study of individuals with PD) recruited an average of 0.42 participants per week at each of its 128 sites [23, 24] . Our cohort was geographically distributed, representing 40 states with 43% of participants residing in Health Professional Shortage Areas, areas with insufficient primary care, dental, or mental health professionals to serve the population [31]. Only 10% of individuals contacted by the FI team participated in this research study, and our recruitment efforts benefited from a large pool of potential participants. Strategies to improve responses to email requests for research participation may include tailoring language to specific groups (e.g. healthy controls), providing payment, and including individuals with PD in the initial development of research studies and recruitment materials. We were able to successfully recruit controls and individuals across disease stages, suggesting that this approach could be used for clinical trials recruiting early, de novo PD to moderate PD. The study was well received with satisfaction rates exceeding 95%. The value of video-based visits has never been more apparent than during the on-going COVID-19 pandemic. COVID-19 has interrupted PD clinical research [25] , with likely long-term consequences for ongoing study conduct. Video-based visits, at-home collection of patientreported outcomes, and the use of digital sensors to capture real-world data can all help move clinical research forward. Patients are eager for new, more accessible research models; 68.5% of our participants indicated willingness to participate in an interventional trial with all virtual research visits. This study was completed prior to the COVID-19 pandemic; with increasing familiarity with video-based visits we suspect that even more patients would be interested in engaging in this type of research now. New, patient-friendly approaches are already in use, such as in an on-going trial of zoledronic acid for fracture prevention in PD [26] . This study is being conducted entirely in the patient's home and, for some, will include remote verification of self-reported diagnosis-an approach supported by the results of our study. Nearly all participants (98.5%) requested a summary of their results. The return of individual results is consistent with the ethical principle that participants have the right to receive data generated from their research participation [27] . While there are reasonable concerns regarding potential psychological distress and a blurring of the line between clinical care and research, recent guidelines have enabled more transparent practices [28] . The high request rate may be attributable to the fact that the majority of participants had already received a PD diagnosis, allaying any anxiety regarding results, or to our uncommon practice of proactively asking participants if they wanted their results. With the proper protocols in place to do so safely and ethically, the return of research results to participants can build trust and encourage participation in future research. Our study had limitations that merit further discussion. First, our 200-person cohort represented only a fraction of participants in the overall FI cohort and was not socio-demographically diverse. While the proportion of Americans with access to high-speed broadband at home is increasing, access is lower for older individuals, minorities, those with lower incomes and those with lower educational levels [29] . The demographics of our PD cohort and the PD cohort in FI were similar (mean (SD) age 66.7 (8.9) vs 65.8 (9.5), 61.0% vs. 54.5% male, 95.0% vs 96.9% white, and 2.6% vs 3.8% Hispanic/Latino [30] ) suggesting that differential access to high-speed broadband may have impacted recruitment for FI and this validation study. Best practices for recruiting diverse cohorts may differ between in-person and virtual studies; more research is needed to identify best practices for the recruitment of individuals under-represented in research into virtual research studies. Second, while interest in future virtual research studies was high, this was a selfselected cohort already engaged in online clinical research and these findings may not translate to a broader population. Third, investigators were blinded only to self-reported diagnosis and, similar to a clinical encounter, were not blinded to medications or any other historical information. We did not capture information on how investigators arrived at the most likely diagnosis or which historical items contributed most to their determination. Lastly, we validated self-reported diagnosis through virtual visits, which are not without limitations. As discussed above, key elements such as rigidity and postural instability as well as reflexes cannot be assessed, and the modified MDS-UPDRS motor examination has not been satisfactorily validated. Potential strategies to mitigate the limitations of virtual assessment and improve the accuracy of remote PD diagnosis, include the collection of clinical records and confirmation through locally-obtained testing (such as dopamine transporter imaging). Notwithstanding these limitations, our study supports the validity of self-reported diagnosis in a large observational study, which is crucial to the interpretation of the validity of FI results. Moreover, our study demonstrates the promise of a much-needed novel model for conducting PD research and highlights the potential of such a model for re-starting and advancing clinical research in PD in the midst of a pandemic. This study was supported by the Michael J. Fox Foundation for Parkinson's Research (15983). Authors MD, LR, and NA are or were formerly employed by the funder, the Michael J. Fox Foundation for Parkinson's Research. Funding acquisition, Writing -original draft What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies Caregiver burden in Parkinson's disease Parkinson disease and driving: an evidence-based review Factors contributing to caregivers' stress and burden in Parkinson's disease What is functional mobility applied to Parkinson's disease? Participant and caregiver perspectives on clinical research during Covid-19 pandemic Virtual research visits and direct-to-consumer genetic testing in Parkinson's disease Feasibility of virtual research visits in Fox trial finder A pilot study of virtual visits in Huntington disease Fox Insight collects online, longitudinal patient-reported outcomes and genetic data on Parkinson's disease -sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results MoCA: a brief screening tool for mild cognitive impairment A feasibility study of conducting the Montreal Cognitive Assessment remotely in individuals with movement disorders, Health Inf Could everyday technology improve access to assessments? A pilot study on the feasibility of screening cognition in people with Parkinson's disease using the Montreal Cognitive Assessment via Internet videoconferencing The reliability of the Montreal Cognitive Assessment using telehealth in a rural setting with veterans Reliability, and Value of Remote Video-Based Trial Visits in Parkinson's Disease Movement disorder society-unified Parkinson's disease rating scale use in the Covid-19 era Accuracy of clinical diagnosis of idiopathic Parkinson's disease: a clinico-pathological study of 100 cases Diagnostic criteria for Parkinson disease Günther Deuschl, MDS clinical diagnostic criteria for Parkinson's disease Characterizing advanced Parkinson's disease: OBSERVE-PD observational study results of 2615 patients A multi-country, observational, cross-sectional study to characterise advanced Parkinson's Disease patients in Movement Disorder Centres I.C. and the impact of the COVID-19 pandemic on Parkinson's disease and MOVEMENT DISORDERS Identifier: NCT03924414, Trial of Parkinson's And Zoledronic Acid (TOPAZ) Return of research results to study participants: uncharted and untested Committee on the Return of Individual-Specific Research Results Generated in Research Laboratories S, behalf of the Fox Insight, comparison of an online-only Parkinson's disease research cohort to cohorts assessed in person Supplementary data to this article can be found online at https://doi. org/10.1016/j.prdoa.2021.100094.