key: cord-0021689-i5rj4u3w authors: Papp, Kathryn V.; Samaroo, Aubryn; Chou, Hsiang‐Chin; Buckley, Rachel; Schneider, Olivia R.; Hsieh, Stephanie; Soberanes, Daniel; Quiroz, Yakeel; Properzi, Michael; Schultz, Aaron; García‐Magariño, Iván; Marshall, Gad A.; Burke, Jane G.; Kumar, Raya; Snyder, Noah; Johnson, Keith; Rentz, Dorene M.; Sperling, Reisa A.; Amariglio, Rebecca E. title: Unsupervised mobile cognitive testing for use in preclinical Alzheimer's disease date: 2021-09-30 journal: Alzheimers Dement (Amst) DOI: 10.1002/dad2.12243 sha: cf9535b764041827513dfbbccab207a8d337dc27 doc_id: 21689 cord_uid: i5rj4u3w INTRODUCTION: Unsupervised digital cognitive testing is an appealing means to capture subtle cognitive decline in preclinical Alzheimer's disease (AD). Here, we describe development, feasibility, and validity of the Boston Remote Assessment for Neurocognitive Health (BRANCH) against in‐person cognitive testing and amyloid/tau burden. METHODS: BRANCH is web‐based, self‐guided, and assesses memory processes vulnerable in AD. Clinically normal participants (n = 234; aged 50–89) completed BRANCH; a subset underwent in‐person cognitive testing and positron emission tomography imaging. Mean accuracy across BRANCH tests (Categories, Face‐Name‐Occupation, Groceries, Signs) was calculated. RESULTS: BRANCH was feasible to complete on participants’ own devices (primarily smartphones). Technical difficulties and invalid/unusable data were infrequent. BRANCH psychometric properties were sound, including good retest reliability. BRANCH was correlated with in‐person cognitive testing (r = 0.617, P < .001). Lower BRANCH score was associated with greater amyloid (r = –0.205, P = .007) and entorhinal tau (r = –0.178, P = .026). DISCUSSION: BRANCH reliably captures meaningful cognitive information remotely, suggesting promise as a digital cognitive marker sensitive early in the AD trajectory. Mobile cognitive assessment may expedite the screening and tracking of participants for secondary prevention. 3 By allowing participants to complete study assessments using their own electronic device, data collection exponentially increases, improving clinical trial efficiency. Interest in using digital and remote assessments is growing and multiple studies have demonstrated the feasibility and validity of computerized assessments in supervised [4] [5] [6] and unsupervised settings 7, 8 using personal devices. 9, 10 Several of these assessments are designed for specific symptomatic groups (e.g., for detection of mild cognitive impairment [MCI]) 9 or broad populations. 6 A smaller subset of mobile assessments have been specifically designed to capture subtle cognitive decline in preclinical AD. [11] [12] [13] [14] Decrements in episodic memory, particularly paired associative memory (i.e., integrating contextual information such as linking a face with a name) as well as associative inference, 15 are observed in preclinical AD. [16] [17] [18] Regions critical for associative memory such as the entorhinal cortex and anterior hippocampus 19, 20 are also early sites for phosphorylated tau neurofibrillary tangles. 21 Likewise, pattern separation (i.e., the ability to discriminate between previously learned items and those that are perceptually similar 22 ) is an aspect of memory performance reliant on the dentate gyrus and connections to the entorhinal cortex and hippocampus. 23 Finally, measures which facilitate learning using semantic cues have been shown to be particularly useful in identifying decrements in memory encoding in preclinical AD. 24, 25 Digital cognitive tasks that target cognitive processes that decline during preclinical AD, such as the Boston Remote Assessment for Neurocognitive Health (BRANCH), would be particularly relevant to prevention trials that are seeking cognitive outcomes that can track putative therapeutic benefit. Here, we provide initial validation for BRANCH, a web-based cognitive battery designed for unsupervised completion on a personal electronic device (e.g., smartphone). Tasks were designed to challenge the aforementioned memory processes in clinically normal (CN) older adults. We designed an intuitive interface to facilitate unsupervised testing and used stimuli relevant to everyday life (faces, groceries, street signs). We deployed BRANCH to two groups of CN older adults: (1) a registry sample without in-person contact to assess BRANCH feasibility "in the wild" and an (2) observational sample to assess validity in relation to in-clinic traditional assessments and AD biomarkers. If truly a valid cognitive measure, we expected lower BRANCH performance to be associated with increasing age given well-known agerelated cognitive decline, even in the absence of neurodegenerative disease. 26 Additionally, we expected BRANCH performance to be correlated with traditional in-clinic cognitive assessments. Furthermore, we expected those with higher AD pathologic burden (i.e., amyloid and tau measured with biomarkers) to be associated with lower BRANCH performance. In addition to testing the aforementioned hypotheses, BRANCH reliability was assessed using a re-test paradigm in a subset of registry participants. Finally, we examined the feasibility of BRANCH in MCI participants in a separate well-characterized cohort, to ensure BRANCH remained feasible for participants who progress to MCI over the course of a secondary prevention trial. The registry sample was recruited from two online local registries; exclusion criteria included self-report of MCI/dementia and participants were presumed CN by self-report. BRANCH initial piloting efforts are described in Supp A. BRANCH was developed using a web-based platform, which met hospital data privacy and security requirements. BRANCH was sent to participants via e-mail/text and can be completed on any web-enabled device (Figure 1 ). Accuracy serves as the primary outcome across tests; however, exploratory analyses were complete on reaction time outcomes. After BRANCH, participants were surveyed regarding technical difficulties. Participants rated instruction clarity (yes/no), task difficulty (very easy-very difficult; five options), and task engagement (not engaging-highly engaging; five options) by task. To determine the usability of BRANCH data, we implemented two cutoffs, both of which were required for data to be usable. First, tasks were considered complete if > 90% of items were completed. Second, participants were required to exhibit > 66% accuracy on the learning portion of the Categories test. Observational sample participants underwent positron emission tomography (PET) 7 with 11C Pittsburg Compound-B (PiB; n = 144) and F18 Flortaucipir (FTP; n = 129) using previously published procedures 43 within 3 and 2 years of BRANCH administration, respectively. FTP images were acquired from 75 to 105 minutes and PiB images were acquired using a 60-minute dynamic acquisition on a Siemens ECAT HR+ PET scanner. PET images were co-registered to corresponding T1 images using FreeSurfer-based(v6) structural regions of interest (ROIs) mapped into native PET space using SPM12. FTP was expressed as a standardized uptake volume ratio (SUVR) and PiB as the distribution volume ratio (DVR). The reference region was cerebellar gray using an magnetic resonance imaging (MRI)-based method; FTP-PET data were corrected for partial volume effects. For PiB, a global cortical aggregate was calculated. For FTP, entorhinal cortex was used because of its importance to memory and status as an early site of tau deposition. Statistical analyses were completed using R (v4. Demographics for registry (n = 79) participants and observational (n = 155) participants are shown in Table 1 Seventy-one percent of participants used smartphones/tablets. Remaining participants used a desktop/laptop. Rates of unusable data (as defined above) were low at 3% suggesting good feasibility (few task discontinuations). Rates of self-reported difficulty completing the task (2.97%) and having difficulty understanding task instructions (2.97%) were low ( Table 2 ) further supporting feasibility. Self-reported technical difficulties were higher (15.84%) with the most common concern being finger tap response latency followed by slow-loading task images. Participants reported that the Face-Name-Occupation and Groceries tasks were the most challenging tasks while Categories was least challenging (Table 2 ). Most participants found the tasks engaging ( Psychometric properties of BRANCH are shown in Table 3 . BRANCH Four of the six accuracy outcomes exhibited mean performance within 70% to 80%, consistent with our original goals. An exception, by design, was the Categories recall task, which was much less challenging (mean = 93%). Groceries recall was more challenging (mean = 53%) than initially intended. However, incorporating both an easier and more challenging task is useful for populations that may include both CN and MCI. There were no floor effects. Ceiling effects were observed for the Categories test (65% at ceiling) but otherwise minimal across other measures. Apart from the categories test, all BRANCH outcomes were normally distributed (defined by skewness ranging from -0.5 to 0.5; Table 3 ). Older age was associated with worse BRANCH performance (r = -0.190, P = .004; Figure 2 ). The magnitude of the age effect on cognition was similar across samples (observational: r = -0.262, P = .001; registry: r = -0.217, P = .055). Across both cohorts, slower reaction time on BRANCH outcomes was associated with older age (Supp C in supporting information). Registry participants re-taking BRANCH were of comparable age, sex, Among the observational sample with neuroimaging (Supp E in supporting information), lower BRANCH composite performance was F I G U R E 3 Associations between Boston Remote Assessment for Neurocognitive Health (BRANCH) composite performance and age, positron emission tomography amyloid and tau, and Preclinical Alzheimer Cognitive Composite (PACC-5) score. For age sample, n = 234; for PACC-5 sample, n = 160; for Pittsburgh Compound B (PiB) sample, n = 144; for flortaucipir (FTP) sample, n = 129; r values for PiB and FTP are controlled for age associated higher cortical amyloid (r = -0.205, P = .007) and entorhinal tau (r = -0.178, P = .026; Figure 2 ; Table 3 ). As a point of comparison, the association between PACC-5 and cortical amyloid was r = -0.191, P = .033 and r = -0.159, P = .076 for entorhinal tau. For individual BRANCH tests, amyloid was negatively correlated with both Groceries test outcomes and Categories Recall. Greater entorhinal tau was associated with lower performance on both Face-Name-Occupation tasks. Signs was not related to amyloid, but was related to tau but in an unexpected direction. Examining BRANCH reaction time data, slower response speed across a few outcomes (Categories, Groceries Pattern Separation, Signs) was associated with higher amyloid (Supp E). No significant associations between reaction times and BRANCH tasks were observed for entorhinal tau. Here, we showed the feasibility of BRANCH to be deployed remotely in an unsupervised setting, both among participants well-versed in cognitive testing in an observational sample and a purely remote registry To compare, a meta-analysis of studies of CN older adults found an association of 0.12 between traditional memory measures and PET amyloid. 52 Interestingly, associations between AD biomarkers were numerically stronger for the BRANCH composite versus the PACC-5, which may be attributable to the emphasis on sensitive memory measures in BRANCH. We did not initially set out to examine reaction time data because web-based cognitive testing is confounded by internet speed/connectivity and inter-device variability. However, we were encouraged to see that slower reaction time on BRANCH tasks was associated with older age and slower performance on convergent processing speed measures (e.g., DSST, TMTB). These data suggest that reaction time data from a web-based program may be more promising than initially imagined and, in the future, may supplement accuracy measures. A potential limitation of any unsupervised cognitive testing is whether the person assigned to the task is the person completing the task. To address this, we now ask participants to "attest" that they are the des- to enrich and contextualize cognitive data. 54 Our studies recruited individuals with their own digital devices who were comfortable with this technology, which is unlikely representative of the larger US older adult population. Another limitation of our study was the high (college) education level. However, we were able to achieve a balanced representation by sex (54% female) and a relatively racially diverse group (28% non-White registry sample). 55 Further work will be needed to determine feasibility of BRANCH in samples with lower educational attainment. Finally, while we demonstrated the feasibility of BRANCH in MCI in-clinic, future work must confirm BRANCH feasibility among MCI in remote/unsupervised settings. Initial remote testing of BRANCH among MCI patients (n < 5) has been promising and we are continuing to collect this data. Additionally, we plan to examine to extent which BRANCH may track cognitive decline over time, whether BRANCH change predicts MCI progression, and the diagnostic utility of BRANCH in differentiating between CN and MCI. Future work will explore short-term learning curves by capturing BRANCH more frequently and developing BRANCH for use in different languages. In this proof-of-concept study, we demonstrated that BRANCH provides reliable and meaningful cognitive data among CN individuals and shows small but persistent associations with AD biomarkers despite being deployed remotely/unsupervised. Digital capture of cognition, unlike traditional measures, has multiple benefits including increased accessibility, automated data capture and storage, scalability, and costeffectiveness. There is a need in AD secondary prevention for digital cognitive tests which target fundamental aspects of early memory decline and which can be completed on a personal device. Here, we provided evidence for the feasibility, reliability, and validity of BRANCH as a cognitive measure for use in preclinical AD. K. Johnson has received consulting fees from Novartis and Cerveau. NIA-AA research framework: toward a biological definition of Alzheimer's disease Toward defining the preclinical stages of Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease Mobile and pervasive computing technologies and the future of Alzheimer's clinical trials Associations between NIH Toolbox Cognition Battery and in vivo brain amyloid and tau pathology in nondemented older adults Computerized cognitive testing for use in clinical trials: a comparison of the NIH Toolbox and Cogstate C3 batteries Evaluation of the usability of a brief computerized cognitive screening test in older people for epidemiological studies The Feasibility of At-Home iPad Cognitive Testing For Use in Clinical Trials Diminished learning over repeated exposures (LORE) in preclinical Alzheimer's disease Unsupervised online neuropsychological test performance for individuals with mild cognitive impairment and dementia: results from the Brain Health Registry Computerized cognitive testing for older adults: a review Gallery Game: smartphone-based assessment of long-term memory in adults at risk of Alzheimer's disease Age impairs mnemonic discrimination of objects more than scenes: a web-based, large-scale approach across the lifespan Current advances in digital cognitive assessment for preclinical Alzheimer's disease. Alzheimers Dement (Amst) Remote cognitive assessment approaches in the Dominantly Inherited Alzheimer Network (DIAN) Using digital technology to drive clinical innovation in brain-behavior relationships: a new era in neuropsychology Memory integration: neural mechanisms and implications for behavior Hippocampal dysfunction in patients with mild cognitive impairment: a functional neuroimaging study of a visuospatial paired associates learning task Putting names to faces: successful encoding of associative memories activates the anterior hippocampal formation Novel cognitive paradigms for the detection of memory impairment in preclinical Alzheimer's disease Individual differences in associative memory among older adults explained by hippocampal subfield structure and function The anatomy of memory: an interactive overview of the parahippocampal-hippocampal network Neuropathological stageing of Alzheimer-related changes A task to assess behavioral pattern separation (BPS) in humans: data from healthy aging and mild cognitive impairment Strong evidence for pattern separation in human dentate gyrus Memory impairment on free and cued selective reminding predicts dementia Cued memory decline in biomarker-defined preclinical Alzheimer disease Relation of neuropathology with cognitive decline among older persons without dementia Aging Brain Study: dataset and accessibility Aging Brain Study: dataset and accessibility Clinical meaningfulness of subtle cognitive decline on longitudinal testing in preclinical AD Cognition, reserve, and amyloid deposition in normal aging Revised associative inference paradigm confirms relational memory impairment in schizophrenia Memory for grocery prices in younger and older adults: the role of schematic support Associative recognition in Alzheimer's disease: evidence for impaired recallto-reject Becoming a "Greeble" expert: exploring mechanisms for face recognition Contributions of human hippocampal subfields to spatial and temporal pattern separation Subtle memory decline over 12 months in mild cognitive impairment Aging associations: influence of speed on adult age differences in associative learning A computerized continuous-recognition task for measurement of episodic memory Does the Female Advantage in Verbal Memory Contribute to Underestimating Alzheimer's Disease Pathology in Women versus Men Screening for cognitive impairment among older people in black and minority ethnic groups Optimizing the preclinical Alzheimer's cognitive composite with semantic processing: the PACC5 Early and late change on the preclinical Alzheimer's cognitive composite in clinically normal older individuals with elevated amyloid beta Tau positron emission tomographic imaging in aging and early A lzheimer disease A basis for analyzing test-retest reliability Validation of a digitally delivered visual paired comparison task: reliability and convergent validity with established cognitive tests The Survey for Memory, Attention, and Reaction Time (SMART): development and Validation of a Brief Web-Based Measure of Cognition for Older Adults A comparison of the Cambridge Automated Neuropsychological Test Battery (CANTAB) with "traditional" neuropsychological testing instruments A computerized, self-administered test of verbal episodic memory in elderly patients with mild cognitive impairment and healthy participants: a randomized, crossover, validation study When does age-related cognitive decline begin The computerized cognitive composite (c3) in a4, an alzheimer's disease secondary prevention trial Amyloid deposition in younger adults is linked to episodic memory performance Meta-analysis of amyloidcognition relations in cognitively normal older adults Using technologies to prevent cheating in remote assessments during the COVID-19 pandemic Cognition in context: understanding the everyday predictors of cognitive performance in a new era of measurement The Alzheimer's Prevention Registry: a large internet-based participant recruitment registry to accelerate referrals to Alzheimer'sfocused studies Additional supporting information may be found in the online version of the article at the publisher's website.