key: cord-0289732-t2v9n4hw
authors: Saban, W.; Ivry, R.
title: PONT: A Protocol for Online Neuropsychological Testing
date: 2021-05-28
journal: nan
DOI: 10.1101/2021.05.27.21257953
sha: 156dd69f3264e580f927215dcb52fdf87fea1934
doc_id: 289732
cord_uid: t2v9n4hw

A major challenge for neuropsychological research arises from the fact that we are dealing with a limited resource: The patients. Not only is it difficult to identify and recruit these individuals, but their ability to participate in research projects can be limited by their medical condition. As such, sample sizes are small and considerable time (e.g., 2 yrs) is required to complete a study. To address limitations inherent to lab-based neuropsychological research, we developed a protocol for online neuropsychological testing (PONT). We describe the implementation of PONT and provide the required information and materials for recruiting participants, conducting remote neurological evaluations, and testing patients in an automated, self-administered manner. The protocol can be easily tailored to target a broad range of patient groups, especially those that can be contacted via support groups or multi-site collaborations. To highlight the operation of PONT and describe some of the unique challenges that arise in on-line neuropsychological research, we summarize our experience using PONT in a research program involving individuals with Parkinsons disease and spinocerebellar ataxia. In a 10-month period, by contacting 646 support group coordinators, we were able to assemble a participant pool with over 100 patients in each group from across the United States. Moreover, we completed six experiments (n>300) exploring their performance on a range of tasks examining motor and cognitive abilities. The efficiency of PONT in terms of data collection, combined with the convenience it offers the participants, promises a new approach that can increase the impact of neuropsychological research.

Understanding the functional organization of the brain requires the use of converging methods of 31 exploration. The spatial resolution of functional magnetic resonance imaging (fMRI) and 32 temporal resolution of electroencephalography (EEG) can reveal the dynamics of computations 33 in local regions, as well as coordination of activity across neural networks. A limitation with 34 these methods is that they are fundamentally correlational, revealing the relationship between 35 cognitive events and brain regions. Traditionally, perturbation methods such as optogenetics or 36 lesion studies, have been seen as approaches that provide stronger tests of causality. Lesion 37 studies, such as neuropsychological studies on patients with focal brain pathology, could provide 38 us with causal evidence for the potential role of a specific brain region in a cognitive function. 39 Empirical investigations on patients with brain pathology have been central in the 40 emergence of behavioral neurology as an area of specialization, and, with its more theoretical 41 focus, cognitive neuroscience. Information obtained through neuropsychological testing has 42 been fundamental in advancing our understanding of the nature of brain-behavior relations 43 (Luria, 1966) with important clinical implications (Lezak, 2000) . Neuropsychological testing is 44 not only important in evaluating the cognitive changes associated with focal and degenerative 45 disorders of specific brain regions but has also been an essential tool in determining the 46 contribution of targeted brain regions and particular cognitive operations. For example, 47 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 3 neuropsychological observations from the 19 th century led to the classic "Broca-Wernicke- 48 Lichtheim-Geschwind" model of language function (Geschwind, 1970; Poeppel, 2014) . 49 50 Challenges with neuropsychological testing 51 While neuropsychological research has been a cornerstone for cognitive neuroscience, its relative 52 importance to the field has diminished over the past generation with the emergence of new 53 technologies. This point is made salient when scanning the table of contents of the journals of the 54 field: Papers using fMRI and EEG dominate the publications, with only an occasional issue 55 including a study involving work with neurological patients. For example, in PubMed searches 56 restricted to papers since 2011, the key word "fMRI" yielded 409,310 hits, whereas the key word 57 "neuropsychological" yielded only 72,835 hits. 58 There are several methodological issues that constrain the availability and utility of 59 neuropsychological research. First, the studies require access to patients who have relatively 60 homogenous neurological pathology. This is quite challenging when drawing on patients with 61 focal insults from stroke or tumor, or when studying rare degenerative disorders. As such, the 62 sample size in neuropsychological research tends to be small, such as 8 participants (Casini & 63 Ivry, 1999). Often, researchers have very limited access to these hard-to-reach populations, 64 especially with rare disorders such as spinocerebellar ataxia (SCA). Second, given that the 65 recruitment is usually from a clinic or local community, the sample may not be representative of 66 the population; for example, samples may be skewed if based on patients who are active in 67 support groups. Third, given their neurological condition, the patients may have limited time and 68 energy. Their disabilities make it challenging to recruit the participants to come to the lab, and 69 for those that do, the experiments have to be tailored to avoid taxing the participants' mental 70 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 4 capacity. As a result, it can take quite a bit of time to complete a single experiment, let alone a 71 package of studies that might make for a comprehensive story. Hence, the progress and 72 efficiency of each study can be very restricted. 73 

Addressing the challenges with neuropsychological testing 75 One solution that we have pursued in our research on the cerebellum has been to take the lab into 76 the field, setting up a testing room at the annual meeting of the National Ataxia Foundation. This 77 conference is patient-focused, providing people with ataxia a snapshot of the latest findings in 78 basic research and clinical interventions, and an opportunity to share their experiences with their 79 peer group. For the past decade, we have set up a testing room at the conference and, over a 3-80 day period, been able to test about 15 people, a much more efficient way to complete a single 81 study compared to the more traditional approach of enlisting ataxia patients in the Berkeley 82 community to come to the lab. However, this approach is not ideal for multi-experiment projects, 83 and entails considerable cost to send a team of researchers required to recruit the participants and 84 coordinate the testing schedule. Conferences in which researchers are reaching out to patients are 85 still a slow and expensive solution. 86 An alternative and simpler solution might come from the use of the internet. Testing. The PONT protocol entails a comprehensive package that addresses challenges involved 110 in recruitment, neuropsychological evaluation, behavioral testing, and administration to support 111 neuropsychological research. In the initial ten-month period, we contacted 646 support group 112 coordinators to recruit large samples of individuals with degenerative disorders of the cerebellum 113 (SCA) or basal ganglia (Parkinson's disease (PD)). We established a workflow for online 114 recruitment, neuropsychological assessment, behavioral testing, and follow-up. During this ten-115 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 6 month period, we have completed six experiments testing motor and cognitive abilities, and 116 report one of these studies in this paper. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. can pass along to their group members. Although we don't explicitly request a response from the 156 administrators, our experience is that those who pass along our recruitment information inform 157 us of this in a return email. The response rate is modest: Following our initial email and a follow-158 up two weeks later to those who haven't responded, we estimate that approximately 15% of the 159 support groups forward our information to their members. While this number is low, it does 160 mean that our recruitment information has reached the membership of around 100 support 161 groups.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 8 This procedure is a variant of snowball sampling (Goodman, 1961) , a method that has 163 been employed to recruit patients and other hard-to-reach groups for clinical studies. Here we 164 modify this procedure to enlist participants via support groups and, cascading from there, word 165 of mouth for our behavioral studies. The materials used in this phase of the recruitment process 166 can be found at this online Outreach Forms link (Step #1; full URL address is provided in the 167 Appendix). These can be readily adopted by any research group that is targeting a population 168 associated with support organizations and web-based groups, or when the sample might be 169 enlisted as part of a multi-lab collaborations. is much more geographically diverse than would occur in a typical laboratory-based study. 181 Control participants were recruited via advertisements posted on the Craigslist website. 182 The advertisement instructed interested individuals to complete our online form. The 183 advertisement indicated that participation was restricted to individuals between the ages of 35 184 and 80. Over a 9-month period, we have enrolled 159 individuals in the control group.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Modifications were made to these assessment instruments to make them more appropriate 222 for online testing. For the MoCA test, we eliminated "Alternating Trail Making" since this would 223 require that we provide a paper copy of the task. For the UPDRS and SARA, we modified items 224 that require the presence of a trained individual to ensure safe administration. We eliminated the 225 "Postural Stability task" from the UPDRS since it requires that the experimenter apply an abrupt 226 pull on the shoulders of the participant. We modified three items on the UPDRS ("Arising from 227 Chair", "Posture", and "Gait"), obtaining self-reports from the participant rather than the 228 standard evaluation by the experimenter. Similarly, we obtained self-reports of stance and gait 229 for the SARA rather than observe the participant on these items. For the self-reports, we 230 provided the scale options to the participant (e.g., on the SARA item for gait, 0, normal/no 231 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint difficulty, to 8, unable to walk even supported). The scores for the MoCA and UPDRS batteries 232 were adjusted to reflect these modifications. For the online MoCA, the observed score was 233 divided by 29 (the maximum online score), and then multiplied by 30 (the maximum score on 234 the standard test). Hence, if a participant obtained a score of 26, the adjusted score will be 235 (26/29)*30, or 26.9. The same adjustment procedure was performed for the UPDRS. No 236 adjustment was required for the SARA. 237 The interview took around 30 minutes for the control participants and 40-60 minutes for 238 the PD and SCA participants. 

After completing the online experiment, the participant is sent an automated thank you note and 263 a short online form for providing feedback about the task (e.g., rate difficulty, level of 264 engagement, see Table 2 ). In addition, we sent the participant an email with information about To demonstrate the feasibility and efficiency of PONT, we report the results from an experiment 278 on motor sequence learning in which we used the discrete sequence production (DSP) task. On 279 each trial, the participant produces a sequence of four keypresses in response to a visual display. 280 We chose to use a motor sequence learning task for two reasons. response demands compared to tasks requiring a single response (e.g., 2-choice RT). As such, we 288 can evaluate the performance of our code across the range of hardware protocol s used by the 289 participants, as well as evaluate participants' performance on a relatively demanding motor task. 290 Skill acquisition has been associated with two prominent processes: 1) Memory retrieval 291 that becomes enhanced with practice (Logan, 1988) and 2) improved efficiency in the execution 292 of algorithmic operations (Tenison & Anderson, 2016). We modified the DSP task to look at 293 how degeneration of the BG or cerebellum impact these two learning processes. We compared 294 practice benefits for repeated items (memory-based learning) with practice benefits for non-295 repeated items (algorithm-based learning). While we expected the PD and SCA groups would be 296 slower than the controls overall, we expected sequence execution time would become faster over 297 the experimental session for all three groups. Our primary focus was to make group comparisons 298 of the learning benefits for the repetition and non-repetition conditions, providing assays of 299 memory-based and algorithm-based learning, respectively. 300 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (idiopathic ataxia). The mean duration since diagnosis for the SCA group was 5.9 years (SE=1.8) 316 and the mean SARA score was 9.1 (SE=.9, range: 2 -14.5). The mean duration since diagnosis 317 for the PD group was 6.5 years (SE=.9) and mean score on the motor section of the UPDRS was The participant used their home computer and keyboard to perform the experiment. Given this 330 and the fact that we did not control for viewing distance, the size of the stimuli varied across 331 individuals. All of the instructions were provided on the monitor in an automated manner, with 332 the program advancing under the participant's control. 333 The participant was instructed to place his or her fingers (thumbs excluded) on the 334 keyboard, using the keys "z", "x", "c", "v" for the left hand and the keys "m", "<", ">", and "?" 335 for the right hand. Eight placeholders were displayed on the screen, with each placeholder 336 corresponding to one of the keys of the keyboard in a spatially compatible manner ( Figure 2) . 337 At the start of each trial, a black fixation cross appeared in the middle of a white 338 background. After 500 ms, the fixation cross was replaced by a stimulus display that consisted of 339 the numbers, "1", "2", "3" and "4", with each number positioned over one of the placeholders. 340 The numbers specified the required sequence for that trial. In the example shown in Figure 2 , the 341 correct sequence required sequential key presses with the left index finger, right pinky, right 342 index finger, and left pinky. The instructions emphasized that the sequence of responses should 343 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 16 be performed as quickly and accurately as possible. If a keypress was not detected within 3000 344 ms for each element of a given sequence, the phrase "Respond faster" appeared for 200 ms. If a 345 single response was not detected within 4000 ms or the wrong key was pressed, the trial was 346 aborted. In the case of an erroneous keypress, a red "X" was presented above the placeholders. 347 When the entire sequence was executed successfully, a green "√" appeared above the 348 placeholders. The feedback screen remained visible for 500 ms, after which a fixation cross 349 appeared for 500 ms indicating the start of the inter-trial interval. 350 To test memory-based and algorithm-based learning, we created two nonoverlapping To ensure that participants remained attentive, we included five "attention probes" on the 363 instruction pages that appeared in the start of the experiment or during the experimental block. 364 For example, an attention probe might instruct the participant to press a specific key rather than 365 selecting the "next" button on the screen to advance the experiment (e.g., "Do not press the 366 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 17 "next" button. Press the letter 'A' to continue"). If the participant failed to respond as instructed 367 on these probes, the experiment continued, but the participant's results were not included in the 368 analysis. 369 The experiment took approximately 45 minutes to complete. 370 371 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. 

Accuracy rates were 80%, 78%, and 78% for the Control, SCA, and PD groups, respectively 387 (F(2,59)=.198, p=.821; one-way ANOVA). Most of the errors involved an erroneous keypress 388 (16% of all trials), and the remaining errors were due to a failure to make one of the keypresses 389 within 4 s (5% of all trials). We excluded trials in which participants failed to complete the full 390 sequence within 4 s (3% of the remaining trials). 391 Our primary dependent variable of interest was execution time (ET), the time from the 392 first key press to the time of the last key press. Before turning to these data, we considered 393 potential trade-offs in performance. First, to determine if there was a speed-accuracy trade-off, 394 Screen display Figure 2 . Each line on the screen corresponded to a finger position in a spatially compatible manner. The numbers indicate the order of the keypresses for the current trial. In this example, the participant should press left index ("v"), right pinky ("/"), right index ("m"), and left pinky ("z").

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 20 adults (Jouen, 2013) . the improvement was more pronounced for the repetition condition than for 418 the no-repetition condition (difference in slope: -11.8 ms/cycle, SE=3.7, p=.001). 419 The patient groups also got faster over the experimental session with an overall slope of 420 -29.2 ms/cycle for the PD group (SE=1.9, p<.0001) and -13.1 ms/cycle for the SCA group 427 We next compared the learning effects for each patient group to the Control group. 428 Both patient groups showed less improvement than the controls in the no-repetition condition, One exception is a study ) that observed normal learning when the 463 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 22 responses were made directly to the stimulus positions. It may be that these "direct" cues either 464 support the formation of distinct memory associations not impacted by cerebellar pathology 465 (e.g., direct stimulus-response mappings, not requiring an intermediary mapping operation), or 466 that the direct cues provide a boost to memory formation. One could consider the random blocks 467 as a long "non-repeating" sequence. For example, it was demonstrated that individuals with SCA 468 showed reduced practice benefits on both sequence and random blocks (Tzvi et al., 2017) , 469 similar to the deficits we observed for both repeating and non-repeating sequences, respectively. c.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 25 many experimental psychologists to move to on-line studies over the past decade, yet it is 512 tailored to address the unique demands of patient-based research. The main focus of this paper 513 was to outline the key steps required for this approach, with our report of the results from a 514 sequence learning included to provide a concrete example of the application of PONT. 515 PONT was developed to address five main challenges facing researchers who conduct 516 neuropsychological studies. Perhaps the most profound limiting factor that this valuable 517 approach addresses is access to a targeted population (see Figure 4 ), be it a disorder with a 518 relatively high prevalence rate in the population (e.g., 0.3% for PD) or one that is rare (e.g., 519 0.04% for SCA). Not only can it be difficult to recruit these individuals, but the patients 520 themselves may have limited time and energy given their neurological condition. As a result, it 521 can take a long time to complete a single experiment (e.g., 1-2 years), a problem that is 522 magnified for a project requiring multiple experiments. Indeed, neuropsychological papers tend 523 to involve just a single patient group and single experiment. PONT provides a protocol that 524 makes data collection extremely efficient. In the first 10 months following the launch of PONT, 525 we were able to complete six experiments, with each having around 20 participants per group. 526 Moreover, as our patient participant pool grows, we should be in a position to run studies with 527 much larger sample sizes, something that is relatively rare in the cognitive neuroscience 528 literature but desirable for looking at factors underlying heterogeneity in performance within a 529 particular group (e.g., due to genetic subtype, pattern of pathology). 530 Second, recruitment for laboratory-based experiments usually comes from a local 531 community. As such, the sample may be small and not representative of the population (see 532 Figure 4 ). For example, our PD sample from the Berkeley community tend to be highly educated 533 compared to the general PD population in the USA. In addition, if the study requires a relatively 534 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 26 homogenous pathology or genetic subtype (e.g., a particular variant of SCA), the sample size 535 likely to be small (e.g., 8 participants). By casting the recruitment net across the entire country 536 (or, as we envision, internationally), PONT will be ideal for running experiments with larger and 537 more diverse samples that will better represent the general population. 538 Third, while we developed PONT to facilitate our research program on subcortical 539 contributions to cognition, this protocol can be readily adopted for different populations. It can 540 be readily adopted for disorders such as Alzheimer's disease, stroke, and dyslexia which are 541 associated with support groups or online social networks. We anticipate recruitment will be more 542 challenging for any individual lab when these points of contact are absent. However, PONT can 543 provide a common test protocol to support multi-lab collaborative operations, an alternative way 544 to increase the participant pool that may be especially useful for studies aimed at more targeted 545 populations (e.g., medulloblastoma, non-fluent aphasia). As such, PONT provides a valuable 546 approach to conduct neuropsychological research to explore a wide range of brain regions and 547 neurological disorders. 548 An important benefit of PONT is that it is user-friendly for the participants. The 549 participants complete the experiment at home, choosing a time that fits into their personal 550 schedule. As we have learned from our experience over the past three decades working with 551 SCA and PD participants, arranging transportation to and from the lab can present a major 552 obstacle, especially when the travel is to participate in a study that offers no direct clinical 553 benefit to the participants. The participants also seem to enjoy the challenge of their 554 "assignment," although they, like our college-age participants may complain about the repetitive 555 nature of a task when hundreds of trials are required to obtain a robust data set. Table 2 provides 556 a sample of the comments we have received in our post-session surveys. Although we had 557 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. "There were a few that I struggled with but over-all it helped me practice my mental math. Thank you." 2 "I tried so hard to do them faster each round and just wanted to get them right which caused me to push the wrong key several times. Competing against myself was good for my brain. This old lady thanks you for the challenge." 3 "I thought this was fun. A little repetitive but enjoyable." 4 "Thanks for inviting me. Would like to know more about the study outcomes." 5 "At times I knew the correct answer and pressed the wrong finger. I feel that there was a learning curve in my responses." 6 "I needed a blowing the nose break" 7 "what's to say? my eyes got tired" 8 "I had issues with my computer -twice I inadvertently brought up an uinwanted window and had to click to get rid of it, and during the last set I lost the whole page, had to click on the link the email and it took me to where I believe I left off." 9 "Test is too long"

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) Although there are many advantages to this online approach, there are also some notable 578 limitations that need to be taken into account. First, the procedure for online experiments is 579 likely to be less standardized across participants. A major source of variability comes about 580 because participant uses his/her own computer and response device. These are, of course, 581 standardized for in-lab studies, including positioning the participants such that the visual displays 582 are near-identical. It may be possible to impose some degree of control on viewing angle by 583 specifying how the participants' position themselves, but this is likely going to be challenging 584 when the participants are self-administering the experimental task. 585 Second, online experiments are unlikely to be well-suited for all domains of study. For 586 example, given the variability in hardware, it would be difficult to run experiments that require 587 precise timing; for example, studies in which stimuli are presented for a short duration followed 588 by a mask. Moreover, we have avoided studies that use auditory stimuli since computer speakers 589 are so variable and can be of poor quality, and it is not possible use tactile or olfactory stimuli. 590 Third, the use of an automated, self-administered system comes with a cost. In our in-591 person studies, it is not uncommon for a participant to misunderstand the instructions or need 592 extra practice when they find a particular task difficult. These situations can be readily identified 593 and addressed when an experimenter is present. Moreover, a participant can get discouraged if 594 they find the task too challenging or have an error rate higher than their personal "standard." An 595 encouraging experimenter can help ensure motivation remains high. We incorporate various 596 feedback messages to keep the participants' morale and motivation high, but an online system 597 will be less flexible than in-person testing (which could be seen as a positive for maintaining test 598 uniformity). We imagine there will be some experiments that may require the virtual presence of 599 an experimenter. We have also conducted post-experiment live check-ins with some participants 600 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 28, 2021. neuroscience-of-ataxia-3556742?e=ae8c913a37). 605 Lastly, but of critical importance, is concern about the quality of the data when testing is results are encouraging in that, while the data may be noisier, the general patterns are similar. 613 We note that this is especially true with large sample sizes, something that is unlikely to be true 614 in many neuropsychological studies. One recent review provides recommendations for 615 improving data quality that are certainly appropriate for PONT (Grootswagers, 2020 The emergence of online protocol s has provided behavioral scientists with the opportunity to 621 conduct studies that involve large and diverse samples, while being cost efficient. The PONT 622 protocol described in this paper describes how this general approach can be adopted to meet the 623 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint 31 challenges associated with neuropsychological testing. Currently, our work is limited to 624 individuals with ataxia and PD, allowing us to expand our research program on subcortical 625 contributions to cognition. The materials provided with this paper can be readily adopted by 626 researchers working with any patient population, especially when recruitment can be conducted 627 via support groups, web-based groups, or through collaborations across multiple labs/clinics. In 628 terms of the latter, we see PONT as a fertile tool to support multi-national collaborative research 629 operations. We expect PONT will significantly increase the sample size, the number of studies 630 conducted, and the overall pace of neuropsychological research. As such, it offers a powerful 631 tool for this field, one that has and will continue to yield fundamental insights into brain-632 behavior relationships. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint

Big data in psychology: A framework for research 650 advancement

QRTEngine: 652 An easy solution for running online reaction time experiments using Qualtrics

Fitting linear mixed-effects 655 models using lme4

Amazon's Mechanical Turk

Effects of Divided Attention on Temporal Processing in Patients 660

Separate but equal? A comparison of participants 662 and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing

Conducting Clinical Research Using Crowdsourced 666

CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity

Experience-dependent changes in cerebellar contributions to motor sequence learning

Implicit sequence learning in people with Parkinson's disease

The organization of language and the brain

Mental health of 682 spinocerebellar ataxia patients during COVID-19 pandemic: a cross-sectional study

Snowball Sampling

A primer on running human behavioural experiments online

Cerebral Palsy: The Cerebral Palsy Research Network. Physical 690 Medicine and Rehabilitation Clinics

Discrete sequence production with and without a pause: the role of cortex, 693 basal ganglia, and cerebellum

International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity

Nature, Applications, and Limitations of Neuropsychological Assessment 696 following Traumatic Brain Injury

Toward an Instance Theory of Automatization

Cerebellum and procedural learning: Evidence from focal cerebellar lesions

Motor procedural learning in 702

October 1). The neuroanatomic and neurophysiological infrastructure for 704 speech and language

Crowdsourcing--harnessing the masses to advance health and medicine, 708 a systematic review

Movement sequencing disorders in 711 parkinson's disease

Sequential movement skill in Parkinson's disease: A state-of-the-art

CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted

Sequence learning in Parkinson's disease: Focusing on action dynamics and the 718 role of dopaminergic medication

Scale for the assessment and rating of ataxia: Development of a new 722 clinical scale

Spatial and Temporal Sequence Learning in Patients with 725

Parkinson's Disease or Cerebellar Lesions

Spatial and Temporal Sequence Learning in Patients with 728

Parkinson's Disease or Cerebellar Lesions

Collecting response times using Amazon Mechanical Turk and 731

Sequence learning is preserved in individuals with 734 cerebellar degeneration when the movements are directly cued

Sequence learning is preserved in individuals with 737 cerebellar degeneration when the movements are directly cued

Online recruitment and testing of infants 740 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted

Movement chunking during sequence learning is a dopamine-dependant process: A study 744 conducted in Parkinson's disease

Cerebellar degeneration affects cortico-cortical connectivity in motor learning networks

 644 We thank Sharon Binoy, Nandita Radhakrishnan, Rachel Woody, Sidra Seddiqee, and 645 Sravya Borra for their assistance. This research was supported by funding from the 646 National Institute of Health (NS116883).

Open Practices Statement: Materials and data are available at https://osf.io/fktn9/. 648 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint

CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.27.21257953 doi: medRxiv preprint