key: cord-0317430-kxtnfxyd authors: Schechtman, Eitan; Heilberg, Julia; Paller, Ken A. title: Made together, replayed together: Context reinstatement during sleep guides memory consolidation date: 2022-03-29 journal: bioRxiv DOI: 10.1101/2022.03.28.486140 sha: ad28d22378d3086e99ddbe70f2780674b69dd1eb doc_id: 317430 cord_uid: kxtnfxyd Newly formed memories aren’t quarantined one from another when first encoded; rather, they are interlinked with other memories that were encoded in temporal proximity or share semantic features. In this study, we selectively biased memory processing during sleep in humans to test the hypothesis that memories’ contexts influence their consolidation. Participants formed 18 idiosyncratic narratives, each linking four objects together. They then memorized random on-screen spatial positions assigned to each of the 72 objects. During sleep, 12 object-specific sounds were unobtrusively presented, thereby cuing the associated memories and improving spatial recall for those objects. As predicted, memory for uncued objects that were contextually linked with cued objects benefited as well. The correlational structure of spectral power for electrophysiological responses revealed that sleep spindles supported context reinstatement and predicted context-related memory benefits. Moreover, context-specific electrophysiological activity patterns emerged during sleep. We conclude that when an individual memory is reactivated during sleep, the corresponding encoding context is also reinstated, impacting consolidation and enhancing subsequent retrieval of associated knowledge. Individual memories are supported by an intricate network of interconnections-not independent and detached from other memories. These connections are central to the organization of memories in the brain and impact subsequent retrieval. The term "context" has been used to describe the elements that surround a core memory, which share some features with it such as time, space, or semanticrelatedness (Smith, 1994; Stark et al., 2018) . Memories that are formed within temporal proximity of others are said to share a temporal context, whereas memories that share semantic relatedness are said to share a semantic context or be semantically clustered. Both types of contexts impact subsequent retrieval (Kahana, 1996; Howard and Kahana, 2002a, b; Polyn et al., 2009; Stark et al., 2018) . When a specific memory learned in some context is retrieved (e.g., the decorations for a recent party), other contextually related memories may effortlessly come to mind as well (e.g., the guests attending the party). On the neural level, this process, termed contextual reinstatement, is manifested by increased similarity between the observed neural patterns during encoding and retrieval (Manning et al., 2011; Howard et al., 2012) . In this study, we explored contextual reinstatement when memories were reactivated during sleep, as well as the consequences of reinstatement on later retrieval. Whereas the role of context at encoding and retrieval has been repeatedly demonstrated, its role in the intermediate period of time has not been systematically explored. During these offline periods, including sleep, memory traces are cemented in cortical networks through a set of processes collectively termed consolidation (Diekelmann and Born, 2010; Paller et al., 2021) . The consolidation of declarative memories (i.e., explicit memories for facts and autobiographical events) is thought to primarily occur during non-rapid-eye-movement (NREM) sleep, which consists of the deepest stages of sleep (stages 2 and 3). Consolidation is thought to rely on memory reactivation, which, like contextual reinstatement, involves the selective activation of memory-specific neural circuits (Cairney et al., 2018; Schreiner et al., 2021) . The extent to which contextually related memories are reinstated over the course of consolidation during sleep remains unclear. Recently, it has been hypothesized that the benefits of NREM to memory stem from it being a state devoid of context, thus preventing the damaging effects of contextual interference (Yonelinas et al., 2019 ; but see Antony and Schapiro, 2019) . To explore context reinstatement during sleep, we biased memory processing during sleep using unobtrusive stimuli, a technique termed targeted memory reactivation (TMR; Oudiette and Paller, 2013) . TMR has been used to improve different forms of memory, including declarative and nondeclarative (see Hu et al., 2020 for a recent meta-analysis). In this study, participants created unique stories that each linked a place with four objects. The stories thus bound each set of objects together and served as semantic contexts. Next, participants studied the spatial positions of the objects on a 2D spatial grid. Later, during sleep, sounds related to some of the objects were presented during NREM sleep. Using this causal manipulation, we demonstrated that consolidation benefits were not limited to memories that were directly targeted, but extended to other memories that were contextually bound to them. These results suggest that context plays a role in the process of memory consolidation during sleep. By examining electrophysiological waveforms following stimulus presentation during sleep, we demonstrated that spindles-which have been linked with memory consolidation Fernandez and Luthi, 2020; Schechtman et al., 2021a )-reflect the process of contextual reinstatement during sleep and predict subsequent performance on a memory task. We recruited participants from the local university community who claimed to be able to nap in the afternoon and reported not having a hearing impairment or a history of any neurological or sleep disorders. Participants were asked to go to bed later than usual on the night prior to the study and to wake up earlier than usual on the day of the study and to avoid caffeine. In total, 48 participants were recruited (14 identified as men, 33 identified as women, and one identified as gender queer; average age = 22.6 years). Data from 19 of these participants were excluded from the final analysis (16 who were not exposed to all stimuli during NREM sleep and three with poor recall of which objects were associated with each place, as described below). Including these 19 participants was deemed inappropriate because the logic of our manipulation relied on both reactivation during sleep and strong place-object learning. In total, 29 participants were included in the final analysis (8 identified as men, 20 identified as women, and one identified as gender queer; average age = 22.8 years). The Northwestern University Institutional Review Board approved the procedure. Visual stimuli were presented on a screen (1920 × 1080 pixels, P2418HT, Dell Inc., TX). Sounds were delivered over a pair of speakers (AX-210, Dell Inc., TX). Participants' spoken responses were recorded using a Lavalier clip-on microphone (PoP voice Inc.). Stimulus presentation and participant responses were controlled by Presentation (v17.2, Neurobehavioral Systems, Inc.). Visual stimuli were used for both for the main task and for the functional localizer task. For the main task, visual stimuli consisted of 76 images of objects and 19 images of places. The object images were square and portrayed either inanimate objects (e.g., a telephone) or animals (e.g., a cat) on a white background. During the spatial task described below, they were each shown at 125 × 125 pixels (34.4 × 34.4 mm). Most images were taken from the BOSS corpus (Brodeur et al., 2010; Brodeur et al., 2014) , and some were taken from copyright-free online image databases (e.g., http://www.pixabay.com). Each object image was matched with a distinguishable, congruent sound with a maximal duration of 0.6 s (e.g., a ringing sound; a meow sound). The place images portrayed distinct real-life places (e.g., a movie theater; a desert) and were shown horizontally with a 1:2 aspect ratio. Images were taken from copyright-free online image databases (e.g., http://www.pixabay.com). Three of the 76 object images and one of 19 place images were used in a pre-task practice block. One additional object image was never displayed, but the sound associated with it was presented during sleep along with a subset of task-relevant sounds, as detailed below. The remaining 18 place images were each associated with four objects to create contextually bound sets. Object images were each assigned a random position on a 2D circular grid (radius -540 pixels, 148.5 mm). The positions of the 72 objects were set to be at least 50 pixels from the center and the perimeter of the grid and at least 55 pixels from all other object positions. The positions of each set of four objects associated with the same place were at least 425 pixels one from the other. This allowed us to separately estimate errors that stem from confusion between the positions of two objects (swap errors) and errors that stem from imprecise object placement (accuracy errors; Schechtman et al., 2021a) . For the functional localizer, a total of 120 images were used, including 40 images belonging to each of three categories: faces, places, and abstract images. All images were cropped to be square and were presented on-screen at 450 x 450 pixels. The face images were taken from the Psychological Image Collection at Stirling (pics.stir.ac.uk). The place images consisted of the same images used for the main task, cropped, and supplemented by additional images taken from the BOLD5000 database (Chang et al., 2019) . The abstract images are scrambled place images, created by scrambling the Fourier transforms of place images from the same database. (a) Upon coming to the lab, participants underwent several tasks before and after a 90-minute afternoon nap. (b) In story building, participants constructed a story including the four objects and the locale pictured. The four objects, linked together by the story, formed a contextually bound set. Participants constructed 18 such stories. (c) In position learning, each of nine blocks featured two contextually bound sets. Participants learned the on-screen positions of the eight objects in conjunction with answering questions about their stories. (d) After a memory test for object positions, participants had a 90-minute nap opportunity. During their sleep, sounds associated with 12 objects were unobtrusively presented. These objects were systematically selected from only six of the nine blocks. In these six blocks, two objects from the same set were cued. The critical conditions were thus cued objects, non-cued objects from a set with cued objects (i.e., a cued set), and non-cued objects that do not belong to a cued set (either from a cued block or an uncued block). The panel shows an example of a cued block (top) and an outline of the two types of blocks (bottom). After consenting to participate in the study, participants were fitted with an electroencephalography (EEG) cap. EEG data was collected continuously throughout all phases of the study (Figure 1a ). Since data were collected during the COVID-19 pandemic, participants wore their masks throughout the study, except during the nap portion. After entering the experimental chamber, participants completed a task to measure their response times, as described elsewhere (e.g., Schechtman et al., 2021b) and rated their sleepiness level using the Stanford Sleepiness Scale (Hoddes et al., 1973) . Next, participants conducted a functional localizer task (Supplementary Figure 1a) . The task included 150 trials, equally divided between three image categories, with an inter-trial interval ranging between 2.5 and 3.5 s. Each trial included a 1 s exposure to an image of either a place or a face, or a scrambled, abstract image. Participants were instructed to left-click the mouse when an image was repeated; 20% of the images were identical to the previous image. The first two participants run did not undergo the functional localizer task, and their data was not used for analyses incorporating data from this task. Participants next began the main task, which consisted of three parts: story-building, position-learning and test. In the story-building part (Figure 1b ), participants were instructed to build idiosyncratic stories, one for each contextually bound set (i.e., images of a place and four objects). Each set was presented together on the screen, and participants had to indicate when they have developed a story for it. Then, they recorded an audio rendition of the story. Finally, they were asked two questions about each object in each story: "Did the object appear throughout the whole story, start to end?" and "was the object in motion (not static) during the story?" These questions were chosen because they were applicable to all objects, yet the answers did not merely concern object attributes but rather required retrieving the constructed story. The answers for both questions with respect to all four objects were recorded before moving on to develop a story for the next contextually bound set. After completing this part of the task, participants started the position-learning part ( Figure 1c ). This part consisted of nine blocks, each including a pair of contextually bound sets and eight objects in total. Set pairings and block allocation were randomized. In this part of the task, participants had to encode the on-screen positions for the objects on a two-dimensional on-screen circular grid. Once all object positions were learned, as defined below, the next block commenced. At the start of each block, participants viewed the two places featured in the block and were given the option to listen to their recordings of the associated stories to refresh their memories. Before the first block commenced, participants engaged in a practice block which included four objects that were designated as practice objects. At the start of each block, the eight objects were presented in their positions sequentially. Each object was presented for 4.5 s, with its congruent sound presented twice, once at trial onset and again at the end of the trial (with sound offset synchronized to object offset). A 1-s inter-trial interval followed. After being exposed to the object positions, participants trained on placing them in the true positions. Each trial included a single object, and trials were presented in a pseudo-random order, such that objects linked to the same story were seldom presented sequentially. At the start of each trial, an object-specific contextual question was presented. These questions were the same ones presented in the story-building part. Participants had to get each question correct to proceed with the trial; answering incorrectly terminated the trial (Figure 1c , bottom). The purpose of presenting these questions was to repeatedly reinstate the encoding context during the positionlearning part of the task. Overall, 88.92% ± 0.9% (mean ± SEM) of questions were answered correctly during training. Next, the object was presented at a random position on the grid, along with its associated sound. Participants used the computer mouse to attempt to drag the object to the position where it was initially seen on the grid. Trials with a Euclidean error of less than 100 pixels relative to the true position were considered correct. After correctly placing an object near its true position twice in a row, it was considered as learned and was dropped out from the block. On average, each object was presented in 3.52 ± 0.15 trials (mean ± SEM). Both correct and incorrect trials included feedback: the true object position was presented for 2 s along with the user-selected position. The sound was then presented again, co-terminating with the feedback display. A 1-s inter-trial interval followed. The last part of the main task including a test on object positions. In each trial, participants had to drag one of the objects to its true position. All 72 objects were presented in a pseudorandom order, preceded by the three practice objects. Objects were each presented in a random position on the grid, accompanied by their sounds. No context-related questions were presented, nor was feedback given. A 1-s inter-trial interval was used. Following the test, participants' pre-sleep error rates were calculated (i.e., the Euclidean distance between the chosen and true on-screen positions). Out of a total of 72 objects, 12 were designated to be cued during sleep. These objects were selected in a manner that obeyed the following logic: Six of the nine blocks included cued objects. Each of these blocks consisted of two contextually bound sets: one including cued objects; the other not. The six contextually bound sets which included cued objects each included two cued objects and two non-cued objects. Out of the nine blocks, the remaining three blocks did not include any cued objects. The condition designated to each object, set, and block were determined using an algorithm that minimized variability between the average error rates among conditions. In addition to the 12 object-related sounds designated for cuing, another sound which was not used during the wake portions of the task was presented during sleep as a control sound. Immediately after ending the test, participants were permitted to nap for 90 minutes with the lights out on a foldable futon in the same experimental room. Throughout their nap, white noise was presented (~47 dB). Sleep was monitored online by an experimenter skilled at sleep staging. Upon detection of stage 3 of NREM sleep, sounds were unobtrusively presented in the experimental room (<53 dB). EEG data was monitored continuously and sound presentation was terminated immediately upon signs of arousal or transition to REM sleep. The inter-stimulus interval (i.e., offset-to-onset) was randomly set to either 6, 6.5, or 7 s. If the participants did not reach NREM stage 3 after 45 minutes, sounds were presented in either stage 2 or 3 throughout the remainder of the nap. After the nap, participants were required to wait at least five minutes before resuming the task. After once again completing a task to measure their response times and after rating their sleepiness level, participants started the post-nap test, which was identical to the pre-nap test. Then, participants completed a self-paced recall test, in which they had to type in, for each picture of a place, which objects were linked with it. This part of the task was used as a manipulation check, since the expected effects of TMR critically depended on a strong, over-trained link between objects, stories, and places. Three participants who failed to recall at least 75% of the objects were excluded from analysis. Finally, participants were asked if they heard sounds presented during the nap. Out of the 29 participants used for analyses, seven reported hearing task-related sounds. These seven participants then underwent a task in which they were required to indicate which sounds they remember hearing during sleep. Their responses indicated that they were not significantly different than chance at identifying which sounds were presented (p = 0.8, Sign Rank Test). Participants were then allowed to clean up, after which they were dismissed. EEG was recorded using Ag/AgCl active electrodes (Biosemi ActiveTwo, Amsterdam). In addition to the 64 electrodes at 10-20 system scalp locations, contacts were placed on the mastoids, next to the eyes, and on the chin. Recordings were made at a sampling rate of 512 Hz. Analyses were conducted using the FieldTrip (Oostenveld et al., 2011) and sleepSMG (http://sleepsmg.sourceforge.net) packages for Matlab 2018b (MathWorks Inc, Natick, MA). EEG channels were re-referenced offline to averaged mastoids and filtered using a two-way least-squares FIR highpass filter with a cutoff of 0.4 Hz. Additionally, a notch filter was used to remove noise at 60 Hz. Noisy channels were replaced with interpolated data from neighboring electrodes using the spherical interpolation method in FieldTrip, and noisy segments were detected manually and removed from further analyses. For the data collected during wake, ICA was used to detect and remove artifacts associated with eye blinks and horizontal eye movements. Sleep staging (i.e., determining the stage of sleep for each 30-s epoch) was based on the guidelines published by the American Academy of Sleep Medicine (Iber et al., 2007) and conducted by two independent raters, both of whom were not privy to when sounds were presented. Any discrepancies were subsequently reconciled by one of the two raters. Supplementary Table 1 shows the amount of time spent in each stage of sleep and number and percentage of cues presented in each stage. For each trial in the tests conducted before and after sleep, the error was measured in pixels as the Euclidean distance between the true object's position and the position indicated by the participant. For each trial, objects were flagged as "swapped" if they were placed closer to the position of another object belonging to the same contextually bound set. These gross errors, stemming from either pure guesses or confusion between objects, were omitted from further analysis (Schechtman et al., 2021a) . Error rates for all the remaining trials were Z-scored within participants and used to evaluate the effects of cuing during sleep across participants using the following mixed linear model (fitglme function in Matlab): For both errors, Z-scores were used. "Participant" is a categorical variable, denoting the participant number of each individual participant. Two different analyses were run, one focusing only on semantic context and one considering semantic and temporal context separately. For the former, "Condition" was a categorical variable with three possible values: (1) cued object ∈ cued set; (2) non-cued object ∈ cued set; (3) non-cued object ∉ cued set (Figure 1d ). For the latter, "Condition" was a categorical variable with four possible values: (1) cued object ∈ cued set; (2) non-cued object ∈ cued set; (3) (non-cued object ∉ cued set) ∈ cued block; (4) non-cued object ∉ cued block (Figure 2d ). An ANOVA was used to report the statistical significance of the components of the model, and dummy variables were used for comparisons between conditions. In order to quantify the extent of the cuing effect on cued and non-cued objects within a contextually bound set for each participant (Figure 3e, Figure 4e ), a similar model was run for each participant, omitting the random effects. The obtained coefficients for each condition [i.e., condition-specific forgetting slopes relative to the forgetting slope of the (non-cued object ∉ cued set) group] were then used to calculate correlations between behavior and physiology across participants, as outlined below. Spectral analyses were run on data collected during sleep from electrode Cz and limited to epochs for task-related sounds presented during NREM sleep. Trials were segmented around sound onset (1.25 s before to 4.75 s after). For each trial, we first subtracted its overall mean and then calculated a spectrogram between 0.25 Hz and 25 Hz in 0.25 Hz intervals, using 0.5-s time windows with 87.5% overlap. For each participant, the average baseline (i.e., t < 0 s) activity was calculated per frequency band, and each trial's spectrogram was converted to percent change by subtracting and dividing the activity during baseline for each frequency band. These trial-specific spectrograms were used to extract power in specific time-frequency clusters on the single trial level, as detailed below. To identify significant clusters of sound-related activity, we first averaged the trial-specific spectrograms within participant. Then, each point in the time-frequency representation was compared to zero, with an alpha level of 0.01 (corrected for the number of data-points). The results, shown in Figure 3b , indicated two significant clusters of activity. These clusters were used for further analyses. The higherfrequency cluster, reflecting activity in the sigma range (see Figure 3b ), putatively encapsulates sleep spindle activity which commonly commences approximately 1 s after sound onset (e.g., Cairney et al., 2018; Schechtman et al., 2021a) . We therefore considered both the full cluster, as well as the two putatively separate component comprising of it, in our analyses. For each trial, we extracted and summated the values confined by each cluster from the spectrogram, resulting in a single scalar value per trial and cluster. We hypothesized that trials involving the same sounds within the same participants would have correlated power in certain clusters. To test this, we used intraclass correlation (Koo and Li, 2016) . This metric, ICC, is symmetrical (i.e., whereas interclass correlations predict Y from X, ICCs predict how clustered together different values of X are) and can be used to calculate the correlation between more than two values. We calculated ICCs for each participant and cluster, and then ran a permutation test with mixed labels (n = 10,000) for each participant. Finally, we conducted a paired t-test between the true ICC and the average ICC value calculated using the permutation test. To ensure that we had sufficient data and to avoid biases due to a small number of trials, ICC analyses on spectral data were limited to participants who had at least five presentations for each sound (N = 16). Next, we hypothesized that contextually bound memories would elicit correlated spectral activity. We leveraged the fact that for each contextually bound set, two sounds were presented. We averaged the power per sound and per cluster, producing two values for each contextually bound set within participants. The ICC was then calculated within set for each cluster. Like before, we used a permutation test (n = 10,000) and paired t-tests to consider evidence in support of our hypothesis. Finally, the true ICC values obtained for each participant and cluster were then correlated with the forgetting slopes calculated per participant. Using the data collected in the functional localizer task, a classifier was trained for each participant to distinguish faces, places, and abstract images (Supplementary Figure 1b) . For each image, trials that were not contaminated with artifacts were segmented between 1.5 s before and 3.5 s after image onset. Classification was calculated for each time point independently using time-series data from the 64 scalp electrodes as features. Data was smoothed using a 51.2-ms rectangular smoothing window. This data was used to train a support vector machine (SVM) classifier and tested using 5-fold crossvalidation protocol (Bae and Luck, 2018) . This procedure was repeated for 20 iterations and averaged across iterations. To calculate the significance level, clusters of contiguous time points that were significantly higher than chance (p < 0.001) were identified, and each time point's t-values were summed together to produce a cluster-level t mass. Then, a permutation test was initiated by reconducting the classification analysis and identifying significant clusters using shuffled labels. Significance for each true cluster-level t mass was evaluated relative to the random distribution of clusters generated based on 100,000 permutations, with an alpha of 0.001. Using the Functional Localizer data and the data from the sleep phase, we correlated (1) place-imagerelated and abstract-image-related wake EEG patterns with (2) patterns observed following sound presentation during sleep. First, time-series data were segmented around both the wake and sleep trial onsets, starting 1.5 s before and ending 3.5 s after stimulus onset. Trials containing artifacts were omitted. Only sleep trials that included task-related sounds were considered. Data was smoothed using a 51.2 ms smoothing window. Data from the 64 scalp electrodes at each time point during wake was correlated with data from the same electrodes at each time point during sleep, resulting in a time X time correlation matrix showing wake-sleep correlations across time points. Two matrices were created, one correlating place-image-related wake EEG patterns with sleep-related EEG patterns, and one correlating abstract-image-related wake EEG patterns with sleep-related EEG patterns. The subtraction between the two was used to assess evidence for place-specific activation patterns. The difference matrix was collapsed over the sleep-time axis to create a vector of correlation coefficients over the course of the wake trial, and the time period during which wake classification was significantly above chance was extracted. These values, across participants, were then correlated with the forgetting slopes calculated per participant. Participants (N=29) engaged in a single-session experiment that included an afternoon nap (Figure 1a ). They first invented idiosyncratic stories involving a place (e.g., a movie theater) and four objects ( Figure 1b) . Next, they encoded the positions of objects on a 2-d grid (Figure 1c) . To keep the encoding context (i.e., the story) salient, this training also involved answering story-specific questions that required contextual reinstatement. Spatial recall was then tested for all object positions. The average positioning error, calculated as the Euclidean distance between the true position and placed position, was 76.7 ± 1.9 (mean ± SEM) pixels (see Methods). Next, participants were allowed to nap for up to 90 minutes. During NREM sleep, participants were exposed to sounds that had been associated with some of the objects. Finally, participants completed another recall test. We hypothesized that biasing reactivation for certain memories would impact other memories that belonged to the same contextually bound set. To test this, we first divided objects for each participant into three groups (Figure 1d) : (1) objects that were directly reactivated using sounds during sleep (cued objects ∈ cued set); (2) objects that were not directly reactivated, but were contextually linked with objects belonging to the previous group (non-cued objects ∈ cued set); (3) objects that were neither directly reactivated nor shared an encoding context with those that were (non-cued objects ∉ cued set) (Figure 1d, bottom) . We hypothesized that the second group would show less forgetting during sleep relative to the third group. To test this hypothesis, we fitted a mixed linear model predicting how postsleep positioning errors would be modulated by experimental conditions (i.e., the three groups of objects), while accounting for pre-sleep positioning errors (Figure 2a, b) . As expected, pre-sleep errors were positively correlated with post-sleep errors (F(1,1537) = 258.8, p < 0.001). Although we did not find a main effect of condition (F(2,1537) = 0.28, p = 0.75), we found a significant interaction between condition and pre-sleep errors (F(2,1537) = 7.1, p < 0.001). This interaction reflects differences between conditions in forgetting slopes. Higher coefficients reflect steeper slopes between pre-and post-sleep errors (i.e., more forgetting). We therefore compared the slopes for the three conditions to unpack the interaction effect (Figure 2c ). Directly reactivated memories (i.e., cued objects ∈ cued set) showed smaller forgetting slopes relative to memories that were neither reactivated nor contextually bound to reactivated memories (i.e., non-cued objects ∉ cued set; t(1537) = -3.1, p < 0.01). Crucially, non-cued objects that were contextually bound with cued objects (i.e., non-cued objects ∈ cued set) showed smaller forgetting slopes relative to memories that were neither reactivated nor contextually bound to reactivated memories (i.e., non-cued objects ∉ cued set; t(1537) = -2.2, p < 0.05). Taken together, these results indicate that the benefits of memory reactivation during sleep extend beyond targeted memories, impacting other memories that were contextually linked with them. Using the same dataset, we next examined the differential roles of semantic context (i.e., conceptual links between objects, operationalized by the idiosyncratic story for each set) and temporal context (i.e., links between objects learned within temporal proximity, operationalized by our block design; Figure 1c ). Temporal encoding context has been shown to reinstate during wake, thereby impacting retrieval (Manning et al., 2011; Bornstein and Norman, 2017; Hoskin et al., 2019) , and we hypothesized that it would similarly be reinstated and impact reactivation during sleep. However, our results did not support this hypothesis. For this analysis, we divided our data into four groups (Figure 2d ). The first two were identical to those used for the previous analysis (cued objects ∈ cued set; non-cued objects ∈ cued set). Another group included objects that were not linked with a cued set, but were learned within the same block as cued objects [(non-cued objects ∉ cued set) ∈ cued block]. The final group included objects that were neither learned in the same block nor linked with the same set as the cued objects (non-cued objects ∉ cued block). The crucial difference between these last two groups is that objects in the former group -but not the latter group -share a temporal context with the cued objects. Like before, we found main effects of pre-sleep errors (F(1,1535) = 150.2, p < 0.001) but not condition (F(3,1535) = 0.2, p = 0.9), as well as a significant interaction (F(3,1535) = 4.6, p < 0.01; Figure 2e ). Contrasting the forgetting slopes for the three groups revealed no significant difference between the [(non-cued objects ∉ cued set) ∈ cued block] and (non-cued objects ∉ cued block) groups, suggesting that temporal context, as operationalized by learning block, had no effect on performance (t(1535) = 0.03, p = 0.98; Figure 2f ). We found significantly higher slopes for the (cued objects ∈ cued set) group relative to the (non-cued objects ∉ cued block) group (t(1535) = -2.89, p < 0.01), and a trend toward higher slopes for the (noncued objects ∈ cued set) group relative to the (non-cued objects ∉ cued block) group (t(1535) = -1.83, p = 0.07). Taken together, results suggest that semantic context -but not temporal context -is reinstated during sleep and guides memory consolidation. Each dot represents error rates for objects positioned by the participant before and after sleep. Colors signify the condition for each object: cued objects ∈ cued set (yellow), non-cued objects ∈ cued set (red), noncued objects ∉ cued set (blue). Steeper slopes signify more forgetting between the pre-and postsleep tests. (b) Forgetting curves across all participants. Slopes and intersects were estimated using a mixed linear model. (c) Statistical comparison of forgetting slopes across conditions. The same data is reflected in panels b and c. (d) The same dataset was submitted to another analysis, which distinguishes between two groups of objects: [(non-cued objects ∉ cued set) ∈ cued block] and (noncued objects ∉ cued block). For conciseness, the former group is designated as (non-cued objects ∈ cued block). Conventions follow Figure 1d , bottom. (e) Forgetting curves across all participants for the four conditions. (f) Statistical comparison of forgetting slopes across conditions. Error bars signify standard errors of the mean. ** -p < 0.01; * -p < 0.05; ~ -p < 0.1; n.s -p > 0.1. Next, we explored the role of sleep EEG oscillations in the process of contextual reinstatement during sleep. First, we calculated the time-locked time-frequency response to sounds presented during sleep for all task-related sounds (Figure 3a) . Across participants, we identified two clusters in the timefrequency representation following sound onset (p < 0.01, corrected): one cluster at lower frequencies (<10 Hz) peaking around 0.5 s after sound onset, and another between 15-20 Hz, which involved two components peaking before and after the 1 s mark after sound onset (Figure 3b , left). The first cluster (i.e., cluster 1; Figure 3b , right), consisting of frequencies in the delta and theta range, putatively reflects activity related to slow oscillations and K-complexes, which are typical of NREM sleep. The second cluster (i.e., cluster 2), consisting of frequencies in the sigma range, may reflect the occurrence of sleep spindles, a sleep-specific waveform which has been linked to memory consolidation during sleep Fernandez and Luthi, 2020) . However, previous research has shown that spindles typically commence approximately 1 s after sound onset (Antony et al., 2018; Cairney et al., 2018; Schechtman et al., 2021a) . Together with the conjoined pattern of the observed cluster, this may suggest that only the late component of the cluster (cluster 2B) truly reflects spindle activity, whereas the early component (cluster 2A) reflects some high-frequency component of the K-complex (Donoghue et al., 2021) . Therefore, we conducted further analyses on the full high-frequency cluster, as well as on its two separable components. Throughout NREM sleep, sounds were often presented multiple times (Supplementary Table 1 ). We hypothesized that waveforms resulting from repeated presentations of the same sound would be more similar one to the other than those resulting from different sounds. To test this, we quantified power in each cluster on a trial-by-trial basis and submitting the results, per participant, to an intraclass correlation coefficient analysis (Figure 3c ). For most clusters, results indeed indicated that coefficients were higher than those expected by chance, as assessed using a permutation test (cluster 1, p < 0.001; cluster 2, p < 0.01; cluster 2A, p < 0.01; cluster 2B, p = 0.15 ; see Figure 3b , right, for designations). These correlations may be the result of the memory content related to a sound being reactivated similarly across trials, but a more parsimonious explanation may be that they stem from the acoustic properties of the presented sounds creating similar electrophysiological responses. We then used a similar approach to test whether specific waveforms reflected contextual relationships between objects within the same set. We hypothesized that responses would be more similar across sounds if these sounds were linked within the same contextually bound set. Put differently, we predicted that if a specific waveform is involved in the process of contextual reinstatement, spectral power linked with that waveform would be more similar for two sounds that share a context and less similar for two sounds that do not share a context. To test this, we calculated the intraclass correlation coefficient across sounds for each cluster and each participant (Figure 3d ). Results showed higher-thanchance coefficients for cluster 2 (p < 0.05), and specifically for cluster 2B, putatively reflecting post-sound spindles (p < 0.001). Unlike the previous analysis conducted within sounds, this analysis considered similarity between sounds and therefore does not reflect trivial sources of correlation, such as the acoustic properties of sounds. These results suggest that sleep spindles may be involved in the process of context reinstatement during sleep. Finally, we explored a more direct connection between within-context spindle correlations and the aforementioned behavioral effects depicted in Figure 2c . For each participant, we calculated (a) the TMR-induced changes in forgetting slopes for cued objects and for non-cued objects within cued sets; and (b) the intraclass correlation coefficients between sounds belonging to the same contextually bound set. We then correlated these measures across participants and found that participants with more similar sigma power within sets also showed less forgetting for cued objects (r = -0.71, p < 0.01 for cluster 2; r = -0.62, p < 0.01 for cluster 2B; p > 0.18 for all other clusters; Figure 3e , left). The correlation for non-cued objects, which reflects the contextual effects of cuing on performance, was significant only for the late sigma cluster, putatively reflecting sleep spindles (r = -0.5, p < 0.05 for cluster 2B, p > 0.36 for all other clusters; Figure 3e , right). Taken together, results demonstrate that post-cue spindle-band power is correlated within set and this correlation predicts contextually determined benefits. Our findings therefore support the hypothesis that power in the spindle band reflects contextual reinstatement during sleep. Correlations between intraclass correlation coefficients and conditionspecific changes in forgetting slopes across participants. Left -for cued objects ∈ cued sets. Rightfor non-cued objects ∈ cued sets. Error bars and dashed lines signify standard errors of the mean. *** -p < 0.001; ** -p < 0.01; * -p < 0.05; ~ -p < 0.1; n.s -p > 0.1. Previous studies operationalized contextual reinstatement during wake using measures of similarity between brain states during encoding and retrieval (Manning et al., 2011; e.g., Howard et al., 2012) . To test whether place-specific wake-like activity is involved in sleep reactivation, we had participants observe consecutive images belonging to one of three categories: places, faces, or abstract images (Supplementary Figure 1) . Using time-series data from all scalp electrodes to train a support-vector machine classifier, we identified clusters of time-points distinguishing places and abstract images (p < 0.001). The largest cluster spanned between 0.28 and 0.73 s after image onset during wake (Figure 4a ). Using place-related activity as a marker for context, we next correlated the sleep and wake EEG data from all scalp electrodes to reveal reinstatement of context-related activity. Averaging across wake trials (i.e., around image onset for place images in the functional localizer) and sleep trials (i.e., around sound onset for sleep-related sounds), we calculated the time-point-by-time-point correlation matrix. The result was a time X time matrix of correlation coefficients, with a peak in correlation around 0.5 s after place image onset during wake and 0.75 s after sound onset during sleep (Figure 4b, top) . This increase in correlation may reflect genuine neural reactivation of place-related representations. However, it could be that the correlations are not place-specific, but rather reflect similarities between sleep and image-viewing during wake. Indeed, conducting the same correlation analysis between task-related sound presentation data and EEG data following the presentation of an abstract image revealed similar temporal dynamics (Figure 4b , bottom). We defined place-specific reactivation as the subtraction of the place and abstract related matrices ( Figure 4c ). The resulting matrix showed a peak in correlation around 0.6 s after place image onset during wake and 0.75 s after sound onset during sleep. Although correlations peaked following sound onset, we found that they persisted throughout the time course of reactivation during sleep (e.g., correlations are above baseline even before sound onset). This extended period of place-specific reactivation is likely due to the protocol used for cuing, which involved continuous, repetitive presentations of task-related sounds, all of which were part of contextually bound sets involving places. Therefore, we collapsed the place-specific reactivation matrix over the time course of sleep trials ( Figure 4d ). To reveal whether place-specific reactivation predicts changes in performance, we focused on the period of time during which the classifier distinguished between places and abstract images (Figure 4a ; gray dashed frame in Figure 4d ). Average correlations during this time period were correlated with changes in performance for cued objects ∈ cued sets, indicating less forgetting for cued objects in participants who demonstrated higher wake-sleep place-specific correlations (r = -0.39, p < 0.05; Figure 4e , left). However, reactivation patterns did not significantly correlate with non-cued objects ∈ cued sets (r = -0.05, p = 0.82; Figure 4e , right). Taken together, results show that place-specific neural representations are reinstated by cues during sleep and this reinstatement predicts benefits for cued memories. Figure 4: Scene-specific neural reactivation predicts changes in performance for cued objects. (a) Classification accuracy for a classifier trained to distinguish images of places and abstract images. Gray lines denote significant classification above chance (p < 0.001). Gray dashed boxes in the following panels signify the time period for the largest cluster of continuously significant classification. (b) Correlation patterns between EEG patterns following sound presentation during sleep (y-axis) and image presentation during wake (x-axis). Top -correlations for images of places. Bottomcorrelations for abstract images. (c) The place-specific correlation patterns, defined as the subtraction between the upper and lower matrices shown in panel b. Note that place-related activity starts before sound onset during sleep and persists throughout most of the sound-locked time-course. (d) Place-specific correlation patterns collapsed over the sound-locked time-course during sleep. (e) Correlations between the place-specific patterns during the classifiable time period (gray dashed box in panel d) and changes in forgetting slopes across participants. Left -for cued objects ∈ cued sets. Right -for non-cued objects ∈ cued sets. Shaded areas and dashed lines signify standard errors of the mean. * -p < 0.05; n.s -p > 0.1. In this study, we tested whether the context in which memories are encoded impacts sleep-related consolidation. Participants first developed idiosyncratic stories linking different objects with a physical place, and then encoded on-screen positions for each object. By presenting object-specific sounds during NREM sleep, we selectively biased reactivation towards specific memories, thereby reducing forgetting for these memories. Crucially, this manipulation also reduced forgetting for contextually bound memories that were not cued directly. Spectral analyses revealed that EEG responses likely reflecting post-sound sleep spindles were correlated when sounds were linked to contextually bound objects. Furthermore, these correlations within contextually bound sets predicted changes in retrieval performance: with greater similarity there was less forgetting for both cued and non-cued objects within a set. Finally, our analyses revealed that neural representations related to places were reinstated following sound onset during sleep, and this reinstatement predicted changes in retrieval performance for cued objects. Our results reveal that context is reinstated during sleep in a manner that impacts memory processing. From a broader perspective, these findings fit with the growing literature linking context and memory. The notion of context-dependent memory pertains to improved retrieval in a context similar to the encoding context (Abernethy, 1940; Godden and Baddeley, 1975) . The natural process of autobiographical retrieval involves the experience of mentally travelling back in time (Tulving, 1983; Tulving, 1993) , which in itself involves contextual reinstatement (Howard and Kahana, 2002b) , thereby improving retrieval by increasing the similarity between the neurocognitive states at encoding and retrieval. Contextual reinstatement through mental time travel results in the effortless retrieval of multiple intertwined memories that reside together in the same rich context. Despite much theoretical and empirical research on how context bridges memory encoding and retrieval, the question of context's role is memory consolidation during sleep has been scarcely addressed. Some evidence suggests that sleep serves to strengthen the links between memories and the contexts in which they are encoded (van der Helm et al., 2011; Kurinec et al., 2021) . Indeed, consolidation over time seems to increase the similarity between neural representations of memories linked with the same context (Tompary and Davachi, 2017) , although the specific role of sleep is yet to be explored. The question of whether contexts are reinstated during sleep is separate from the question of sleep's effects on context-item binding. Some consolidation theories assume that sleep is a context-less state, and this property of sleep shelters memories by preventing context-related interference (Yonelinas et al., 2019) . However, to the best of our knowledge, the question of context reinstatement during sleep has not been systematically and empirically explored. Recently, we showed that the capacity for reactivation and consolidation during sleep is not limited to a single memory at any given time, suggesting that memories that are tightly and conceptually interlinked can be reactivated simultaneously (Schechtman et al., 2021a) . Our current results suggest that this capacity for simultaneous reactivation extends beyond tightly interlinked memories (e.g., different memories related to cats) and also applied to contextually interlinked memories (e.g., memories that reside within the same narrative). These findings complement recent studies that have found that reactivating memories during wake has a retroactive beneficial impact on conceptually related memories that were not directly reactivated (Dunsmoor et al., 2015; Patil et al., 2017) . Taken together, these findings show that consolidation during both wake and sleep involves contextual reinstatement and impacts memories that were not directly cued. Our design attempted to tease apart two interlinked forms of context -semantic context and temporal context. Results revealed that semantic context, operationalized using an idiosyncratically constructed narrative, impacted sleep-related effects on memory, whereas temporal context, operationalized using a temporally structured block design, did not ( Figure 2 ). However, we acknowledge some limitations in our design that warrant a more nuanced interpretation of these results. First, it should be noted that the contextually bound sets, which were used to operationalize semantic context, in fact also share a temporal context; they not only shared a narrative, but were also learned in the same learning block. Second, temporal context is not impacted exclusively by the mere passage of time. Instead, salient events act as event boundaries, creating abrupt shifts in temporal context (Zacks et al., 2007; Clewett et al., 2019; Pu et al., 2022) . It could be argued, therefore, that blocks in the position learning part of the task did not uniformly reflect temporal context, but rather that switches between trials within a block acted as event boundaries. Effectively, this framing, if correct, would mean that memories in our task never shared temporal context, providing an alternative explanation for our null findings. Despite the aforementioned limitations, the possible distinction between the roles of semantic and temporal context during sleep aligns with other experimental findings. In a recent study, we used the same task design to explore the role of context in undisturbed, overnight sleep (Schechtman et al., In Preparation) . By considering the similarities between memory trajectories over a delay period within the same contextually bound sets, we found that semantic -but not temporal -context drives performance changes over a 10-hour period including sleep, but not over a 10-hour period that did not include sleep. In a separate study, Liu and Ranganath (2021) found the sleep is crucial for binding together memories that are semantically related but learned in different episodes. In contrast, they found that sleep did not impact binding between memories that were semantically unrelated but learned within temporal proximity one of the other. Taken together, these results suggest that there may be a qualitative difference between the roles semantic and temporal contexts play in memory processing during sleep. Over the last decades, sleep's active role in memory consolidation has gradually been revealed and acknowledged. Our understanding of how memory representations are reactivated and evolve during sleep is still incomplete. Day-to-day memories are best understood when considering the connections amongst them, yet these connections are not accounted for in our models of memory processing during sleep. Our demonstration of a role for context in sleep consolidation opens the door for further exploration of how memory interconnections impact consolidation during sleep. More generally, this study underscores the notion that memory processing orchestrated by the sleeping brain is as rich and complex as when we are awake. The Effect of Changed Environmental Conditions Upon the Results of College Examinations Active and effective replay: systems consolidation reconsidered again Sleep Spindles and Memory Reprocessing Sleep Spindle Refractoriness Segregates Periods of Memory Reactivation Dissociable Decoding of Spatial Attention and Working Memory from EEG Oscillations and Sustained Potentials Reinstated episodic context guides sampling-based decisions for reward Bank of Standardized Stimuli (BOSS) Phase II: 930 New Normative Photos The Bank of Standardized Stimuli (BOSS), a new set of 480 normative photos of objects to be used as visual stimuli in cognitive research Memory Consolidation Is Linked to Spindle-Mediated Information Processing during Sleep BOLD5000, a public fMRI dataset while viewing 5000 visual images Transcending time in the brain: How event memories are constructed from experience The memory function of sleep Methodological considerations for studying neural oscillations. The European journal of neuroscience Emotional learning selectively and retroactively strengthens memories for related events Sleep Spindles: Mechanisms and Functions Context-Dependent Memory in Two Natural Environments: On Land and Underwater Quantification of Sleepiness: A New Approach Refresh my memory: Episodic memory reinstatements intrude on working memory maintenance When Does Semantic Similarity Help Episodic Retrieval? A Distributed Representation of Temporal Context Ensembles of human MTL neurons "jump back in time" in response to a repeated stimulus Promoting memory consolidation during sleep: A meta-analysis of targeted memory reactivation AASM manual for scoring sleep. • Kahana MJ (1996) Associative retrieval processes in free recall A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research Sleep Deprivation Impairs Binding of Information with Its Context. Sleep Resurrected memories: Sleep-dependent memory consolidation saves memories from competition induced by retrieval practice. Psychonomic bulletin & review Oscillatory patterns in temporal lobe reveal context reinstatement during memory search FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data Upgrading the sleeping brain with targeted memory reactivation Memory and Sleep: How Sleep Cognition Can Change the Waking Mind for the Better Reward retroactively enhances memory consolidation for related items A context maintenance and retrieval model of organizational processes in free recall Event boundaries shape temporal organization of memory by resetting temporal context In Preparation) Context matters: Changes in memory over a period of sleep are driven by context Multiple memories can be simultaneously reactivated during sleep as effectively as a single memory Sleep reactivation did not boost suppression-induced forgetting Endogenous memory reactivation during sleep in humans is clocked by slow oscillation-spindle complexes Theoretical principles of context-dependent memory What's in a context? Cautions, limitations, and potential paths forward Consolidation Promotes the Emergence of Representational Overlap in the Hippocampus and Medial Prefrontal Cortex Elements of episodic memory What Is Episodic Memory? Sleep-dependent facilitation of episodic memory details A contextual binding theory of episodic memory: systems consolidation reconsidered Event perception: a mind-brain perspective Figure 1 : Functional localizer task and classification during wake. (a) Participants were presented with images and required to indicate when two successive images were identical (i.e., a 1-back task). (b) EEG data collected during the task was used to train an SVM classifier to distinguish between image categories. For all four comparisons, classification was significantly higher than chance. Yellow shades indicate the standard errors of measurement between participants. Blue lines indicate classification significantly above chance (p < 0.001).