key: cord-102595-3lbrfsrh
authors: Adam, Kirsten C.S.; Chang, Lillian; Rangan, Nicole; Serences, John T.
title: Steady-state visually evoked potentials and feature-based attention: Pre-registered null results and a focused review of methodological considerations
date: 2020-10-13
journal: bioRxiv
DOI: 10.1101/2020.08.31.275602
sha: 
doc_id: 102595
cord_uid: 3lbrfsrh

Feature-based attention is the ability to selectively attend to a particular feature (e.g., attend to red but not green items while looking for the ketchup bottle in your refrigerator), and steady-state visually evoked potentials (SSVEPs) measured from the human electroencephalogram (EEG) signal have been used to track the neural deployment of feature-based attention. Although many published studies suggest that we can use trial-by-trial cues to enhance relevant feature information (i.e., greater SSVEP response to the cued color), there is ongoing debate about whether participants may likewise use trial-bytrial cues to voluntarily ignore a particular feature. Here, we report the results of a preregistered study in which participants either were cued to attend or to ignore a color. Counter to prior work, we found no attention-related modulation of the SSVEP response in either cue condition. However, positive control analyses revealed that participants paid some degree of attention to the cued color (i.e., we observed a greater P300 component to targets in the attended versus the unattended color). In light of these unexpected null results, we conducted a focused review of methodological considerations for studies of feature-based attention using SSVEPs. In the review, we quantify potentially important stimulus parameters that have been used in the past (e.g., stimulation frequency; trial counts) and we discuss the potential importance of these and other task factors (e.g., feature-based priming) for SSVEP studies.

Attending to a specific feature leads to systematic changes in the firing rates of 76 neurons that encode the relevant feature space. For example, when looking for a ripe 77 tomato, the firing rate of neurons tuned to red will be enhanced and the firing rate of 78 neurons tuned to other colors will be suppressed (e.g. responses to green; Ipata et al., 79 2006; Martinez-Trujillo & Treue, 2004; Störmer & Alvarez, 2014) . Although there is broad 80 agreement that participants may learn to suppress irrelevant distractors with sufficient 81 experience, there is disagreement about whether these behavioral suppression effects 82 may be volitionally implemented on a trial-by-trial basis in response to an abstract cue 83 (i.e., a "volitional account"), or if they instead are solely implemented via implicit or 84 statistical learning mechanisms (i.e., a "priming-based" account). Consistent with a 85 volitional account, some work has found that participants can learn to use a trial-by-trial 86 cue to ignore a particular color ( suppress a color on a trial-by-trial basis independent of target enhancement (i.e., a strong 135 version of a volitional suppression account), we predicted that the time-course of 136 enhancement vs. suppression of the SSVEP signal would be reduced or reversed (i.e., 137 that suppression of the cued, to-be-ignored color may happen even prior to enhancement 138 of the other color). Alternatively, if participants recode the "ignore" cue to serve as an 139

indirect "attend" cue (e.g., "Since I'm cued to ignore blue, that means I should attend red"; 140 Beck & Hollingworth, 2015; Becker et al., 2015) , then we predicted that target 141 enhancement would always precede distractor suppression regardless of whether 142 participants were cued to attend or ignore a particular color. 143

To preview the results, we were unable to fully test our hypotheses about the time-144 course of feature-based enhancement and suppression because we did not find evidence 145 for an overall attention effect with our task procedures. Despite robust SSVEP amplitude 146 (Cohen's d > 5), we observed no credible evidence that the SSVEP response was higher 147 for an attended versus unattended color in either cue condition. Positive control analyses 148 revealed that our lack of SSVEP effect was not due to a complete lack of attention to the 149 attended color: ERP responses (P3) to the targets were modulated by attention as light of our inconclusive results, we also performed a focused methodological review of 152 key potential task differences between our work and prior work that may have resulted in 153 our failure to detect the effect of feature-based attention on SSVEP amplitude. We 154 considered whether task factors such as stimulus flicker frequency, sample size, stimulus 155 duration, and stimulus color might have impacted our ability to observe an attention effect. 156

No single methodological factor that we considered neatly explains our lack of effect. 157

Given our results and literature review, we propose that future work is needed to 158 systematically explore two key factors: (1) variation in feature-based attention effects 159 across stimulus flicker frequencies and (2) the extent to which feature-based priming 160 modulates SSVEP attention effects. 161

We published a pre-registered research plan on the Open Science Framework 163 prior to data collection (https://osf.io/kfg9h/). Our raw data and analysis code will be made available online on the Open Science Framework at https://osf.io/ew7dv/ upon 165 acceptance for publication. 166

Healthy volunteers (n = 32; gender = 17 female, 15 male; mean age = 21.5 years 168 [SD = 3.84, min = 18, max = 39]; handedness not recorded; corrected-to-normal visual 169 acuity; normal color vision) participated in one 3.5 to 4 hour experimental session at the 170 University of California San Diego (UCSD) campus, and were compensated $15/hr. 171

Procedures were approved by the UCSD Institutional Review Board, and all participants 172 provided written informed consent. Inclusion criteria included normal or corrected-to-173 normal visual acuity, normal color vision, age between 18 and 60 years old, and no self-174 reported history of major neurological disorders (e.g., epilepsy, stroke). Data were 175 excluded from analysis if there were fewer than 400 trials in either cue condition (either 176 due to leaving the study early or after artifact rejection). A sample size of 24 was pre-177 registered, and artifact rejection criteria were pre-registered (see section "EEG 178 preprocessing" below for more details). After running each participant, we checked 179 whether the data were usable (i.e., sufficient number of artifact-free trials) so that we 180 would know when to stop data collection. To reach our final sample size (n = 23 181 participants with usable data), we ran a total of 32 participants. Nine participants' data 182

were not used for the following reason: Subjects with an error in the task code (n = 3), 183 subjects who stopped the study early due to technical issues or to participants' 184 preferences (n = 4), subjects with too many artifacts (n = 2). Note, we were one subject 185 short of our pre-registered target sample size of 24 because data collection was 186 suspended due to COVID-19. However, as our later power analyses will show, we do not 187 believe the addition of 1 further subject would have meaningfully altered our conclusions. 188

Heterochromatic flicker photometry task. We chose perceptually equiluminant colors 190 for each participant using a heterochromatic flicker photometry task. Participants viewed 191 a large circular, flickering stimulus (8º radius) on a black screen (0.08 cd/m2). We 192 generated 5 circular color spaces in CIELAB-space with varying luminance (circles 193 centered on: L = 35-65, a = 0, b = 0; 5 colors equally spaced around circle with radius = 194 35) for use in the task. Participants matched each of the 5 colors to a medium-gray 195 reference color (RGB = 105.6 105.6 105.6). 196

On each trial, the circular background was flickered between two different colors. 197

One color was always medium-gray, and the other color was the to-be-matched color on 198 that trial. The colors of circular background were phase reversed at a rate of 24 Hz, giving 199 the appearance of a fast flicker when the subjective luminance values were not matched. 200

On top of the flickering circular stimulus small oriented bars were drawn in the medium-201 gray reference color (the bars changed locations at a rate of 1Hz). The oriented bars 202 served no purpose other than subjectively making it easier to discriminate fine-grained 203 differences in luminance between the flickering colors (i.e., these bars gave secondary 204 visual cues about equiluminance via the "minimally distinct border" phenomenon, Kaiser, 205 1988) . Participants increased or decreased the luminance of the to-be-matched color 206 (using up and down arrow keys) until the amount of perceived flicker was minimized -the 207 point of perceptual equiluminance. The luminance starting value of the to-be-matched 208 color was chosen at random on each trial. Once satisfied with their response, the 209 participant pressed spacebar to continue to the next trial. Each to-be-matched color was 210 repeated 3 times (15 trials total). 211 Feature-based attention task. All stimuli were viewed on a luminance calibrated 212 CRT monitor (1024 x 768 resolution, 120 Hz refresh rate) from a distance of ~50 cm in a 213 dimly lit room. Stimuli were generated using Matlab 2016a and the Psychophysics toolbox 214 (Brainard, 1997; Kleiner et al., 2007; Pelli, 1997) . Participants rested their chin on a chin-215 rest and fixated a central dot (0.15º radius) throughout the experiment. The stimulus was 216 a circular aperture (radius = ~9.5º) filled with 120 oriented bars (each bar ~1.1º long and 217 ~.1º wide). Bars were centered on a grid and separated by ~1 bar length such that they 218 never overlapped with one another. On each individual frame (~8.33 ms) this grid was 219 randomly phase shifted (0:2π in x and y coordinates) and rotated (1:360º), thus giving the 220 appearance of random flicker. To achieve Steady State Visually Evoked Potentials 221 (SSVEP) half of the bars flickered at 24 Hz (3 frames on, 2 frames off) and the other half 222 flickered at 30 Hz (2 frames on, 2 frames off). Due to the jittered rotation of bar positions 223 and to the random assignment of colors to bars on each "on" frame, this means that the 224 individual pixels that were "on" for each color varied from frame to frame. The 225 unpredictable nature of each bar's exact position is thus quite similar to unpredictable 226 stimuli that have been used in past work (e.g., Andersen et al., 2008) . For each "off frame" 227 no bars of that color were shown (e.g., if 24 Hz had an "on" frame and 30 Hz had an "off" 228 frame, then only 60 out of 120 bars would be shown on the black background). If both the 229 24 Hz and 30 Hz bars were "off", then a black screen would be shown on that frame. See 230 Figure S1 for an illustration of some example frame-by-frame screenshots of the stimuli. 231

On each trial (Figure 1 ), the participants viewed the stimulus array of flickering, 232 randomly oriented bars presented on a black background (0.08 cd/m2). Half of these bars 233 were shown in one color (randomly chosen from the 5 possible colors) and the other half 234 were in another randomly chosen color (with the constraint that the two sets of bars must 235 be two different colors). During an initial baseline (1,333 ms), participants viewed the 236 flickering dots while they did not yet know which color to attend; during this baseline, the 237 fixation point was a medium gray color (same as the reference color in the flicker 238 photometry task). After the baseline, the fixation dot changed color, cuing the participants 239 about which color to attend or ignore. In the "attend cue" condition, the color of the fixation 240 point indicated which color should be attended. In the "ignore cue" condition, the color of 241 the fixation point indicated which color should be ignored. These two conditions were 242 blocked, and the order was counterbalanced across participants (further details below). 243

During the stimulus presentation (2,000 ms), participants monitored the relevant color for 244 a brief "target event" (333 ms). During this brief target event, a percentage of lines in the 245 relevant color will be coherent (iso-oriented). Critically, the orientation of each coherent 246 target or distractor event was completely unpredictable (randomly chosen between 1-180 247 degrees); thus, participants could not attend to a particular orientation in advance in order 248 to perform the task. A target event occurred on 50% of trials, and participants were 249 instructed to press the spacebar as quickly as possible if they detected a target event. 250 Importantly, physically identical events (iso-oriented lines in a random orientation, 333 251 ms) could also appear in the distractor color (50% of trials). Participants were instructed 252 that they should only respond to target events; if they erroneously responded to the 253 distractor event, the trial was scored as incorrect. The target and/or distractor events could begin as early as cue onset (0 ms) and no later than 1,667 ms after stimulus onset). 255

Participants could make responses up to 1 second into the inter-trial interval. If both a 256 target and distractor event were present, their onset times were separated by at least 333 257 ms. target events, and distractor events in the 2 main conditions. In the 'attend cue' condition, 268

participants made a response when the iso-oriented lines were the same color as the cue 269 (target event) and did not respond if the iso-oriented lines occur on the uncued color 270 (distractor event). In the 'ignore cue' condition, participants made a response when the 271 iso-oriented lines occurred on the uncued color (target event), and they did not respond 272

if the iso-oriented lines occurred on the cued color (distractor event). Note, all lines were 273 of equal size in the real experiment; lines are shown at different widths here for easier 274 visualization of the target and distractor colors. Here, the iso-oriented lines are drawn at 275 vertical in all 4 examples. In the actual task, the iso-oriented lines could be any orientation 276

(1-180). 277 278

To ensure that the task was effortful for participants, the coherence of the lines in 279 the target stimulus was adapted at the end of each block if behavior was outside the range 280 of 70 -85% correct. At the beginning of the session, the target had 50% coherent iso-281 oriented lines. If accuracy over the block of 80 trials was >85%, coherency decreased by 282 5%. If block accuracy was <70%, coherency increased by 5%. The maximum allowed 283 coherence was 80% iso-oriented lines (so that participants would not be able to simply 284 individuate and attend a single position to perform the task) and the minimum allowed 285 coherence was 5%. The presence and absence of target and distractor events was 286 balanced within each block yielding a total of 4 sub-conditions within each cue type (25% 287 each): (1) target event + no distractor event, T1D0 (2) no target event + distractor event, 288 T0D1 (3) target event + distractor event, T1D1 (4) no target event + no distractor event, 289 T0D0. 290

Participants completed both task conditions (attend cue and ignore cue). The two 291 conditions were blocked and counterbalanced within a session (i.e., half of participants 292 performed the "attend cue" task for the first half of the session and the "ignore cue" task 293 for the second half of the session.) Each block of 80 trials took approximately 6 min 50 294 sec. Participants completed 18 blocks (9 per condition) for a total of 720 trials per cue 295 condition. Note, we originally planned for 20 blocks (10 per condition) in the pre-296 registration, but the block number was reduced to 18 after the first few participants did 297 not finish all blocks. 298 Summary of deviations from the registered procedures. As described in-line 299 above, there were some minor deviations from the pre-registration: (1) We made changes 300 to the pre-registered task code to fix errors that we discovered while running the first 3 301 subjects (e.g., incorrect cues and behavioral feedback in the 'ignore cue' condition). (2) 302

We included code for eye-tracking, which allowed us to give participants automated real-303 time feedback if they blinked when they were not supposed to, i.e., during the stimulus 304 period. (3) We reduced the total number of experimental blocks from 20 (10 per cue 305 condition) to 18 (9 per cue condition) due to time constraints. (4) We had to prematurely 306 stop data collection at n = 23 out of 24 due to COVID-19. (5) We forgot to specify a specific 307 statistical test for quantifying the robustness of overall SSVEPs in section "Checking that 308 an SSVEP is elicited at the expected frequencies before collecting the full sample", so we 309 have described our justification for the statistical tests we present here. (6) Due to unanticipated failure to detect an overall attention effect, we performed additional non-311 pre-registered control analyses to attempt to rule out possible explanations of this null 312 effect (see section: "Non pre-registered control analyses" below). 313 314 EEG pre-processing 315

Continuous EEG data were collected online from 64 Ag/AgCl active electrodes 316 mounted in an elastic cap using a BioSemi ActiveTwo amplifier (Cortech Solutions, 317

Wilmington, NC). An additional 8 external electrodes were placed on the left and right 318 mastoids, above and below each eye (vertical EOG), and lateral to each eye (horizontal 319 EOG). Continuous gaze-position data were collected from an SR Eyelink 1000+ eye-320 tracker (sampling rate: 1,000 Hz; SR Research, Ottawa, Ontario). We also measured 321 stimulus timing with a photodiode affixed to the upper left-hand corner of the monitor (a 322 white dot flickered at the to-be-attended color's frequency; the photodiode and this corner 323 of the screen were covered with opaque black tape to ensure it was not visible). Data 324 were collected with a sampling rate of 1024 Hz and were not downsampled offline. Data 325 were saved unfiltered and unreferenced (see: Kappenman & Luck, 2010) , then 326 referenced offline to the algebraic average of the left and right mastoids, low-pass filtered 327 (<80 Hz) and high-pass filtered (>.01 Hz). Artifacts were detected using automatic criteria 328 described below, and the data were visually inspected to confirm that the artifact rejection 329 criteria worked as expected. We excluded subjects with fewer than 400 trials remaining 330 per cue condition. 331

Eye movements and blinks. We used the eye-tracking data and the 332 HEOG/VEOG traces to detect blinks and eye movements. Blinks were detected on-line 333 during the task using the eye tracker. If a blink was detected (i.e., missing gaze position 334 returned from the eye tracker), the trial was immediately terminated and the participant 335 was given feedback that they had blinked (i.e., the word "blink" was written in white text 336 in the center of the screen). If eye-tracking data could not be successfully collected (e.g., 337 calibration issues), the VEOG trace was used to detect blinks and/or eye movements 338 during offline artifact rejection. To do so, we used a split-half sliding window step function 339 (Luck, 2005 ; window size = 150 ms, step size = 10 ms, threshold = 30 microvolts.) We also used a split-half sliding-window step function to check for eye-movements in the 341 gaze-coordinate data from the eye-tracker (window size = 80 ms, step size = 10 ms, 342 threshold = 1º) and in the horizontal electrooculogram (HEOG), window size = 150 ms, 343 step size = 10 ms, threshold = 30 microvolts, and to detect blinks and/or eye movements 344 We also pre-registered an analysis plan for examining the time-course of SSVEP 372 amplitude. However, because our data failed to satisfy pre-registered pre-requisite 373 analyses, we do not report these time-course effects here (for completeness, we show 374 the time-course of SNR in Figure S2 ). 375

Checking that an SSVEP is elicited at the expected frequencies before 376 collecting the full sample. At n = 5, we planned to confirm that our task procedure 377 successfully produced reliable SSVEP responses (i.e., check that we observed peaks at 378 the correct stimulus flicker frequencies). If our task procedures failed to elicit an SSVEP 379 at the expected frequencies, we had planned to stop data collection and alter the task to 380 troubleshoot the problem (e.g., optimize timing, choose different flicker frequencies, make 381 stimuli brighter, etc.). We planned to begin data collection over again if we failed this 382 trouble-shooting step. Note, at this early stage we only verified if the basic method worked 383 (SSVEP frequencies were robust): we did not test whether any hypothesized attention 384 effects were present, as this could inflate our false discovery rate (Kravitz & Mitroff, 2017) . 385

Note, in the original pre-registration we failed to specify what test we would run to 386 determine if SSVEP frequencies were robustly represented in the EEG signal. Theoretical 387 chance for SNR would be 1, so the simplest test would be to compare the SNR for our 388 stimulation frequencies (24 and 30 Hz) to 1 using a t-test, which we report. However, it is 389 often is better to compare to an empirical baseline with a reasonable amount of noise 390 (Combrisson & Jerbi, 2015) . As such, we opted to also compute an effect size comparing 391 the SNR for our stimulation frequencies to all other frequencies (with the exception that 392 we did not use frequencies +/-2 Hz of 24 or 30 Hz as baseline values, since SNR was 393 calculated as the power at frequency F divided by the power in the 2 adjacent 1-hz bins). 394

Checking achieved power for the basic attention effect. Without We did not anticipate our failure to find an overall attention effect with these task 428 procedures and set of pre-registered "sanity check" analyses described above. To further understand the lack of SSVEP attention effect, we performed additional non-pre-430 registered control analyses. We first confirmed that our SSVEP procedure was effective at eliciting robust, 477 frequency-specific modulations of the EEG signal. After collecting the first 5 participants, 478 we checked that overall SSVEP amplitudes for our two target frequencies (24 and 30 Hz) 479 were robust when collapsed across conditions (Fig 2A) before proceeding with data 480 collection. We indeed found that the SSVEP signal was robust during the stimulus period 481 even with n=5 for both the 24 Hz frequency (mean SNR = 4.45, SD = .14, SNR > 1: p < 482

.001) and for the 30 Hz frequency (mean SNR = 2.97, SD = .15, SNR > 1: p < .001). 483

These values were similar for the full n=23 sample (Fig 2B) . To compute an effect size, 484 we compared SNR values for each target frequency (24 Hz and 30 Hz) to the SNR values 485 for each baseline frequency (frequencies from 3-33 Hz not within +/-2 Hz of 24 or 30 Hz). 486 SNR values for the target frequencies were significantly higher than baseline, mean 487

Cohen's d = 5.10 (SD = 1.11) and 5.99 (SD = 2.66), respectively (See Figure S2) . As 488 planned, we also confirmed that the electrodes we selected a priori (O1, Oz, and O2) were reasonable given the topography of overall SSVEP amplitudes (i.e., they fell 490 approximately centrally within the brightest portion of the heat map; Figure experimental conditions. Color scale indicates SNR. As expected, the a priori electrodes 500 O1, O2, and Oz (magenta circles) showed robust SNR during the stimulus period. 501 502 503

Next, we checked for a basic attention effect, defined as a larger amplitude 504 response evoked by the attended frequency compared to the ignored frequency). Note, 505

for the sake of clarity, all conditions are translated into "attend" terminology. That is, if a 506 participant was cued to "ignore blue" (24 Hz) during the "ignore cue" condition (and the 507 other color was red and 30 Hz), this will instead be plotted as "attend red" (30 Hz). Figure  508 3 shows the Gaussian wavelet-derived frequency spectra during the stimulus period (500-509 2000 ms) as a function of cue type (attend versus ignore) and attended frequency (attend 510 24 Hz or attend 30 Hz). We found a main effect of measured frequency, whereby SNR 511 was overall higher for 24 versus 30 Hz, F(1,22) = 57.89, p < .001, η 2 p = .73. However, we 512 found no main effect of attended frequency (p = .27) or cue type (p = .83), and we found 513 no significant interactions (p >= .18). Collapsed down to a paired t-test, the observed 514 effect size for attended versus unattended SNR values was Cohen's d = .03. To detect 515 an effect of this size with 80% power (1-β = .8; α = .05) would require a sample size n > 516 7,000 * . Given that we did not find an overall attention effect, we did not analyze or interpret 517 analysis of the SSVEP time-course. However, for completeness we have shown the time 518 course in Figure S3 . Frequency spectra in the attend cue (A) and ignore cue (B) conditions during the stimulus 523 period. Although we observe expected peaks at 24 Hz and 30 Hz, this SSVEP response 524

is not modulated by the attention manipulation. (C-D). Violin plots of the signal-to-noise 525 ratio at the SSVEP frequencies in the attend cue (C) and ignore cue (D) conditions. 526 527

Although we pre-registered that we would analyze all trials (those with and without 528 target/distractor events), most prior studies have included only trials without any target or 529 distractor events in the main SSVEP analysis (e.g., Andersen et al., 2008; Müller et al., 530 2006) . To ensure that our null result was not due to this analysis choice, we also planned 531 in our pre-registration to examine the SSVEP attention effect for trials with and without 532 target and distractor events. When restricting our analysis to only trials without targets or 533 distractors (25% of the 1440 trials, or 360 trials total before artifact rejection), we likewise 534 found no attention effect. As before, we found a main effect of measured frequency (24 > 535 30 Hz), p < .001, but no effect of cue condition (p = .053) or attended frequency (p = .073), 536 and, most critically, we found no interaction between measured frequency and attended 537 frequency (p = .33). Frequency spectra for all combinations of target and distractor 538 presence are shown in Figures S4 and S5 . 539

Finally, we also pre-registered that we would check whether the similarity of the 540 target and distractor colors (72 versus 144 degrees apart on a circular color wheel; Figure  541 1B) would modulate the SSVEP attention effect. We likewise found that the similarity of 542 the distractor colors did not significantly modulate the SSVEP response, and we found no 543 attention effect (interaction of measured frequency and attended frequency) in either color 544 distance condition (p >= .26; Figure S6 ). 545 546

We conducted additional control analyses to rule out possible sources of our failure 548 to find an attention effect. First, we examined the photodiode recording to rule out any 549 failures due to trial indexing. The photodiode measured the luminance of a white dot that 550 flickered at the attended frequency on each trial. As expected, performing an FFT on the 551 photodiode time-course thus yielded near-perfect tracking of the attended frequency 552 ( Figure 4A-B, p < .001) . On the other hand, we again found null results for the main 553 attention manipulation ( Figure 4C -F) when using an FFT analysis that more closely 554 followed prior work. We ran a repeated measures ANOVA on the signal to noise ratio 555 values during the stimulus period, including the factors Measured Frequency (24 Hz, 30 556 Hz), Attended Frequency (24 Hz, 30 Hz), and Cue Type (Attend, Ignore). We found no main effect of measured frequency (p = .91), attended frequency (p = .45), or cue type (p 558 = .54), and we found no significant interactions (p >= .38). However, the average signal-559 to-noise ratio of the stimulus frequencies was overall robust (M = 4.45, SD = 1.45, greater 560 than chance value of 1: p < 1x10 -9 ), so our inability to observe the attention effect was not 561 due to lack of overall SSVEP signal. 562

Given that some work has reported significant effects only for the second harmonic 563 Figures S7 and S8) . We found no significant attention effects for either second harmonic 566 frequency. We also re-ran the FFT analysis with other electrode-selection methods to 567 ensure our a priori choice of electrodes did not impede our ability to observe an effect (M. 568 X. Cohen & Gulbinaite, 2017). We found no evidence that electrode choice led to our null 569 effect, as exploiting information from all 64 electrodes by implementing rhythmic 570 entrainment source separation (RESS) likewise yielded null effects ( Figure S9 -S10). To 571 ensure that inconsistent task performance did not lead to null effects, we repeated the 572 main FFT analysis on only accurate trials. We likewise found null attention effects when 573 analyzing only accurate trials ( Figure S11) . 574

Finally, we tested whether phase consistency, rather than power, may track 575 attention in our task (e.g., Nunez et al., 2015; Tallon-Baudry et al., 1996) . To do so, we 576 performed an FFT on single trials rather than on condition-averaged waveforms, and we 577 extracted single-trial phase values. We calculated a phase-locking index by computing 578 mean-resultant vector length on each condition's histogram of single-trial phase values. 579

Mean-resultant vector length ranges from 0 (fully random values) to 1 (perfectly identical 580 values), for reference, see Zar (2010) . We found no effect of attention on this phase-581 locking index ( Figure S12 ). was not due to using Gaussian wavelets rather than an FFT, we repeated the main 590 analysis with an FFT. Frequency spectra for the attend cue condition (C) and ignore cue 591 condition (D) reveal an overall robust SSVEP signal at 24 Hz and 30 Hz, but no 592 modulation by attention. Likewise, violin plots of signal-to-noise ratios again show robust 593 signal but no modulation by attention in either the attend cue condition (E) or the ignore 594 cue condition (F). 595 Positive control: Analysis of event-related potential (P3b) for an attention 597 effect. Consistent with prior work, we found a significantly larger P3 component for target 598 onsets compared to distractor onsets ( Figure 5) . A repeated measures ANOVA with 599 within-subjects factors cue type (attend cue or ignore cue) and event type (target or 600 distractor onset) revealed a robust main effect of event type (target > distractor), F(1,22) 601 = 51.64, p < 1x10 -5 , η 2 p = .70, and a main event of cue type (attend > ignore), F(1,22) = 602 4.96, p = .037, η 2 p = .18, but no interaction between event type and cue type (p = .65). 603

Control analyses confirmed this P3 modulation was not due to differential rates of making 604 a motor response for targets and distractors ( Figure S13 ). The main effect of event type 605 (target > distractor) remained when analyzing only trials where participants made a motor 606 response (p < .001). Thus, the P3 was overall larger for target than distractor events, 607 consistent with prior work that found this ERP attention effect alongside an SSVEP 608 attention effect. We defined "feature-based attention manipulation" as having the following 632 characteristics: (1) Participants were cued to attend a feature(s) within a feature 633 dimension (e.g., attend red, ignore blue) rather than across a feature dimension (e.g. 634 attend contrast, ignore orientation), (2) The attended and ignored feature were both 635 frequency-tagged in the same trials (rather than only 1 feature tagged per trial), (3) Each 636 frequency was both "attended" and "ignored" on different trials, so that the amplitude of a 637

given frequency could be examined as a function of attention, (4) The task could not be 638 performed by adopting a strategy of splitting spatial attention to separate spatial locations. 639

After applying these screening criteria, some of the studies that we initially We identified a total of 34 experiments from 28 unique papers (Tables S1-S4)  645 meeting our inclusion criteria. From these experiments, we quantified variables such as 646 the number of subjects, number of trials, frequencies used, and the presence or absence 647 of an attention effect in the expected direction (attended > ignored). If more than one 648 group of participants was used (e.g., an older adults group) then we included the study but only quantified results for the healthy young adult group (Quigley et al., 2010; Quigley 650 & Müller, 2014) . 651

The tasks used in these studies fell broadly into one of 4 categories: (1) a 653 competing gratings task, (2) a whole-field flicker task, (3) a hemifield flicker task and (4) 654 a central task with peripheral flicker. 655

In the competing gratings task (Table S1 ), participants viewed a stream of 656 centrally-presented, superimposed gratings (e.g., a red horizontal grating and a green 657 vertical grating). Because colored, oriented gratings were typically used, participants 658 could thus generally choose to attend based on either one or both features (color and/or 659 orientation). Each grating flickered at its own frequency (e.g. green grating shown at 7.41 660

Hz, red grating shown at 8.33 Hz, as in Chen et al., 2003) . Because the gratings were 661 superimposed, on any given frame only one of the two gratings was shown. On frames 662

where both gratings should be presented according to their flicker frequencies, a hybrid 663 "plaid" stimulus was shown. Studies using a competing gratings task include: (Allison et In the whole-field flicker task (Table S2) , participants viewed a spatially global 666 stimulus comprised of small, intermingled dots or lines. Typically, half of the dots or lines 667 were presented in one feature (e.g., red) and the other half of the lines were presented in 668 another (e.g., blue); each set of dots flickered at a unique frequency. Although the most 669 common attended feature was color, some task variants included (1) attending high or 670 low contrast stimuli (2) attending a particular orientation or (3) attending a particular 671 conjunction of color and orientation. The whole-field flicker task was the most common 672 task variant, and it is also most similar to the task performed here. Studies using a whole- First, we examined whether insufficient power (e.g., fewer subjects and/or trials 695 relative to prior work) could have led to our failure to detect an attention effect. The 696 number of studies employing each task variant is plotted in Figure 6A , the number of 697 subjects per experiment is plotted in Figure 6B , the number of trials per experiment is 698 plotted in Figure 6C , and stimulus duration is plotted in Figure 6D Next, we examined the percentage of trials where the attended feature was 731 repeated (e.g., if the attended color was red on trial n, what was the chance that red would 732 also be attended on trial n+1?). The priming-based account of feature-based attention 733 posits that participants cannot use trial-by-trial cues to enhance a particular feature, but 734 rather, feature-based enhancement happens automatically when a particular feature is 735 repeated (Theeuwes, 2013) . Thus, if there is a substantial proportion of trials where the 736 repeated color was attended (e.g. with 2 possible colors, both the attended and ignored 737 color will be repeated on 50% of trials), then the observed attentional enhancement 738 effects might be driven primarily by incidental repetitions of attended features. In our 739 study, we used 5 different colors to reduce the potential effect of inter-trial priming on the 740 observed SSVEP attention effects (20% repeats of the attended color, 4% repeats of the 741 attended color and the ignored color). We quantified the approximate percentage of trials 742 on which an attended feature on one trial is repeated on the next trial (within a given block 743 of trials). In some studies, participants were cued to attend more than one feature on a 744

given trial, or they sometimes attended to a conjunction of features. In these cases, we 745 in the studies with 0% repeats might be equally attributed to their low trial counts (median 758 of repeats as the present study (Störmer & Alvarez, 2014 ). Störmer and Alvarez found a 760 significant attention effect while using 5 unique colors (intermixed randomly from trial to 761 trial). The findings by Störmer and Alvarez provide evidence against the feature-based 762 priming account, and suggest the task factor "number of colors" cannot definitively explain 763 our inability to observe an attention effect. However, given the lack of extant work using 764 unpredictable color cues, we think future, systematic work is needed to determine the 765 degree to which inter-trial priming effects may modulate the size and reliability of feature-766 based attention effects. 767 768 SSVEP Frequencies. 769

We examined frequencies that have been most commonly used in the literature. 770

In our study, we chose relatively high frequencies (24 and 30 Hz) in order to have 771 increased temporal resolution for detecting potential time-course effects. In addition, 772 some have argued that using higher frequencies as advantages for driving a more Finally, we examined whether the type of task and task difficulty may have 793 influenced our ability to detect an attention effect. In particular, the specific targets that 794 we used may differ slightly from prior work. In our experiment, participants detected a 795 brief period (333 ms) of an on average ~75% coherent orientation (the coherent line 796 orientation was a random, unpredictable direction, from 1-180 degrees). In this task, 797 participants performed well above chance, but the task was still fairly challenging overall 798 (d' = 1.25). This raises the possibility that, compared to prior SSVEP studies, subjects 799 were giving up on some percentage of the trials and that this contributed to the lack of 800 attention effects. 801

For the reviewed papers in which participants detected a target within the flickering 802 stimulus ("whole-field flicker task" and "hemifield flicker task"), we compiled information 803 about participants' accuracy, the duration of the target, the type of target, and the 804 percentage of dots/lines that comprised the target (Table S5) . We found that our particular 805 task (detect a coherent orientation in the cued color) was slightly different from the other 806 tasks that have been used. Two other prior studies did not use a behavioral task at all: 807 participants were simply instructed to monitor a particular feature without making any 808 overt response (Pei et Although the particulars of the luminance and motion tasks subtly differ from our 816 orientation task, it is not clear why SSVEPs would track attention when the target is a 817 coherent luminance value or motion direction, but not when the target is a coherent 818 condition. Thus, because we found no overall SSVEP attention effect, we were unable to 849 test our hypotheses about how this attention effect was modulated by being cued to attend 850 versus cued to ignore. Despite the lack of an SSVEP attention effect, positive control 851 analyses indicated that that participants did successfully select the cued target color (i.e., 852 we observed a significantly larger P3 component for target events in the attended color 853 than in the ignored color). 854

Given our failure to observe an effect of attention on SSVEP amplitude with our 855 task procedures, we performed a focused review of the literature to quantify key 856 methodological aspects of prior studies using SSVEPs to study feature-based attention. 857

Based on this review, we concluded that sample size and trial counts likely did not explain 858 our failure to find an effect; our sample size and trial counts were near the maximum 859 values found in the surveyed literature. Likewise, the range of accuracy values found in 860 the literature suggests that task difficulty does not explain our failure to find an attention 861 effect. However, two key, intentional design differences may have hampered our ability 862 to find an effect: (1) the number of colors in our stimulus set and (2) the frequencies used 863 to generate the SSVEP. 864

The first key design difference in our study was the number colors in our stimulus 865 set. We purposefully minimized the influence of inter-trial priming on our estimates of 866 feature-based attention (Theeuwes, 2013) by using 5 unique colors and randomly chose 867 target and distractor colors on each trial. According to a priming account of feature-based 868 attention, a relatively high proportion of trials where the attended color is repeated back-869 to-back could inflate or even entirely drive apparent feature-based attention effects. Using 870 5 colors somewhat protects against this possibility, because it ensures that the attended 871 color is repeated on 20% of trials, and both the attended/ignored colors are repeated on 872 only 4% of trials. In the literature, we found that most studies had back-to-back color 873 repeats on at least 50% of trials. It is thus plausible that inter-trial priming could contribute 874 to observed attention differences in these studies. Contrary to a priming account, 875 however, one study found robust feature-based attention effects using a set of 5 unique 876 colors (Störmer & Alvarez, 2014) , suggesting that participants can use a cue to direct 877 feature-based attention even when the proportion of repeated trials is relatively low. To 878 date, however, no study has directly manipulated the proportion of repeated trials or the 879 number of possible stimulus colors in an SSVEP study. Given emerging evidence that 880 history-driven effects play an important role in shaping both spatial and feature-based 881 attentional selection ( The second key design difference in our study was the chosen set of frequencies. 886

To ensure adequate temporal resolution to characterize time-course effects, we chose to 887 use slightly higher frequencies (24 and 30 Hz). We believed these values would be 888 reasonable, because an initial study of the time-course of spatial attention used SSVEP It is perhaps puzzling that frequencies above 20 Hz have been commonly used in 907 the spatial attention literature but have not been used in the feature-based attention literature. The truncation of the frequency distribution in the reviewed literature could be 909 a piece of the "file drawer" in action. It is possible that other researchers likewise 910 discovered that they were unable to track feature-based attention using certain 911 frequencies, but that these null results were never published due to journals' and authors' 912 biases toward publishing positive results (Ferguson & Heene, 2012; Rosenthal, 1979 ) 913 and biases against publishing negative results (i.e., "censoring of null results", Guan & 914 Vandekerckhove, 2016; Sterling, 1959; Sterling et al., 1995) . Thus, our results highlight 915 the practical and theoretical importance of regularly publishing null results. On the 916 practical side, if prior null results had been published, we may have better known which 917 frequencies to use or avoid, and we would have been able to test our key hypotheses. 918

On the theoretical side, our results highlight how seemingly unimportant null results can 919 have implications for theory when viewed in the context of the broader literature. For 920 example, if certain frequencies track spatial but not feature-based attention, this may 921 inform our understanding of the brain networks and cognitive processes differentially 922 modulated by flicker frequency (Ding et al., 2006; Srinivasan et al., 2006) . 923

In short, we found no evidence that SSVEPs track the deployment of feature-based 924 attention with our procedures, and future methodological work is needed to determine 925 constraints on generalizability of the SSVEP method for tracking feature-based attention. 926

We performed a focused review of prior studies using SSVEPs to study feature-based 927 attention, and from this review we identified two key factors (frequencies used; likelihood 928 of inter-trial feature priming) that should be systematically investigated in future work. 929

History-driven modulations of population codes in 930 early visual cortex during visual search

Top-Down Attention Is Limited Within but 933 Not Between Feature Dimensions

The Berger Rhythm: Potential Changes from the 936

Towards an independent brain-computer interface using steady state visual evoked potentials

Effects of Feature-selective and Spatial 942

Attention at Different Stages of Visual Processing

Attention Facilitates Multiple Stimulus 945 Features in Parallel in Human Visual Cortex

Global Facilitation of Attended Features 948

Is Obligatory and Restricts Divided Attention

Behavioral performance follows the time course of 951 neural facilitation and suppression during cued shifts of feature-selective attention

Color-selective attention need not be 955 mediated by spatial attention

Tracking the allocation of attention in 957 visual scenes with steady-state evoked potentials

Attentional Selection of Feature 960

Conjunctions Is Accomplished by Parallel and Independent Selection of Single Features

Bottom-Up Biases in Feature-Selective 964

Too little, too late, and in the wrong 967 place: Alpha band activity does not reflect an active mechanism of selective attention

Attentive and pre-attentive aspects of figural 970 processing

Templates for rejection: Configuring 972 attention to ignore task-irrelevant features

Top-down versus bottom-up attentional 975 control: A failed theoretical dichotomy

Evidence for negative feature guidance in visual search 978 is explained by spatial recoding

No templates for rejection: A failure to 981 configure attention to ignore task-irrelevant features

How many trials does 984 it take to get a significant ERP effect? It depends

Attention to a threat-related 987 feature does not interfere with concurrent attentive feature selection

The Psychophysics Toolbox

Distinct Attention Networks for Feature Enhancement 992 and Suppression in Vision

High Gamma Power Is Phase-Locked to 996 Theta Oscillations in Human Neocortex

Location-based explanations do not account for active 999 attentional suppression

Feature-based attention resolves 1002 differences in target-distractor similarity through multiple mechanisms

The power of human brain 1005 magnetoencephalographic signals can be modulated up or down by changes in an 1006 attentive visual task

Tracking feature-based attention

Normal Electrocortical Facilitation But Abnormal 1011

Target Identification during Visual Sustained Attention in Schizophrenia

Using Neuronal Populations to Study the 1014

Mechanisms Underlying Spatial and Feature Attention

Rhythmic entrainment source separation: Optimizing 1017 analyses of neural responses to rhythmic sensory stimulation

Exceeding chance level by chance: The caveat of 1020 theoretical chance levels in brain signal classification and statistical assessment of 1021 decoding accuracy

Feature guidance by negative 1024 attentional templates depends on search difficulty

Taming the White Bear: Initial Costs and Eventual 1027 Benefits of Distractor Inhibition

Attentional Modulation of SSVEP Power 1030

Depends on the Network Tagged by the Flicker Frequency

Cortical 1033 Mechanisms of Prioritizing Selection for Rejection in Visual Search

Statistical 1036 regularities induce spatial as well as feature-specific suppression

A Vast Graveyard of Undead Theories: Publication Bias 1040 and Psychological Science's Aversion to the Null

Global Enhancement but Local 1043

Suppression in Feature-based Attention

Near-Real-Time Feature-Selective 1046

Feature-based attention is constrained to attended 1049 locations in older adults

A Bayesian approach to mitigation of publication bias

Attention differentially modulates 1055 the amplitude of resonance frequencies in the visual cortex

Feature-based attentional tuning during 1058 biological motion detection measured with SSVEP

The 1061 functional organization of human extrastriate cortex: A PET-rCBF study of selective 1062 attention to faces and locations

Human EEG responses to 1?100 Hz flicker: Resonance phenomena 1065 in visual cortex and their potential correlation to cognitive phenomena

LIP responses to a 1068 popout stimulus are reduced if it is overtly ignored

When Conflict Cannot be Avoided

Relative Contributions of Early Selection and Frontal Executive Control in Mitigating 1072

Stroop Conflict

Temporal dynamics of divided spatial 1075 attention

A category-specific top-down attentional set can affect the 1078 neural responses outside the current focus of attention

Sensation luminance: A new name to distinguish CIE luminance from 1081 luminance dependent on an individual's spectral sensitivity

The effects of electrode impedance on data quality and 1084 statistical significance in ERP recordings

Time Courses of Attentional 1087

Modulation in Neural Amplification and Synchronization Measured with Steady-state 1088

Visual-evoked Potentials

Audio-visual synchrony and feature-selective attention co-1091 amplify early visual processing

Attention induces 1094 synchronization-based response gain in steady-state visual evoked potentials

What's new in Psychtoolbox-3?

Estimates of a priori power and false discovery rates induced by 1100 post-hoc changes from thousands of independent replications

Large-scale network-level 1103 processes during entrainment

An Introduction to the Event-Related Potential Technique

Feature-Based Attention Increases the Selectivity of 1108

Population Responses in Primate Visual Cortex

Cortical summation and attentional modulation of 1111 combined chromatic and luminance signals

Neural 1114 mechanisms of divided feature-selective attention to colour

The ignoring paradox: Cueing distractor features leads first to 1117 selection, then to inhibition of to-be-ignored items. Attention, Perception

Feature-Based Attention

Selective attention to stimulus location 1123 modulates the steady-state visual evoked potential

Feature-selective attention enhances color signals in early visual areas of the 1127 human brain

It takes two to tango: 1130 Suppression of task-irrelevant features requires (spatial) competition

Concurrent recording of steady-state and transient event-1133 related potentials as indices of visual-spatial selective attention

Effects of spatial selective attention on the steady-state visual evoked potential 1137 in the 20-28 Hz range

The time course of cortical 1140 facilitation during cued shifts of spatial attention

The 1143 steady-state visual evoked potential in vision research: A review

Individual differences in attention 1146 influence perceptual decision making

Memory for object features versus 1149 memory for object location: A positron-emission tomography study of encoding and 1150 retrieval processes

Causal involvement of visual area MT in 1153 global feature-based enhancement but not contingent attentional capture

Neural Responses to Target 1156 Features outside a Search Array Are Enhanced during Conjunction but Not Unique-1157

Neural correlates of object-based attention

The VideoToolbox software for visual psychophysics: Transforming numbers 1162 into movies

Feature-1164 selective attention: Evidence for a decline in old age

Feature-Selective Attention in Healthy Old Age: A Selective 1167 Decline in Selective Attention

Cortical evidence for negative search 1170 templates

Steady-state evoked potentials

The file drawer problem and tolerance for null results

Expectations Do Not Alter Early Sensory Processing during Perceptual Decision-Making

Capture versus suppression of attention by salient singletons: 1181 Electrophysiological evidence for an automatic attend-to-me signal

Steady-State Visual Evoked Potentials

Distributed Local Sources and Wave-Like Dynamics Are Sensitive to Flicker Frequency

Rapid adaptive adjustments of selective attention 1187 following errors revealed by the time course of steady-state visual evoked potentials

Publication Decisions and their Possible Effects on Inferences Drawn 1190 from Tests of Significance-Or Vice Versa

Publication Decisions Revisited

The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa

Feature-Based Attention Elicits Surround Suppression 1197 in Feature Space

Stimulus Specificity of 1200 Phase-Locked and Non-Phase-Locked 40 Hz Visual Responses in Human. The Journal 1201 of Neuroscience

Attentional capacity for processing 1204 concurrent stimuli is larger across sensory modalities than within a modality

Feature-based attention: It is all bottom-up priming

Selection of Visual 1210 Objects in Perception and Working Memory One at a Time

Using frequency tagging to 1213 quantify attentional deployment in a visual divided attention task

Inhibition in selective attention

Experience-dependent attentional tuning of distractor 1218 rejection

Attention Selects Informative Neural 1221 Populations in Human V1

Steady-state visually evoked 1224 potentials: Focus on essential paradigms and future perspectives

Protecting visual short-1227 term memory during maintenance: Attentional modulation of target and distractor representations

Statistical regularities modulate attentional capture

How to inhibit a distractor location? Statistical learning 1234 versus active, top-down suppression

The neural correlates of feature-based selective 1237 attention when viewing spatially and temporally overlapping images

Effect of higher 1240 frequency on the classification of steady-state visual evoked potentials

Biostatistical Analysis

An independent brain-1244 computer interface using covert non-spatial visual selective attention

Feature-based attention modulates feedforward visual 1247 processing

orientation. For example, just like in the coherent motion direction tasks used by others, 819 the angle of the coherent orientation in our task was completely unpredictable. Thus, 820 participants in our task and in other tasks could not form a template of an orientation or 821 motion direction they should attend in advance and instead had to attend to an orthogonal 822 feature dimension such as color. In addition, in both prior tasks and the current task there 823were an equal number of coherent events in the cued and uncued color. If participants 824 failed to attend to the cued color and instead responded to any orientation event, their 825 performance in the task would be at chance. 826Behavioral performance in the reviewed studies ranged from d' = 0.8 to d' = 3.25 827 (Table S5 ). In many studies, performance was quite high (d' > 2 or accuracy > 90%) 828 relative to performance in our study (Adamian et In this pre-registered study, we sought to test whether cuing participants to ignore However, we failed to replicate this basic overall attention effect; we found no difference 847 in SSVEP amplitude as a function of attention in either the attend cue or the ignore cue