key: cord-103358-1hbw3yo3 authors: Gisbert-Muñoz, Sandra; Quiñones, Ileana; Amoruso, Lucia; Timofeeva, Polina; Geng, Shuang; Boudelaa, Sami; Pomposo, Iñigo; Gil-Robles, Santiago; Carreiras, Manuel title: MULTIMAP: Multilingual picture naming test for mapping eloquent areas during awake surgeries date: 2020-06-08 journal: bioRxiv DOI: 10.1101/2020.02.20.957282 sha: doc_id: 103358 cord_uid: 1hbw3yo3 Picture naming tasks are currently the gold standard for identifying and preserving language-related areas during awake brain surgery. With multilingual populations increasing worldwide, patients frequently need to be tested in more than one language. There is still no reliable testing instrument, as the available batteries have been developed for specific languages. Heterogeneity in the selection criteria for stimuli leads to differences, for example, in the size, color, image quality, and even names associated with pictures, making direct cross-linguistic comparisons difficult. Here we present MULTIMAP, a new multilingual picture naming test for mapping eloquent areas during awake brain surgery. Recognizing that the distinction between nouns and verbs is necessary for detailed and precise language mapping, MULTIMAP consists of a database of 218 standardized color pictures representing both objects and actions. These images have been tested for name agreement with speakers of Spanish, Basque, Catalan, Italian, French, English, Mandarin Chinese, and Arabic, and have been controlled for relevant linguistic features in cross-language combinations. The MULTIMAP test for objects and verbs represents an alternative to the DO 80 monolingual pictorial set currently used in language mapping, providing an open-source, standardized set of up-to-date pictures, where relevant linguistic variables across several languages have been taken into account in picture creation and selection. Human language is a complex system of communication that supports the decoding, encoding, and transfer of information between individuals. It is a system that allows for communication not only about the here and now, but also about the past, the future, truths, lies, hopes, and desires. It is important for personal growth and socialization, but also for human development as the vehicle for cultural transmission. Losing or having impaired language ability can be a traumatic event that incurs hardship for affected individuals and those around them. Brain surgery procedures can unintentionally damage the language substrate, inducing impairments that can be irreversible (Duffau et al., 2005) . For this reason, a patient is tested to identify eloquent areas that should not be removed in order to preserve or, in some cases, even improve, their quality of life (Ilmberger et al., 2008) . Since the late 1970s, when Ojemann and Mateer first reported using a visual object naming test during cortical stimulation in awake surgery (Ojemann & Mateer, 1979) , the technique has become the gold standard in testing brain lesions involving language-related areas (De Witte & Mariën, 2013; Miceli, Capasso, Monti, Santini, & Talacchi, 2012) . This language mapping procedure allows for the assessment of different language-related operations (i.e., access, retrieval and production of lexical-semantic information); the patient, presented with a series of drawings or pictures, is asked to name the depicted objects using a noun. Although the use of object naming tasks is widespread across surgical teams in many different geographical locations, heterogeneity in the stimuli selection criteria of previous batteries (i.e., differences in picture size, color, image quality, name agreement), in addition to the use of morphologically and typologically different languages across different studies (see supplementary material for a table review), has greatly hindered the comparison and generalization of results. To our knowledge, a battery allowing for direct comparison between two languages in awake surgery has never been designed, even if some teams have tested multilingual patients in more than one language (Giussani, Roux, Lubrano, Gaini, & Bello, 2007) . The critical need for a multilingual approach is evident given reports on bilingual aphasia where, in some cases, patients proficient in two languages prior to the lesion are selectively impaired in one after surgery (Fabbro, 2001) . It is also known from the aphasic literature that linguistic variables, such as word frequency and imageability, among others, influence lexical performance (Luzzatti et al., 2002) . However, multilingual brain stimulation studies have not reported how they controlled stimuli across languages, despite the fact that this choice of stimuli could influence findings concerning the brain areas that are common or specific to these languages (Bello & Acerbi, 2006; Cervenka, Boatman-Reich, Ward, Franaszczuk, & Crone, 2011; Roux, Lauwers-Cances, Trémoulet, Mascott, & Démonet, 2004) . For these reasons, and given that the multilingual population in our society continues to increase as more and more people know and employ a second language in their daily lives, a multilingual evaluation tool has become necessary (EuroStat, 2015) . In this paper, we present MULTIMAP, a new multilingual battery of standardized pictures in Spanish, with norms for Basque, Catalan, Italian, French, English, Mandarin Chinese, and Arabic. This is a tool that will allow surgical teams to test patients in bilingual contexts in a controlled and comparable manner. We first conducted a systematic revision of the literature on naming tasks for awake surgery (see supplementary material for a table review). Out of the 52 articles that reported using an object naming task, 10 included an introductory sentence (e.g. "This is…") printed above the picture to elicit the production of a determiner-noun pair (e.g. "This is… an apple" instead of "apple") (Hamberger et al., 2016; Hamberger, Seidel, Goodman, & McKhann, 2010; Ille et al., 2015; Khan, Herbet, Moritz-Gasser, & Duffau, 2014; Lubrano, Roux, & Démonet, 2004; Moritz-Gasser & Duffau, 2009; G. Ojemann & Mateer, 1979; Roux, Borsa, & Démonet, 2009; Roux, Boukhatem, Draper, Sacko, & Démonet, 2009; Rutten, 2015) . Patients are required to overtly produce grammatical information related to the selection of the appropriate determiner, as it encodes number and, in some languages, also gender information (i.e., in Spanish "unaf.sg. manzana f.sg." ["an apple"]). This kind of task allows for the identification of different types of errors, most frequently: speech arrest, in which the patient is unable to speak; and anomia, where the patient can read the introductory phrase, but cannot retrieve and produce the noun1. In addition to the object naming task, some neurosurgery teams have introduced verb tests in their practices. We retrieved 13 studies that reported the use of images to elicit verb production (Chen, Tan, Deng, & Xu, 2010; Conner, Chen, Pieters, & Tandon, 2014; Corina et al., 2005; 1 Other errors that can be identified using this kind of task are (1) semantic paraphasia if, instead of the target noun, a semantically related word is produced (e.g. "tiger" for "lion"); (2) phonological paraphasia, when the patient produces the target word with phonological deviations (e.g. "fable" for "table"); (3) neologism creation, inventing new word; or (4) perseveration, if the patient repeats an item, even after a new image is presented. Other significant limitations can also be detected using this task, such as delays in producing a response and hesitations. Havas et al., 2015; Herholz et al., 1997; Lubrano, Filleron, Démonet, & Roux, 2014; J. G. Ojemann, Ojemann, & Lettich, 2002b; Papagno et al., 2011; Rofes et al., 2017; Roux et al., 2003; Sierpowska et al., 2015; Skrap, Marin, Ius, Fabbro, & Tomasino, 2016; Tomasino et al., 2014) . In these tasks, patients were presented with a drawing or a picture of an action and they were asked to produce the appropriate verb: either a finite or an infinitive form, depending on the specific requirements of the task. The generation of finite verbs, unlike infinitives, requires the production of inflectional features that, depending on the language, may encode for number, gender, person, and/or time. Verbs refer to events and imply the projection of a complex representation in which the agent is associated with a specific thematic role and they differ from nouns at the lexical, semantic, morphological, and syntactic levels. In addition, the number of morphologically inflected forms is higher for verbs than for nouns. This distinction between nouns and verbs has been demonstrated at the behavioral, electrophysiological, and neuroanatomical levels (Vigliocco, Vinson, Druks, Barber, & Cappa, 2011) , demonstrating a double dissociation that should be taken into account when planning an awake surgery. Notably, direct cortical stimulation studies also show this double dissociation when object and action naming tasks are used, and allow for the identification of distinct territories in frontal and temporal brain areas in which stimulation selectively impairs verb or noun production (Corina et al., 2005; Crepaldi, Berlingeri, Paulesu, & Luzzatti, 2011; Lubrano et al., 2014; J. G. Ojemann, Ojemann, & Lettich, 2002a; Rofes et al., 2017) . From these studies, it seems clear that comparing nouns and verbs is necessary for a detailed and precise language mapping procedure. Several instruments have been previously employed in this endeavor. The most common batteries reported for mapping nouns are the Oral Denomination 80 (Metz-Lutz, Kremin, & Deloche, 1991) (DO 80); the images included in the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1972) (BDAE); and the pictures from the Snodgrass and Vanderwaart battery (Snodgrass & Vanderwart, 1980) . These sets of images were not designed for intraoperative language mapping, and name agreement norms and values for relevant linguistic variables, such as word frequency, length, or familiarity, are not provided or are only available for some of the languages in which they have been employed. In addition, payment is required to access both the DO 80 and the BDAE, restricting their use. In some other studies using intraoperative object naming, stimuli are only described as "black-and-white drawings", "simple drawings" or just "pictures representing common objects", and no further information is given about their construction or the linguistic variables of the target words. In studies that have carried out language mapping using verbs, some teams have reported using the same pictures for verb action naming that they use in their object naming tasks (Papagno et al., 2011) , while others have presented pictures depicting actions (Havas et al., 2015) . For verbs, as for nouns, only some studies have reported controlling their stimuli for the relevant linguistic variables or have described how these stimuli were constructed and selected. To address these shortcomings and the critical demand for multilingual material, we developed a multilingual picture naming test (MULTIMAP) for the mapping of eloquent areas during awake brain surgery. MULTIMAP consists of a database of standardized color pictures of common objects and actions, tested for name agreement measures in speakers of Spanish, Basque, Catalan, Italian, French, English, Mandarin Chinese, and Arabic. In the MULTIMAP object and action naming subtests, items were independently selected for each language and controlled for a number of linguistic variables (i.e. frequency, length, and familiarity). We also equated variables for Spanish and each of the other languages separately to facilitate testing in multilingual patients. This new battery will help teams plan surgical interventions, providing them with a sensitive, validated instrument for intraoperative language mapping offering better results in terms of the patient's health and overall quality of life. In the following sections, we include a detailed description of the material and its validation in order to facilitate replication and extension to other languages in future research. One hundred and twenty-three healthy adult volunteers aged 18 to 60 years were recruited and paid for the object naming task, and 124, within the same age range, for the verb naming task. A detailed description of the sample used for each language is included in Table 1 (i.e., sample size, ages, gender, educational level, and linguistic profile). They all had normal or corrected to normal vision and no history of psychiatric, neurological diseases, or learning disabilities. A signed informed consent was collected from each participant as stipulated in the ethics approval procedure of the BCBL2 Research Ethics Committee. All of the participants were from the Basque Country in Spain and were highly proficient Spanish-Basque bilinguals. centimeters; DPI = 300 pixels/centimeter) and similar styles for the whole set. Verbs were depicted with a human agent (see Figure 1 for an example). Figure 1: Example of stimulus presentation for both object and action naming. An acoustic cue is presented during the fixation cross, 500ms before the stimulus onset. The fixation cross appears on the screen for 1s, followed by the target picture which appears for 1 to 4s, self-paced as determined by the speed of the patient's response. The name agreement data was collected using the Lime Survey platform Phase 2 (Validation). Once we had the final subset of 88 images, we prepared the images for use in the awake surgery setting. Above each object, we added the text "Esto es…" ["This is…" in Spanish] to force participants to produce a short sentence that would have to agree in number and gender with the target noun. Above the action pictures, we included a noun phrase to be used as the subject of the sentence, that is either "Él…" or "Ella……" ["He…" or "She…" in Spanish] depending on the gender of the agent. This introductory text was used as a cue for the production of a sentence that started with the given subject and had a finite verb form in 3rd person singular. We used MATLAB version 2012b and Cogent Toolbox (http://www.vislab.ucl.ac.uk/cogent.php) to present the images, as they would be used in a surgery setting. First, a white screen with a black fixation cross appears for 1s, followed by a picture presented for 4s. An acoustic cue is given 500ms before the onset of each stimulus during the fixation cross (the Matlab script and its compiled version are available for use at https://git.bcbl.eu/sgisbert/multimap2). Phase 3 (Cross-language combinations). Seven norming studies were carried out. The first one included the Spanish set of stimuli, described in Phase 1, and the other five comprised the different cross-language combinations (i.e., Spanish-Basque, Spanish-Catalan, Spanish-Italian, Spanish-French, and Spanish-English). For each of these combinations, pictures were tested for name agreement, following the same procedure described in Phase 1, in samples of highly proficient users of the languages. Frequency, length, and information about orthographic neighbors were extracted for the words in each language from the following databases: for Basque, e-Hitz (Perea et al., 2006) Arabic, Aralex (Boudelaa & Marslen-Wilson, 2010) . Stimuli with at least 80% name agreement were selected so that nouns and verbs did not show significant differences in frequency, number of letters, picture-name agreement, and H-index, both within and between the two languages of each pair. Depending on the language, sentence production requires the use of different grammatical devices (e.g., case markings, word order, inflectional morphology, etc.). Thus, in addition to orthographical and lexical factors, some syntactic constraints were applied. For all languages, we took out plural invariant nouns (i.e. "Tijeras", scissors) and kept only transitive verbs. For each of the cross-language combinations, we removed cognates. Data processing. A native speaker of each language checked the answers for writing/spelling errors, standardized the writing using capital letters, and merged basic variants of the same target word (e.g. hyphenated, pluralized forms). After that, we excluded trials where participants did not know the name or did not recognize the concept from all analyses. Name agreement and Hindex (Snodgrass & Vanderwart, 1980) were then computed for the remaining 109 nouns and 109 verbs. Both of these measures reflect the level of agreement across participants: while name agreement represents the percentage of participants who give a certain answer, the H-index reflects response variability in terms of the number of different answers given by participants. Given the name agreement data for each language, we extracted items with a score of at least 80%. With these items, we first created a list for each language, selecting the same number of nouns and verbs, and ensuring that there were no significant differences across the linguistic values we controlled for (i.e. frequency, length, orthographic neighbors) using unpaired twosample T tests. Next, we paired each of the languages with Spanish to create the bilingual tasks. Here, we selected new items from the 80% name agreement list to create lists of the same length between languages, again controlled for frequency, H-index, length, and orthographic neighbors (except for the combinations Spanish-Mandarin Chinese, were length and orthographic neighbors were not contemplated, and Spanish-Arabic, where orthographic neighbors were not contemplated). We calculated ANOVAs between the four lists (objects language 1 x objects language 2 x verbs language 1 x verbs language 2) to check that there were no significant differences in the relevant variables. We tested 109 pictures of objects and 109 pictures of actions in a Spanish speaking sample. From this picture pool with a constraint of at least 80% name agreement, we selected 44 objects and 44 verbs that showed no significant differences in name agreement, frequency per million, number of letters, number of phonemes, number of syllables, and familiarity (see Table 2 for an overview of the variables and t-test results comparing objects and verbs within languages). The values for imageability and concreteness showed statistically significant differences, with objects higher than verbs in both cases. Imageability can be defined as the ease with which a word brings to mind a sensory image, while concreteness is the property of being able to see, hear and touch something (Bird, Howard, & Franklin, 2003) . Verbs as a class are by definition less imageable and concrete than objects, given that objects that can be drawn have stronger sensory features than action verbs, which are primarily functional and motoric (Bird, Howard, & Franklin, 2000) . Phase 3 (Cross-language combinations). For bilingual lists, objects and verbs in both languages were always equated for name agreement (higher than 80% in all cases) and controlled so that linguistic variables did not show significant differences (see Table 3 Language is the main vehicle humans use to communicate and transfer information. Groups of neurons arranged in networks support language functions. Any modification to this system (e.g., brain lesions, epilepsy seizure, etc.) may irreversibly impair language capacity, leading to unexpected and problematic consequences for the affected individual. For this reason, brain surgeries are increasingly planned in an awake setting, making it possible to monitor patients' language function and spare brain tissue that is indispensable (De Witte & Mariën, 2013) . There is a critical need for specific tools, based on linguistic, neuroscientific, and clinical knowledge about how the brain decodes and encodes linguistic information, that can facilitate precise mapping of these eloquent regions during neurosurgery. To address this need, we developed the MULTIMAP test, a multilingual picture naming task including both objects and actions for mapping eloquent areas during awake brain surgery. Images included in the MULTIMAP test are colored drawings of objects and actions that have In spite of empirical evidence demonstrating neuroanatomical distinctions for nouns and verbs (Vigliocco et al., 2011) and native and second/third languages (Giussani et al., 2007) , there is no available material that tackles both of these issues at the same time. In this regard, MULTIMAP constitutes the first tool designed to explore these factors in a structured way, in order to identify and preserve eloquent brain tissue, and to obtain better results in terms of patients' health and overall quality of life. MULTIMAP includes two separate tests, one for objects and one for actions to address the noun/verb issue; both tests are controlled so that there are no significant differences for linguistic variables such as frequency, length, and orthographic neighbors. The need to map both objects and actions is motivated by evidence of a double dissociation, demonstrated at the behavioral, electrophysiological, and neuroanatomical levels (Vigliocco et al., 2011) and, as pointed out in the Introduction, also reported in direct cortical stimulation studies. From the 52 reviewed studies where a picture naming task had been used in the context of awake brain surgery, we found only 13 that included a verb task in addition to object naming. Although these tasks were varied in their requirements and the nature of the stimuli they employed, they all identified distinct territories, mainly in frontal and temporal brain areas where stimulation impaired verb and noun production separately (Corina et al., 2005; Crepaldi et al., 2011; Lubrano et al., 2014; Ojemann et al., 2002a; Rofes et al., 2017) . Moreover, our tasks include an extra level of complexity beyond the extraction of morphosyntactic information, as the production of target objects and actions have been embedded in simple sentences, such as "This is a house" for objects, and "He/she sings" in the case of actions. This entails a higher level of complexity since the generation of such sentences requires the projection of representations in which thematic roles are assigned to different elements in the sentence (i.e., Heagent sings), in addition to matching the target word to its real-world referent. Therefore, combining object and verb processing tasks at the sentence level ensures a more accurate and thorough mapping of language functions, helping to more accurately identify and preserve the neural linguistic substrate essential for a patient's quality of life. The second improvement offered by MULTIMAP is its multilingual nature. Neuroimaging studies in bilinguals suggest that there is a common cerebral organization across languages, but also describe activations specific to each of the languages (Kim, Relkin, Lee, & Hirsch, 1997; Marian, Spivey, & Hirsch, 2003; Rueckl et al., 2015) . Direct electrostimulation studies have also revealed language-specific areas (Giussani et al., 2007) .This study concluded that multilingual patients should be tested in all languages in which they are fluent during brain mapping procedures so as to avoid selective or preferential impairments. With this objective in mind, the images included in MULTIMAP were tested in seven languages taking into account the lexical and morphological features of each language. This resulted in seven separate sets of object and action pictures with at least 80% name agreement in their target language, accounting for relevant linguistic variables like frequency and word length. These sets can be combined in controlled bilingual sets, enabling researchers from different countries to use the same materials, and to compare results not only from monolingual samples but also in cross-linguistic research on multilingual patients. It will also play a role in the postoperative quality of life for multilingual patients, as these materials will facilitate the identification and preservation of areas where interference impairs only one of their languages (Giussani et al., 2007) , areas that might not be detected by monolingual tests. EsploraCoLFIS: Un'interfaccia web per le ricerche sul Corpus e Lessico di Frequenza dell'Italiano Scritto (CoLFIS) Intraoperative Language Lozalization in Multilingual Patients with Gliomas Why is a verb like an inanimate object? Grammatical category and semantic category deficits Verbs and nouns: The importance of being imageable Aralex: A lexical database for modern standard Arabic Language Mapping in Multilingual Patients: Electrocorticography and Cortical Stimulation During Naming Syntactic Prediction in Sentence Reading: Evidence from Eye Movements Category specific spatial dissociations of parallel processes underlying visual naming Dissociation of action and object naming: Evidence from cortical stimulation mapping A place for nouns and a place for verbs? A critical review of neurocognitive data on grammatical-class effects The neurolinguistic approach to awake surgery reviewed EsPal: one-stop shopping for Spanish word properties Contribution of intraoperative electrical stimulations in surgery of low grade gliomas: a comparative study between two series without (1985-96) and with (1996-2003) functional mapping in the same institution Foreign Language Skills Statistics The bilingual brain: Cerebral representation of languages Review of language organisation in bilingual patients: What can we learn from direct brain mapping? The assessment of aphasia and related disorders NIM: A Web-based Swiss army knife to select stimuli for psycholinguistic studies Functional differences among stimulation-identified cortical naming sites in the temporal region Does cortical mapping protect naming if surgery includes hippocampal resection? Electrical stimulation mapping of nouns and verbs in Broca's area Preoperative activation and intraoperative stimulation of languagerelated areas in patients with glioma Combined noninvasive language mapping by navigated transcranial magnetic stimulation and functional MRI and its comparison with direct cortical stimulation Intraoperative mapping of language functions: a longitudinal neurolinguistic analysis The role of left inferior frontooccipital fascicle in verbal perseveration: A brain electrostimulation mapping study Distinct cortical areas associated with native and second languages Anatomical correlates for category-specific naming of objects and actions: A brain stimulation mapping study Writing-specific sites in frontal areas: a cortical stimulation study Verb-noun double dissociation in aphasic lexical impairments: the role of word frequency and imageability Shared and separate systems in bilingual language processing: Converging evidence from eyetracking and brain imaging Standardisation d'un test de dénomination orale: contrôle des effets de l'âge, du sexe et du niveau de scolarité chez les sujets adultes normaux Language testing in brain tumor patients Evidence of a large-scale network underlying language switching: a brain stimulation study Human language cortex: localization of memory, syntax, and sequential motor-phoneme identification systems Cortical stimulation mapping of language cortex by using a verb generation task: effects of learning and comparison to mapping based on object naming Cortical stimulation mapping of language cortex by using a verb generation task: effects of learning and comparison to mapping based on object naming What is the role of the uncinate fasciculus? Surgical removal and proper name retrieval E-Hitz: A word frequency list and a program for deriving psycholinguistic statistics in an agglutinative language (Basque) Mapping nouns and finite verbs in left hemisphere tumors: a direct electrical stimulation study The Mute Who Can Sing": a cortical stimulation study on singing Cortical calculation localization using electrostimulation Language functional magnetic resonance imaging in preoperative assessment of language areas: Correlation with direct cortical stimulation Intraoperative mapping of cortical areas involved in reading in mono-and bilingual patients Universal brain signature of proficient reading: Evidence from four contrasting languages Speech hastening during electrical stimulation of left premotor cortex The Glasgow Norms: Ratings of 5,500 words on nine scales Morphological derivation overflow as a result of disruption of the left frontal aslant white matter tract Brain mapping: a novel intraoperative neuropsychological approach A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity Involuntary switching into the native language induced by electrocortical stimulation of the superior temporal gyrus: A multimodal mapping study Nouns and verbs in the brain: A review of behavioural, electrophysiological, neuropsychological and imaging studies We would like to thank the BCBL Lab Department for data recordings and Magda Altman for her useful comments on the manuscript.