503 Dissonance between Perceptions and Use of Virtual Reference Methods Tara Mawhinney and Sandy Hervieux* This multimethod study investigates differences in question complexity and type between live chat, email, and texting by comparing findings from user interviews and virtual reference transcripts, with the goal of better understanding how different delivery methods can meet user needs in the context of an academic library. Find- ings reveal dissonance between perceptions and use of chat and email. Interviews suggest users consider chat to be for basic queries whereas transcripts coded using the READ Scale, a well-known reference assessment tool, show question complex- ity to be highest in chat. Our analysis also found statistically significant differences in the presence of reference interviews and instruction for chat, email, and texting. Rebranding chat more explicitly for intermediate and advanced queries may succeed in attracting users who consider chat only for basic queries, thus narrowing the gap between user perceptions and actual use. Introduction This research study compares users’ perceptions of different virtual reference methods with how they actually use them in the context of a research-intensive academic library. In the current climate, with COVID-19 rendering virtual reference a necessity in many libraries, understanding and meeting user needs is more important than ever. We investigated differences in question type and complexity between virtual reference methods with the goals of better understand- ing how they can meet different user needs and of identifying best practices for staffing and promotion. The current study is part of a larger research project, investigating preferences and differences among virtual reference methods. In Phase One, the first author analyzed user preferences and factors that account for them from among different virtual reference methods. She conducted interviews with users and reported findings in a previous publication.1 The cur- rent study (Phase Two) is a continuation of the first and investigates differences among chat, email, and texting using two methods: through users’ perspectives as collected during inter- views conducted in Phase One and through content analysis of virtual reference transcripts. In short, in this phase, we examined user perceptions of the differences among chat, email, and texting in a library context using data from the in-person interviews, not yet reported in the Tara Mawhinney is Liaison Librarian at McGill University; email: tara.mawhinney@mcgill.ca. Sandy Hervieux is Liaison Librarian for Political Science, Philosophy and the School of Religious Studies at McGill University; email: sandy.hervieux@mcgill.ca. ©2022 Tara Mawhinney and Sandy Hervieux, Attribution-NonCommercial (https://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC. mailto:tara.mawhinney@mcgill.ca mailto:sandy.hervieux@mcgill.ca https://creativecommons.org/licenses/by-nc/4.0/ 504 College & Research Libraries May 2022 first study. We then compared these findings with what transcript analysis showed to be the differences in question type, question complexity, and the presence of reference interviews and instruction among chat, email, and texting. The current study is unique in its approach of comparing user perceptions with actual use and aims to narrow the gap between them by improving the promotion and delivery of virtual reference services. The findings will be most applicable to university library settings that offer similar virtual reference services. Literature Review In the early literature on virtual reference, there was a perception that chat was not for “real” reference questions,2 that in-person reference was more suitable than virtual reference for re- search questions, especially when instruction was required,3 and that virtual reference comprised fewer research-related queries than in-person reference.4 More recently, the idea that virtual reference is less suitable for and generates fewer complex queries than in-person reference has largely been disproven, with several studies showing virtual reference queries to be generally more advanced than in-person ones.5 Of the predominant types of questions asked, research has found that directional questions account for the largest category for face-to-face reference, while reference questions account for the largest category in email and chat.6 Research conducted at an institution employing proactive chat (the type of chat that opens automatically as a pop-up window on library webpages inviting users to ask a question) shows the difference in question complexity between in-person and virtual reference queries to be even more marked.7 Whereas research shows clear differences in question complexity between virtual and in-person queries, distinctions among chat, email, and texting draw less conclusive findings. While research conducted on users’ perceptions of texting8 and content analysis of texting transcripts9 show this method to be for basic and quick-answer type queries, it seems less clear which generates more complex queries from among chat and email. There is a percep- tion that chat is fit for basic inquiries and not suitable for research-type questions. Dempsey addresses the debate in the literature by asking: “Is chat a mode of reference service that is suited to answer subject research questions, and to what extent do patrons expect this kind of service?” She claims that further research is needed to better understand the expectations of the scope of chat reference, especially in terms of more advanced questions such as those involving subject research.10 Similarly, Chow and Croxton note that chat is perceived to be for factual and directional rather than for reference questions.11 However, their research shows nuance, pointing to demographic differences in perceptions of virtual reference methods, with undergraduate students ranking chat and email equally suitable for reference and research questions, while faculty, university staff, and graduate students prefer email for such queries.12 In contrast, Ward’s research suggests that both undergraduate and graduate students perceive chat to be for all kinds of questions, including research-related ones.13 The idea that chat is unsuitable for delivering reference originates not only from our us- ers but also from librarians ourselves. In comparing email and chat, Kern notes: “Research assistance is highest for chat, even though we do not consider this an effective use of chat. How do we respond to this demand?”14 Similarly, Greenberg and Bar-Ilan’s research shows that librarians do not view chat to be conducive to research questions and find in-person ref- erence to be better for users, although they acknowledge that it depends on users and their individual needs.15 Dissonance between Perceptions and Use of Virtual Reference Methods 505 How do we make sense of this dichotomy if, in fact, chat contains questions that are more advanced and more of a kind that are generally regarded as research or subject-related than other methods? These perceptions affect both how we deliver and advertise virtual reference services. For example, knowing that certain virtual reference methods result in more complex questions may lead to staffing these methods differently from others and/or more explicitly training staff to consider the method when responding to users. If certain methods are more conducive to complex questions than others, libraries may also choose to advertise certain methods more prominently or have them present in different locations on the library website. On the one hand, chat favors reference-type interactions because the technology allows us to engage in reference practices that match well with what we traditionally do at the reference desk by engaging in reference interviews and providing instruction. Yet, in the minds of some (both users and librarians), it is not a medium that is fit for this purpose. Further research is needed to better understand the differences among chat, email, and texting, in terms of the questions asked, their complexity, and what the role of reference interviews and instruction play in each. First, it is important to consider if user and librarian perceptions are well founded. In this regard, studies suggest two contradictory sets of results. The first set suggests that these perceptions may be accurate, with two relatively small-scale studies reporting chat interac- tions to be less complex than email. Greenberg and Bar-Ilan analyzed 116 chat and 213 email interactions from an academic library in Israel and found more email than chat to be in-depth questions (52% of email compared to 21% of chat interactions), leading the authors to conclude that email is preferable for research questions.16 Similarly, Lee’s study comparing 47 chat and 47 email transactions from Murdoch University in Australia concluded that chat may not be suitable for delivering reference services, especially for answering queries involving instruc- tion, despite that chat and email had equal numbers of research and reference questions. His reasoning was that chat interactions lacked many of the aspects needed for a successful reference interaction.17 Although Lee does not specify exactly what aspects he is referring to, some of them likely originate from the time in which he was writing about chat in the early 2000s. For example, he mentions that 34 percent of chat interactions were not successful due to connectivity issues. Technology-related problems within chat have been vastly reduced since the time his article was published. Another aspect that Lee might be referring to is his uncertainty about whether or not users wanted librarians to conduct a reference interview in the context of chat, speculating that they might find this practice too intrusive. More recent research, however, suggests that reference interviews play an important role in virtual refer- ence, as discussed in detail below. One final aspect he refers to is his concern that users might find chat to be impersonal. Although valid, this issue may have dissipated over time with the increased use of many forms of instant messaging in our daily lives such as texting, instant messaging on social media sites like Facebook and other platforms. Furthermore, librarians have found ways to overcome concerns over the impersonal nature of this method by using techniques such as “syntactic mirroring” where librarians mirror users’ level of formality to help users to feel more at ease during virtual reference interactions.18 In contrast, a second set of research studies, consisting primarily of larger-scale ones, reports chat interactions to be more complex than email. Gerlich and Berard examined data from 14 institutions, comparing reference question complexity across various methods (chat, email, telephone, and in person) using the READ (Reference Effort Assessment Data) Scale, 506 College & Research Libraries May 2022 a well-known assessment tool developed by one of the study’s authors. The tool uses a six- point scale to compare question-level complexity, from level 1 being little to no skill or effort required to level 6 being where staff provide in-depth research assistance to users, requiring a great deal of effort, expertise, and time. They determined that there were more advanced questions in chat than email, with 67 percent of chat compared with 42 percent of email be- ing READ 3 and above.19 Likewise, Fennewald’s study comparing 751 email and 405 chat transactions reported that reference questions, the most advanced category in his classifica- tion, accounted for 72 percent of chat and 60 percent of email questions.20 Moreover, Kern’s study, at the University of Illinois Urbana-Champaign analyzing 1,109 chat, email, telephone, and in-person queries, found 30 percent of chat compared to 24 percent of email questions to be research assistance, the category generally considered to be the most complex.21 The first author’s previous research comparing the differences in question level and type for 390 transcripts from McGill University found similar results, with intermediate-level questions accounting for 59 percent of chat and 51 percent of email.22 Similarly, Ward and Phetteplace’s study of chat, email, in-person, and phone questions found chat to be dominant in terms of question difficulty, having become increasingly complex and lengthy over time.23 Given the debate in the literature, the issue of the complexity and types of questions generated in email and chat warrants further consideration. The READ Scale The READ Scale has been widely employed in academic libraries for analyzing virtual refer- ence transactions.24 For analyzing question complexity, it provides enhanced granularity over categorizations such as basic, intermediate, and advanced used in other studies.25 Gerlich and Berard report that the majority of staff in the 14 libraries from their study found it added value to their reference statistics and recommend its usage.26 Previous literature has interpreted the READ Scale differently but researchers often consider READ 3 and above questions to be complex.27 Ward and Phetteplace suggest that READ 3 level questions should be answered by staff with specific reference training, presumably librarians.28 Similarly, research recommends that part-time and student workers only answer READ 1 and 2 level questions and refer higher level ones to librarians.29 Meanwhile, some research recommends that staff answering READ 3 questions have reference knowledge and skills but suggests that only READ 4 and above questions require librarian expertise.30 Studies using the READ Scale report varying degrees of complexity for chat interactions and draw different staffing conclusions based on their findings. Several studies report sizable numbers of READ 3 and above chats, accounting for percentages ranging from 2831 to 6032 to 81 percent33 of all questions received via this method. Despite recommendations outlined above of having librarians respond to READ 3 and above questions, some studies recommend having well- trained students or paraprofessionals staff chat services.34 It is clear that question complexity varies greatly in virtual reference, even when measuring using a common method of assess- ment such as the READ Scale, and that libraries’ subsequent staffing decisions also show wide variation. Reference Interviews and Instruction It is important to consider the frequency and the means by which two regular practices with reference services in general, reference interviews and instruction, occur within different Dissonance between Perceptions and Use of Virtual Reference Methods 507 methods of virtual reference. Previous research has investigated how they fit into the virtual reference landscape, with some perspectives having changed over time and others currently being debated in the literature. Conducting a reference interview is a hallmark of reference librarianship and is an important practice in delivering assistance to users,35 as it is understood that a user’s initial question may or may not reflect their true information need. Although researchers initially expressed ambivalence about reference interviews within virtual refer- ence due to communication challenges,36 more recent research has solidified their importance, with evidence showing them to be a best practice within virtual reference.37 Despite research showing that query clarification considerably improves accuracy of responses,38 Logan, Bar- rett, and Pagotto report that users are less satisfied when staff ask them clarifying questions. The researchers speculate that, although users want to be understood, they find it frustrating to explain themselves quickly and clearly within chat. Despite users’ ambivalence toward reference interviews, the researchers recommend that staff members conduct them to best understand users’ needs.39 Similarly, instruction is foundational within reference services, especially in the academic library context. Perspectives on its role have also changed, with early research questioning its place within virtual reference.40 However, more recent literature demonstrates chat to be a suitable venue for instruction,41 with librarians delivering it and users being receptive to it.42 Dempsey reviewed several studies and claimed that chat transcripts on average include instruction 50%–86% of the time,43 while other research finds that it could take place even more frequently than it does.44 Contribution to the Literature Given the current context of many physical libraries being closed due to COVID-19, virtual reference has become an essential service and, in many cases, the only one that our users can access. As a result, understanding the nuances of virtual reference methods is more important than ever. Whereas several studies outlined in the literature review examine the different types and/or complexity of questions in one or two virtual reference methods, studies comparing three or more methods are rare.45 Furthermore, few virtual reference studies combine multiple methods of analysis, such as transcript analysis and surveys,46 and none combine interview findings and transcript analysis. The current study contributes to the literature by: • Examining users’ perceptions of what virtual reference methods are for and comparing these perceptions with findings from analysis of virtual reference transcripts; • Analyzing question complexity as measured using the READ Scale, which has not yet been done to compare three methods of virtual reference. Research Questions The purpose of this multimethod study is to consider what differences exist between virtual reference methods and compare what these methods of virtual reference are for, both from a user perspective and through an analysis of virtual reference transcripts. We ask the follow- ing research questions: • Research Question 1: What do virtual reference users perceive chat, email, and texting to be for in terms of the level of complexity and categories of questions asked? • Research Question 2: Is there a statistically significant relationship between virtual reference delivery method and the level of complexity of questions asked, as measured 508 College & Research Libraries May 2022 using the READ Scale? • Research Question 3: Is there a statistically significant relationship between virtual reference delivery method and categories of questions asked? If so, what are the most prominent categories for each method? • Research Question 4: Does the presence of reference interviews and instruction differ across virtual reference methods? If so, are the associations between the variables sta- tistically significant? Institutional Context McGill University Library makes a very good case study for investigating different methods of virtual reference because it is part of a research-intensive university and has a long track record offering a variety of virtual reference services. McGill University is a publicly funded institution in Montreal, Canada, with an enrollment of 40,000 students.47 McGill Library em- ploys a nonconsortial model of virtual reference with 14 years of service history (chat and email), provided primarily by liaison librarians and supplementing the service with two or three graduate student workers from the library and information studies program each year, beginning in 2016. The library implemented texting in summer 2016. In August 2019, McGill Library moved from QuestionPoint to LibChat when the latter bought out the former. The library saw marked increases in virtual reference use in the first years of service and has seen steady use since, with more recent years hovering around 6,000 interactions per year. In 2018, 2,570 emails, 3,035 chats, and 195 text messages were received. In 2019, users sent 2,734 emails, 2,953 chats, and 198 text messages. In 2020, due to COVID-19 and the closure of physical refer- ence services, there was a substantial increase in the use of all virtual reference methods, with the library receiving 3,342 emails, 4,769 chats, and 341 text messages. The virtual reference services are marketed through the library’s social media accounts. They are also advertised by a banner on the library website. When liaison librarians visit classes or provide instruc- tion sessions, they encourage students to use the virtual reference services. Chat, email, and texting are marketed equally. Librarians may encourage students to email them directly for discipline-specific inquiries, but students may elect to use virtual reference to receive an im- mediate response. More information about users’ preferred methods of communication with the library can be found in a previously published study.48 Although the library does not have a mandate specific to virtual reference, part of the library’s mission is to facilitate excellence in teaching, learning, and research and to be client- focused, responding to student and faculty needs.49 We value our role in learning, encouraging instruction within virtual reference, and have developed best practices for instruction that are part of the professional development of virtual reference staff. The services are open to anyone, although policies on the website encourage questions from the general public to be related to the university or the library. There are no specific policies for the kinds of questions users can ask by chat, email, and texting. Methods This multimethod study uses two forms of data collection to analyze how library users per- ceive different methods of virtual reference and how they employ them. To answer Research Question 1, we investigated a user point of view by conducting interviews with users on their perceptions of virtual reference methods and the differences between them in terms of ques- Dissonance between Perceptions and Use of Virtual Reference Methods 509 tion complexity and question categories. We examined interview transcripts using grounded theory and subsequently compared the interview findings with statistical analysis of virtual reference transcripts. To answer Research Questions 2–4, we used methods similar to those used in previous research studies of seeking out expert assistance to conduct statistical analysis on virtual reference transcripts.50 We sought to determine if there were statistically significant associations between virtual reference delivery methods and several variables including the following: question level complexity, question category, and the presence of reference inter- views and instruction. The Chi-square test of independence was conducted, as well as Cramer’s V to determine the strength of associations. The study does not compare how users define complexity with how complexity is measured using the READ Scale. Rather, it compares users’ perceptions of different virtual reference methods’ suitability for complex questions with the level of complexity of virtual reference transcripts, as coded using the READ Scale. User Interviews We recognized that perceptions and actual use of virtual reference services could be quite different. If there were differences in users’ perceptions and what transcripts show that us- ers are actually asking via these methods, we wanted to examine them and consider what the implications might be for staffing, training, and promotion. For the interview portion of the research study, the first author obtained McGill University’s Research Ethics Board I Certificate of Ethical Acceptability of Research Involving Humans in fall 2018 and conducted interviews with 14 virtual reference users in winter 2019. Five undergraduate students, six graduate students, two faculty members, and one alumnus participated in the interview por- tion of the study (see appendix for a description of participants). As mentioned earlier, she analyzed these interviews and, in a previous publication,51 reported on participants’ prefer- ences from among different virtual reference methods. These methods consisted of chat, email to the general library, email to the liaison librarian, and texting. In the current study, the first author used NVivo software to analyze the same set of interviews, coding participants’ per- ceptions of these methods and of the differences between them. Participants were recruited through online and other methods, namely posters across campus, solicitation by email from a list of names gathered during library orientation, and solicitation from library staff during their email, text, and chat interactions. Library users were eligible to participate if they had used one or more virtual reference methods before. Interviews were conducted until the first author reached saturation, that is when very few new insights emerged from the interviews. The earlier publication on user preferences outlines the recruitment methods, study sample, and interview protocol in greater detail.52 Interview Questions The first author focused on responses to select interview questions reported in the first pub- lication, as follows: • Tell me about the last time you used chat, email to the general library, email to your liaison librarian, and/or texting. (Probes: Did you find that it was successful or unsuc- cessful? How did the chosen method impact the interaction?) • What factors do you consider when choosing from among virtual reference methods? (Probes: Does the type of question influence your choice of method? Does the device you are using influence your choice of method?) 510 College & Research Libraries May 2022 • From among the methods you have not used, which, if any, would you be interested in trying? What about this or these methods interests you? • Tell me about your expectations of the different virtual reference methods offered by the library. (Probes: Do you have different expectations in terms of level of expertise of staff, formality, response time, hours of service, and so on?) In the current study, she also analyzed participants’ responses to the following questions not reported in the earlier publication: • Would the level of difficulty and/or the type of question matter to you in determining which method to use? • What kinds of questions would you consider asking using each of the methods that interest you? • Describe how these methods of communicating with the library are different from each other. (Probes: How do they compare to each other? In terms of convenience? Speed?) Coding Using NVivo, the first author coded and analyzed the interviews in light of the questions related to participants’ perceptions, as well as past and anticipated uses of virtual reference methods. She used grounded theory to identify themes in the interviews. This method was appropriate because it allowed the codes to emerge from participants’ own words,53 and it allowed her to discover patterns within the data. Transcript Analysis For the transcript analysis portion of the research study, McGill University’s Research Ethics board informed us that this type of research did not require ethics approval. Both researchers participated in coding virtual reference transcripts drawn from a two-year timespan, carrying out the coding in NVivo and then exporting the data to Excel for sta- tistical analysis. The study drew on two years of chat, email, and texting transcripts from January 1, 2018 to December 31, 2019. There were 201 texting transcripts from Question- Point and 49 transcripts from LibChat during the two-year period. This lower number of texting transcripts as compared to those provided above in our annual statistics is due to several factors. QuestionPoint treated each texting response as a new transaction instead of threading them together. For coding purposes, we grouped the responses and counted them as a single transcript. We also removed incomplete transactions and practice ques- tions used for training purposes. Using methods similar to those in previous studies,54 we extracted a random sample of email and chat transcripts from the same time period to match the number of texting transcripts. Only emails received through the virtual reference platform were evaluated. Emails sent to librarians directly were not evaluated. In total, we analyzed 750 transcripts (250 transcripts from each method) for level of complexity, question category, and the presence of a reference interview and instruction. We employed the READ Scale to measure question complexity since using a commonly recognized mode of assessment allows the findings from the current study to be more easily interpreted in light of findings from other studies. To code the transactions, we drew on the READ Scale classification outlined by Gerlich and Berard,55 Maloney and Kemp’s overview of the READ Scale and other classification schemes used to measure question complexity,56 Dissonance between Perceptions and Use of Virtual Reference Methods 511 and examples of questions for the different READ Scale ratings from previous literature.57 We coded not only the user’s initial question but the whole interaction, since, in keeping with the goal of the READ Scale “to reflect the effort expanded, knowledge required, and even the teach- able moment that occurs during the transaction,”58 these factors influenced how the interactions were coded. For example, we coded requests for known items as READ 2 if the staff member simply replied by providing a link to the catalogue record. However, we coded the same request as READ 3 if the staff member determined that the user was located off campus and explained how to set up a remote connection for accessing resources. To facilitate ease of analysis, we coded each transcript with a single question category and READ score; if users asked multiple questions, we chose the question that was the most complex. If there were multiple questions that were each a READ 2 level, we coded the trans- action as READ 3 since the global time and effort involved was greater than a single READ 2 question. Our classification scheme was informed by categories used in previous virtual reference research.59 However, like Stieve and Wallace, we primarily used grounded theory to establish the classification scheme based on the types of questions users asked since it allowed us to discover patterns from within our dataset and did not restrict possibilities of the categories that would emerge.60 We used the constant comparison method, a feature of grounded theory, where data are coded and recoded as new themes emerge, to ensure that themes were applied consistently.61 We coded all transactions for the presence of a reference interview and instruction. We defined the reference interview as instances when staff members asked one or more prob- ing questions to help them better understand the user’s information need. Our definition for instruction was any instance where the staff member guided a user step by step, explained how a service or resource functioned (such as providing instructions for accessing an ebook), or engaged in troubleshooting. We included as instruction those instances where a staff mem- ber pointed to a source of information and provided some explanation but not when a staff member simply pointed to a webpage or the catalogue for information. We coded the questions independently. To enhance intercoder reliability, we generated a codebook and applied it consistently to all transcripts. Using methods similar to Greenberg and Bar-Ilan,62 both authors coded a random sample of 10 percent of the email, chat, and tex- ting transcripts. In comparing our coding, the match rate was 84 percent; after discussion, we reached 100 percent consensus. The first author coded the remaining transcripts. We conducted transcript analysis within NVivo by importing Excel spreadsheets into the software. One advantage of coding in NVivo was the ease with which we could create and modify question categories. NVivo also allowed us to easily cross-analyze different codes to determine, for example, the percentages of different question categories that contained instruction. There were two major drawbacks to using NVivo for transcript analysis, mostly pertaining to how well NVivo and Excel function together. Once we imported the Excel spreadsheets into NVivo, it was not possible to modify them; so, if an incomplete transac- tion was inadvertently included in a spreadsheet, it could not be removed. Therefore, it was necessary to keep detailed records about how many transcripts were actually coded in each spreadsheet. Also, although it was possible to export the coded transcripts back out of NVivo to run statistical analysis in Excel, we needed to seek the expertise of NVivo staff to do so because the procedure was not straightforward. 512 College & Research Libraries May 2022 Findings and Discussion Findings suggest dissonance between users’ perceptions of virtual reference methods and how they actually use them. Study participants considered live chat to be for basic questions, but transcripts revealed chat to be used for intermediate and advanced questions, perhaps due to its conduciveness to the reference interview and to instruction, both of whose presence was more prevalent in chat than in other methods. We address each of the research questions below, outlining findings from user interviews first, followed by findings from the transcript analysis. Perceptions of Virtual Reference Methods To answer Research Question 1, the first author conducted interviews with 14 users whose demographic details are outlined in the appendix. In the current publication, we report find- ings related to participants’ perceptions of the level of complexity and types of questions suitable for chat, email to the general library, email to liaison librarians, and texting. Find- ings from interviews with study participants reinforce perceptions outlined in the literature review that users believe chat to be suitable primarily for basic questions. The first author identified the following themes in the interviews: 1) participants perceive chat and texting to be for basic questions and email to be for more complex ones; 2) participants perceive email to their liaison librarian as the best method for asking research questions; and 3) participants value the back-and-forth nature of chat for facilitating their own understanding. We discuss each of these themes below. Chat and Texting for Basic Questions and Email for Complex Ones In terms of different virtual reference methods, many participants perceived chat and texting to be very similar to one another and very different from email. Participants generally felt that chat and texting were for basic questions and email for more advanced ones. For example, as Blue, an undergraduate student, explained: “If it’s a higher level of complexity, I would prob- ably either choose in person, phone, email, and then chat would be last… Email is more the default go to for specialized information… I don’t expect too much to come out of [chat] just because I wouldn’t type out a long, lengthy question that requires a lot of resources. It doesn’t make sense to me.” He expressed that engaging in a chat for something complex would be too time-consuming, a sentiment shared by other participants in the study. Participants mentioned a variety of reasons why they perceived email, rather than chat or texting, to be for advanced questions. As Priya, an international undergraduate student, stated: “Email is when there is a lot of information and I want to be precise about what I am talking. If it is of high importance I use email and when there are a lot of attachments involved.” As another undergraduate student explained, emails would be for more complex questions because the method would give her more time to compose her message, it would permit her and the librarian to engage in a dialogue over a longer period of time, and she would receive a more thoughtful and complete answer from the librarian than she would by chat. Participants did not perceive chat or texting to be for seeking assistance with advanced and/or research-related questions. As Ashley, an undergraduate student, stated categorically, “It would be a very bad idea to talk about research on chat or texting.” Rather than being for reference/research questions, participants felt that chat would be more useful for library sys- tems questions, such as those pertaining to library accounts and fines. Participants also felt that chat would be more appropriate than other methods for asking questions about specific books. Dissonance between Perceptions and Use of Virtual Reference Methods 513 Several participants expressed an aversion to texting within a library context. They were clear that, if they were to use texting at all, it would be for basic questions. According to Michael, a faculty member: “I could imagine, say, I want to ask, ‘Is the library open right now?’ Quick text.” They perceived texting to be primarily for hours and directional information. Email to the Liaison Librarian for Research-Related Questions During the interviews, we discovered an important nuance pertaining to email. Participants frequently expressed that the library’s general email would be for more general questions and email to their liaison librarian for more research-related ones. For example, as Amy said of the liaison librarian: “They’re an expert in the resources in that field. I mean to book a table, sure, I’d email the general one. But if I’m really interested in this specific aspect of cognitive science and looking for some more primary sources on it or something, then I would definitely email the liaison one instead.” Many participants, and especially faculty, explained that they would reserve their reference and research-related questions for asking their liaison librarian by email because they valued their expertise. Prevalence of the Reference Interview and Instruction in Chat Despite participants considering chat to be for basic questions, they nevertheless deemed this method as being very conducive to reference interviews and instruction. During the interviews, no participants explicitly mentioned the reference interview, since that terminol- ogy would likely be unfamiliar to most people outside of library and information studies. Nevertheless, their repeated mentions of the back-and-forth, or conversational, nature of chat were very similar to what takes place in a reference interview. Those whose first language was not English often remarked that they preferred to use methods, such as chat, where they could easily and quickly clear up any misunderstandings related to explaining their own information need. As reported in the previous study on user preferences, the conversational nature of chat was a main factor for study participants in their choice of virtual reference methods, often leading them to prefer this method over others.63 Similar to the reference interview, participants felt that, among the different virtual refer- ence methods, chat’s conversational aspect rendered it particularly conducive to instruction. As Daniel, an alumnus, stated: “Chat is more of a conversation… If the librarian is telling me okay do this or do this, you know I can follow the instructions and see it right away… When you talk with the librarian, that’s when you realize oh I was doing this wrong or I was in the wrong place and I was looking somewhere else. And then you learn. It makes it easier for the next time.” Participants felt that the back-and-forth nature of the interaction facilitated their own understanding. Some participants mentioned features of the chat interface that particu- larly helped them learn, such as the links a librarian would send that they could follow during the interaction. Despite participants finding that chat facilitated the reference interview and instruction, they did not perceive that these features might enhance chat’s ability to support complex questions. Question Complexity Highest in Chat Findings from the current study show a marked disconnect between perceptions, as revealed in the interviews, and actual use of virtual reference methods, as determined through content analysis of virtual reference transcripts. In response to Research Question 2: Is there a statisti- 514 College & Research Libraries May 2022 cally significant relationship between virtual reference delivery method and the level of complexity of questions asked, as measured using the READ Scale? and in contrast with participants’ views that chat was unfit for advanced and research-related questions, findings reveal a statistically significant relationship between delivery method and READ Scale rating, with the association between the two variables being of moderate strength (Χ2 = 252.30, P < .001, V = .41). In fact, question complexity was highest in chat transcripts. See Table 1 for an outline of the strength of association, listed by effect size, for all the variables that we analyzed in the study. The statistical consultant prepared Table 1 and we created the other tables and figures in consul- tation with her. The perception of chat being for basic questions and not for the more advanced ones contrasts sharply with the transcript analysis findings. Chat interactions were slightly more complex than email and much more complex than texting ones, with READ 4 and above level ratings accounting for 24 percent of chats, 17 percent of emails, and only 6 percent of texts (see Figure 1 for frequencies of READ Scale ratings for chat, email, and texting). There were no questions in the study coded at READ 6. Study participants’ perceptions of texting and chat being for questions of similar levels of complexity were not borne out by the statistical analysis of the transcripts. Chat and email were much closer to one another in their READ Scale distribution (being predominantly READ 3 and above), with chat outranking email in terms of the percentage of questions ranked as READ 3 and above. Texting, on the other hand, was quite different, being mostly READ 1 and 2 level questions. It is possible that chat interactions could become lengthier and more detailed because of their synchronous and conversational nature, thus resulting in these questions requiring more time and effort to answer than other methods that did not so easily facilitate this type of interaction. The data from the current study demonstrates that chat and email generate mainly READ 3 and above questions. Given that research discussed in the literature review suggests that READ 3 and above questions are well suited for professional librarians to answer, our service model of having primarily this employee group deliver chat and email aligns well with the level of questions that we currently receive. Since the majority of texting questions are READ 2 and below, we could easily staff the texting service with library assistants and/or students, rather than librarians. The questions asked via texting are in line with what these staff groups are trained for and can be expected to answer. TABLE 1 Variable Association by Effect Size (N = 750) Variable 1 Variable 2 df* Χ2** Cramer’s V Delivery Method Reference Interview 2 140.55 .43 Delivery Method READ Scale 8 252.30 .41 Delivery Method Question Category 18 177.40 .34 Delivery Method Instruction 2 32.97 .21 Notes: *Degree of Freedom **Significance level p < .001 Dissonance between Perceptions and Use of Virtual Reference Methods 515 Reference/Research Questions Most Frequent in Chat Similar to the way in which the interview findings contrasted with the transcript analysis in terms of READ Scale ratings by delivery method, the interviews also contrasted with the tran- script analysis in terms of question categories by delivery method. Many interview participants were resounding in their opinion that chat was not for reference and research questions. Yet, in response to Research Question 3 (Is there a statistically significant relationship between virtual reference delivery method and categories of questions asked? If so, what are the prominent categories for each method?), our analysis shows that reference/research questions accounted for the larg- est category of questions received via chat. Nearly one in four (24%) chat questions were in FIGURE 1 READ Scale by Delivery Method TABLE 2 Question Categories Question Categories Example Access policies I’m an alumnus. How can I access online articles? Collections acquisitions I would like the library to buy this book. Hours What time does the law library close today? Interlibrary loan How do I obtain this article? It’s not available at McGill Known item Does the library have this article? Library physical facilities How do I print a document? Library systems My books are overdue. Help! Other How do I apply to McGill University? Problem with access to e-resources I can’t open this article. Reference/research How do I find information on my topic? 516 College & Research Libraries May 2022 that category, compared to 18 percent of email questions and 15 percent of texting questions. See Table 2 for the list of question categories and a sample question for each and Figure 2 for percentages of questions for each question category by delivery method. The Chi square test indicated that certain categories of questions were asked more often than others depending on the delivery method and that there was a statistically significant relationship of moderate strength between delivery method and question category (Χ2 = 117.40, p < .001, V = .34). Although there were marked differences between user perceptions and virtual refer- ence transcripts in terms of the use of chat for reference/research questions, perceptions and transcripts showed closer alignment in terms of other prominent question categories asked via chat. Besides reference/research questions, the most popular question categories in chat were library systems, problem with access to e-resources, interlibrary loans, and known items (see Figure 2). Both interviews and transcripts showed these types of questions to be suitable for and prevalent in chat. Interlibrary loan (ILL) questions were particularly important in chat in comparison with other delivery methods. It is possible that systems such as ILL require some explanation, and the back-and-forth nature of chat facilitates this type of interaction. Access policies and “Other” questions figured prominently in email as compared to other methods. Anecdotally, we have observed that users who are not part of the McGill commu- nity often ask these categories of questions. A high proportion of questions in the “Other” category pertained to matters that did not relate to the library but rather to other aspects of the university. Perhaps outside users are less familiar with or less comfortable using live chat and texting to make inquiries about the library’s policies and services and are also unfamil- iar with the methods for contacting other units on campus. However, these are speculations and it is beyond the scope of the current study to determine the types of questions asked by specific user groups. FIGURE 2 Question Category by Delivery Method Dissonance between Perceptions and Use of Virtual Reference Methods 517 Email and chat saw high percentages of questions pertaining to problems with access to e-resources while texting did not, presumably because questions like these normally arise when users are consulting the library’s catalogue, an activity that they may not be so readily able or willing to do while on their cell phones. Hours and library physical facilities questions, such as how to print or directional information, appeared much more frequently in texting than in other methods. It is possible that these types of questions happen more readily when users are on the go and do not necessarily have easy access to information on the library’s webpages as they generally would have when they are on a laptop or desktop computer. The prominent question categories received via texting align with users’ perceptions that texting is well suited to hours and directional-type questions. Similar to the READ Scale ratings by delivery method, the findings pertaining to ques- tion category by delivery method suggest that different methods could be staffed by different categories of employees. Based on the data, we can conclude that texting, where questions pre- dominantly concern hours, library physical facilities, and library systems, could be adequately staffed by library assistants and/or students rather than librarians. The questions asked via this method are in line with what these staff groups are trained in and can be expected to answer. In comparison, chat and email are largely reference/research, problems with access to e-resources, and interlibrary loan questions, suggesting that our service model of primar- ily staffing the services with professional librarians is in line with the types of questions we receive. The current findings concur with previous research suggesting that chat and email should be staffed by librarians due to the prevalence of reference questions received via these methods.64 Reference Interviews and Instruction Most Frequent in Chat In response to Research Question 4 (Does the presence of reference interviews and instruction differ across virtual reference methods? If so, are the associations between the variables statistically signifi- cant?), our analysis shows that there are differences. We found statistically significant associa- tions between the frequency of reference interviews and the delivery method and between the frequency of instruction and the delivery method. Both reference interviews and instruction were most prevalent in chat transcripts. Our analysis showed there was a moderate strength of association between the presence of a reference interview and the delivery method (Χ2 = 140.55, p < .001, V = .43). Reference interviews occurred much more frequently in chat than in email and texting (see Figure 3 for a breakdown of the percentage of questions with reference interviews by delivery method). These results align with those from the interviews where, although participants did not men- tion the reference interview per se, they most often discussed the back-and-forth nature of an interaction in the context of chat. Chat is possibly more conducive to conducting reference interviews than other methods because of its conversational nature,65 which is facilitated by its technological advantages, namely its synchronicity. However, chat’s synchronous nature does not appear to solely explain the high frequency of reference interviews in chat in com- parison with email and texting. If synchronicity increases the frequency of reference interviews in chat, it does not appear to do so for texting, which is also relatively, although not wholly, synchronous. Other factors may be at play. Chat’s level of personalness and informality66 may also increase the frequency of reference interviews in comparison with other methods. The higher percentage of reference interviews in chat mirrors findings from Lee’s study that 518 College & Research Libraries May 2022 question negotiation in the form of a reference interview occurs more frequently in chat than in email.67 Nevertheless, the reference interview, a hallmark of librarianship, is happening less than half the time in chat and much less frequently in email and texting. Although not every interaction requires a reference interview, their overall low rates, especially in email and texting, suggest that efforts should be made to increase their adoption. Similar to the reference interview, our analysis showed instruction to be much more prevalent in chat than in email or texting. There was a statistically significant association be- tween the presence of instruction and the delivery method, although its strength was low (Χ2 = 32.97, p < .001, V = .21). Figure 4 illustrates the percentages of questions with instruction for FIGURE 3 Presence of a Reference Interview by Delivery Method FIGURE 4 Presence of Instruction by Delivery Method Dissonance between Perceptions and Use of Virtual Reference Methods 519 chat, email, and texting. The higher frequency of instruction in chat mirrors findings from the interviews where participants who discussed instruction and learning were most likely to do so in the context of chat. Similar to the reference interview, reasons for the higher prevalence of instruction in chat could be a function of the synchronous nature of the technology, which more readily facilitates a conversation in comparison with email and texting. Although instruc- tion occurs most frequently in chat, it is only taking place less than half the time and much less frequently in email and texting. Given our context as an academic library and in light of the library’s mission discussed earlier, efforts should be made to boost rates of instruction, perhaps through enhanced training on recognizing and responding to “teachable moments.” Limitations and Future Research In light of the findings, some limitations should be considered, several of which present oppor- tunities for future research. During the interviews, many participants differentiated between email to the general library and email to their liaison librarian, expressing that they valued and made use of the latter, especially for research-related queries. In the current study, we did not conduct transcript analysis of interactions conducted by email with liaison librarians. Emails to liaison librarians, like those to service accounts, are not part of the library’s virtual reference software and, as a result, were not amenable to inclusion in this research study. At our institution, anecdotal evidence suggests that emailing liaison librarians directly is an ac- tivity that is taking place more and more frequently and for increasingly complex questions. Similarly, Gerlich and Berard suggest that questions answered off-desk require more effort and skill than those answered at physical reference desks and that users actively seek out specific individuals for their reference expertise.68 Although there is some recent literature investigating the role of in-person consultations or appointments,69 there is little mention of email to subject or liaison librarians in the literature. It is quite likely that emails to liaisons rank higher on the READ Scale than those sent to the general library email account, but fur- ther research would be needed to substantiate this claim. Future research is warranted into investigating the types and level of difficulty of questions asked to liaisons, the extent of these activities, and best practices for delivering these services. It bears mentioning that the current study does report on virtual reference methods delivered at only one institution, which limits its applicability in other settings. McGill Uni- versity is research-intensive and, even at the undergraduate level, is strongly committed to exposing students to leading research and offering undergraduates opportunities to engage in research.70 Virtual reference questions are likely impacted by this environment. Virtual refer- ence interactions in the context of a public library or a four-year college setting would likely be different. As mentioned in the literature review, institutional setups and policy decisions also influence the types of questions received in virtual reference. In this context, it is important to recognize that there is no one-size-fits-all solution with regard to staffing decisions for virtual reference and that it is useful to examine the nature of one’s own virtual reference questions within one’s local context when making staffing decisions. In the current study, we compared users’ perceptions of question complexity with the complexity of virtual reference transcripts when categorized using the READ Scale. It is im- portant to acknowledge the possibility that how users measure complexity differs from how it was measured using the READ Scale. In the current study, we did not analyze how complex users considered their own (or others’) actual questions to be. Instead, we examined their 520 College & Research Libraries May 2022 perceptions of different virtual reference methods in terms of question complexity. Further research could investigate this topic by examining how complex users consider their own questions to be, perhaps by having participants categorize actual virtual reference transcripts. As discussed earlier, during the course of the two-year period from which the virtual reference transcripts were drawn, the library transitioned from QuestionPoint to LibChat soft- ware. As a result, certain aspects about how we handled virtual reference questions changed. For example, in LibChat, it is possible to transfer a chat to an email, incoming texts are treated more similarly to chats, and it is easier to refer questions to outside service accounts. These changes likely influenced how the library delivers virtual reference services. However, it is unlikely that they affected the findings greatly since we controlled for the change in software by extracting chats, emails, and texts that were equal in number to each other from Question- Point and from LibChat. Doing so ensured that comparisons across delivery methods would be consistent. In the current study, we did not compare transcripts drawn from one software with the other, but this could be an area of future research. It is likely that the ways the vari- ous delivery methods are handled in each software affect how users and staff employ them. While this study was conducted with data collected prior to the COVID-19 pandemic, it is important to note that it has had some important effects on the provision of virtual reference services. Virtual reference became a necessity during the pandemic since most physical library spaces were closed to the public. As a result, staffing had to be reevaluated and increased to respond to a substantially higher demand from users. With frequently changing information regarding library policies during the pandemic, it was more important than ever that the virtual reference software allow for easy referrals of users’ questions between library staff members and units who provided virtual reference services and those who did not. Given that not all users may have been aware of the services available to them, there was also a need to enhance the marketing of virtual reference to make it a more visible method of communication with the library. It remains to be seen whether the pandemic will have lasting consequences on virtual reference services, but the authors have already noted an increased use of texting, perhaps due to greater needs for basic library information that were generated by the pandemic situ- ation, such as whether certain library services were open or closed. Summary and Implications for Practice In addition to showing that a disconnect exists between the perceived function of virtual refer- ence methods and their actual uses, the current study also shows that librarians can shape the complexity of the questions received by selecting which services to offer. Findings demonstrate a statistically significant association between question category and delivery method, with texting having many hours and library physical facilities questions, chat having reference/ research questions, and email having access policies questions. These findings suggest that each method serves distinct functions, thereby reinforcing the notion that they are all useful methods for the library to maintain. Although the level of complexity of questions in chat was negligibly higher than in email, its conversational nature led to a higher use of the reference interview and a greater presence of instruction in the transcripts we analyzed. The types of questions received in each method, as well as differences in their READ Scale ratings, could lead to some evaluation of staffing needs. For example, given that most reference/research questions with a READ 3 level and higher were asked on chat and email, librarians should continue to monitor and offer this service. The texting service, however, could potentially be Dissonance between Perceptions and Use of Virtual Reference Methods 521 offered by students and paraprofessionals. Similarly, chat and email receive high numbers of library account and interlibrary loan questions. Better documentation, improved policies, and smoother workflows for these services could ease the process for users and the librarians who deliver virtual reference services. Here are some practical implications: • For a seamless service and to answer multiple types and levels of questions, a virtual reference platform that allows multiple methods (chat, email, text) is advised. • Given the high READ Scale ratings and the number of reference/research questions in chat and email, provide more training on the reference interview and instruction meth- ods in virtual reference to librarians. • Ensure that users are aware when their question needs to be transferred to a subject specialist or a service (such as interlibrary loans). • Advertising of virtual reference services as well as the webpages where they are located could have an impact on the types of questions asked. More promotion is needed to enhance user awareness of chat for reference/research questions, letting users know where and how they can ask their questions. • Increase the availability of virtual reference methods, especially chat, in locations where users conduct library research, such as in library discovery tools and research databases and in university learning management systems. This presence may help users to bet- ter associate library chat with research purposes, thus narrowing the gap between user perceptions and actual use. • A high number of access policies and problems with e-resources questions were asked by users. More information and easier ways to report problems should be considered. • The findings from this study could be used to map information literacy gaps in users. That libraries can influence the level of difficulty and types of questions of their virtual reference interactions by the delivery methods they choose to offer aligns with research show- ing that, through libraries’ policy decisions, they can modify user behavior, increasing the likelihood of complex questions. Changes such as placing a chat box within a university’s learning management system,71 implementing proactive chat,72 and having librarians staff virtual reference from their offices rather than public service desks73 can all influence ques- tion complexity. Many of these factors are within the library’s control, and it is telling that they reveal the extent to which libraries can encourage complex questions within virtual reference. These studies highlight that we can aim high for virtual reference and consider it an essential and perhaps more useful tool for today’s researcher than traditional in-person reference. Through virtual reference, librarians have an opportunity to build on services we have long offered in person, using technology to enhance our services. For example, virtual reference allows users to obtain help when and where they need it, facilitates a step-by-step approach for gathering needed information, provides a written transcript for users to consult later, and enables the easy sharing of links and documents by both librarians and users. These features make virtual reference an increasingly useful and important service for today’s and tomorrow’s users. Acknowledgments The authors would like to thank Jen Sweeney for her assistance with the statistical analysis. Our thanks to Chris Byrne, Svetlana Kochkina, and Jen Sweeney for their feedback on the manuscript. 522 College & Research Libraries May 2022 APPENDIX A. Study Participant Data (as reported in Mawhinney, 2020) Pseudonym Age range Gender Academic status (student, alumnus, faculty, or staff ) and field of study International student? (If so, country of origin) Students Alex 31–35 M Doctoral student, Music No Amy 20 and younger F Undergraduate student, Arts and Science No Ashley 21–25 F Undergraduate student, Science No Blue 21–25 M Undergraduate student, Engineering No Jenna 20 and younger F Undergraduate student, Arts No Kevin 26–30 M Master’s student, Engineering Yes, Indonesia Louise 20 and younger F Undergraduate student, Education Yes, France Margarita (interview conducted via Skype) 41–45 F Doctoral student, Arts Yes, Mexico Priya 21–25 F Master’s student, Engineering Yes, India Ryan 31–35 M Master’s student, Information Studies No Sarah 21–25 F Master’s student, Education No Faculty and staff Kim 41–45 F Professor/Instructor, Nursing No Michael 46–50 M Professor/Instructor, Science No Alumnus Daniel 31–35 M Alumnus, Management No Notes 1. Tara Mawhinney, “User Preferences Related to Virtual Reference Services in an Academic Library,” Journal of Academic Librarianship 46, no. 1 (Jan. 2020): 1–8, https://doi.org/10.1016/j.acalib.2019.102094. 2. Joe Janes, “Live Reference: Too Much, Too Fast?” School Library Journal 48, no. 11 (Fall 2002): 12–14. 3. Ian J. Lee, “Do Virtual Reference Librarians Dream of Digital Reference Questions? A Qualitative and Quantitative Analysis of Email and Chat Reference,” Australian Academic & Research Libraries 35, no. 2 (2004): 95–110, https://doi.org/10.1080/00048623.2004.10755262. 4. Kathleen M. Kern, “What Are They Asking? An Analysis of Questions Asked at In-Person and Virtual Service Points” (paper presented at the ALA Annual Conference, 9th Annual Reference Research Forum, To- ronto, Ontario, Jun. 21, 2003), www.ala.org/rusa/sections/rss/rsssection/rsscomm/rssresstat/2003reference. 5. Paula R. Dempsey, “Chat Reference Referral Strategies: Making a Connection, or Dropping the Ball?” College & Research Libraries 80, no. 5 (Jul. 2019): 674–93, https://doi.org/10.5860/crl.80.5.674; Joseph Fennewald, “Same Questions, Different Venue: An Analysis of In-person and Online Questions,” Reference Librarian 46, no. 95/96 (2006): 21–35, https://doi.org/10.1300/J120v46n95_03; Bella Karr Gerlich and G. Lynn Berard, “Testing the Viability of the READ Scale (Reference Effort Assessment Data): Qualitative Statistics for Academic Reference Services,” College & Research Libraries 71, no. 2 (Mar. 2010): 116–37, https://doi.org/10.5860/0710116; Vera J. Lux and Linda Rich, “Can Student Assistants Effectively Provide Chat Reference Services? Student Transcripts vs. https://doi.org/10.1016/j.acalib.2019.102094 https://doi.org/10.1080/00048623.2004.10755262 http://www.ala.org/rusa/sections/rss/rsssection/rsscomm/rssresstat/2003reference https://doi.org/10.5860/crl.80.5.674 https://doi.org/10.1300/J120v46n95_03 https://doi.org/10.5860/0710116 Dissonance between Perceptions and Use of Virtual Reference Methods 523 Librarian Transcripts,” Internet Reference Services Quarterly 21, no. 3/4 (2016): 115–39, https://doi.org/10.1080/10875 301.2016.1248585; Krisellen Maloney and Jan H. Kemp, “Changes in Reference Question Complexity Following the Implementation of a Proactive Chat System: Implications for Practice,” College & Research Libraries 76, no. 7 (Nov. 2015): 959–74, https://doi.org/10.5860/crl.76.7.959; David Ward and Eric Phetteplace, “Staffing by Design: A Methodology for Staffing Reference,” Public Services Quarterly 8, no. 3 (2012): 193–207, https://doi.org/10.1080/152 28959.2011.621856. 6. Fennewald, “Same Questions, Different Venue,” 21–35. 7. Jan H. Kemp, Carolyn L. Ellis, and Krisellen Maloney, “Standing By to Help: Transforming Online Ref- erence with a Proactive Chat System,” Journal of Academic Librarianship 41, no. 6 (Nov. 2015): 764–70, https://doi. org/10.1016/j.acalib.2015.08.018. 8. Lili Luo and Emily Weak, “Text Reference Service: Teens’ Perception and Use,” Library & Information Sci- ence Research 35, no. 1 (Jan. 2013): 14–23, https://doi.org/10.1016/j.lisr.2012.03.002. 9. William Breitbach and Adolfo G. Prieto, “Text Reference Via Google Voice: A Pilot Study,” Library Review 61, no. 3 (2012): 188–98, https://doi.org/10.1108/00242531211259319; J.B. Hill, Cherie Madarash Hill, and Dayne Sherman, “Text Messaging in an Academic Library: Integrating SMS into Digital Reference,” Reference Librar- ian 47, no. 1 (2007): 17–29, https://doi.org/10.1300/J120v47n97_04; Tara Mawhinney and Svetlana Kochkina, “Is the Medium the Message? Examining Transactions Via Text in Comparison with Traditional Virtual Reference Methods,” Journal of Library & Information Services in Distance Learning 13, no. 1/2 (2019): 56–73, https://doi.org/10 .1080/1533290X.2018.1499236. 10. Dempsey, “Chat Reference Referral Strategies,” 674–93. 11. Anthony S. Chow and Rebecca A. Croxton, “A Usability Evaluation of Academic Virtual Reference Ser- vices,” College & Research Libraries 75, no. 3 (May 2014): 309–61, https://doi.org/10.5860/crl13-408. 12. Anthony S. Chow and Rebecca A. Croxton, “Information-Seeking Behavior and Reference Medium Pref- erences,” Reference & User Services Quarterly 51, no. 3 (Spring 2012): 246–62, https://doi.org/10.5860/rusq.51n3.246. 13. David Ward, “Why Users Choose Chat: A Survey of Behavior and Motivations,” Internet Reference Services Quarterly 10, no. 1 (2005): 29–46, https://doi.org/10.1300/J136v10n01_03. 14. Kern, “What Are They Asking?” 15. Riki Greenberg and Judit Bar-Ilan, “‘Ask a Librarian’: Comparing Virtual Reference Services in an Israeli Academic Library,” Library & Information Science Research 37, no. 2 (Apr. 2015): 139–46, https://doi.org/10.1016/j. lisr.2014.09.005. 16. Greenberg and Bar-Ilan, “Ask a Librarian,” 139–46. 17. Lee, “Do Virtual Reference Librarians Dream of Digital Reference Questions?” 95–110. 18. Maria Kingsbury, “How to Smile When They Can’t See Your Face: Rhetorical Listening Strategies for IM and SMS Reference,” International Journal of Digital Library Systems 5, no. 1 (2015): 31–44, https://doi.org/10.4018/ IJDLS.2015010104. 19. Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37. 20. Fennewald, “Same Questions, Different Venue,” 21–35. 21. Kern, “What Are They Asking?” 22. Mawhinney and Kochkina, “Is the Medium the Message?” 56–73. 23. Ward and Phetteplace, “Staffing by Design,” 193–207. 24. Jason Cabaniss, “An Assessment of the University of Washington’s Chat Reference Services,” Public Library Quarterly 34, no. 1 (2015): 85–96, https://doi.org/10.1080/01616846.2015.1000785; Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37; Bella Karr Gerlich and Edward Whatley, “Using the READ Scale for Staff- ing Strategies: The Georgia College and State University Experience,” Library Leadership & Management 23, no. 1 (Winter 2009): 26–30; Kemp, Ellis, and Maloney, “Standing By to Help,” 764–70; Kelsey Keyes and Ellie Dworak, “Staffing Chat Reference with Undergraduate Student Assistants at an Academic Library: A Standards-Based As- sessment,” Journal of Academic Librarianship 43, no. 6 (Nov. 2017): 469–78, https://doi.org/10.1016/j.acalib.2017.09.001; Maloney and Kemp, “Changes in Reference Question Complexity Following the Implementation of a Proactive Chat System,” 959–74; B. Jane Scales, Lipi Turner-Rahman, and Feng Hao, “A Holistic Look at Reference Statistics: Whither Librarians?” Evidence Based Library and Information Practice 10, no. 4 (2015): 173–85, https://doi.org/10.18438/ B8X01H; Thomas Stieve and Niamh Wallace, “Chatting While You Work: Understanding Chat Reference User Needs Based on Chat Reference Origin,” Reference Services Review 46, no. 4 (2018): 587–99, https://doi.org/10.1108/ RSR-09-2017-0033; Ward and Phetteplace, “Staffing by Design,” 193–207; Adrienne Warner et al., “Proactive Chat in Research Databases: Inviting New and Different Questions,” Journal of Academic Librarianship 46, no. 2 (Mar. 2020): 1–7, https://doi.org/10.1016/j.acalib.2020.102134. 25. Maryvon Côté, Svetlana Kochkina, and Tara Mawhinney, “Do You Want to Chat? Reevaluating Organiza- tion of Virtual Reference Service at an Academic Library,” Reference & User Services Quarterly 56, no. 1 (Fall 2016): https://doi.org/10.1080/10875301.2016.1248585 https://doi.org/10.1080/10875301.2016.1248585 https://doi.org/10.5860/crl.76.7.959 https://doi.org/10.1080/15228959.2011.621856 https://doi.org/10.1080/15228959.2011.621856 https://doi.org/10.1016/j.acalib.2015.08.018 https://doi.org/10.1016/j.acalib.2015.08.018 https://doi.org/10.1016/j.lisr.2012.03.002 https://doi.org/10.1108/00242531211259319 https://doi.org/10.1300/J120v47n97_04 https://doi.org/10.1080/1533290X.2018.1499236 https://doi.org/10.1080/1533290X.2018.1499236 https://doi.org/10.5860/crl13-408 https://doi.org/10.5860/rusq.51n3.246 https://doi.org/10.1300/J136v10n01_03 https://doi.org/10.1016/j.lisr.2014.09.005 https://doi.org/10.1016/j.lisr.2014.09.005 https://doi.org/10.4018/IJDLS.2015010104 https://doi.org/10.4018/IJDLS.2015010104 https://doi.org/10.1080/01616846.2015.1000785 https://doi.org/10.1016/j.acalib.2017.09.001 https://doi.org/10.18438/B8X01H https://doi.org/10.18438/B8X01H https://doi.org/10.1108/RSR-09-2017-0033 https://doi.org/10.1108/RSR-09-2017-0033 https://doi.org/10.1016/j.acalib.2020.102134 524 College & Research Libraries May 2022 36–46, https://doi.org/10.5860/rusq.56n1; Robin Canuel et al., “Developing and Assessing a Graduate Student Reference Service,” Reference Services Review 47, no. 4 (2019): 527–43, https://doi.org/10.1108/RSR-06-2019-0041. 26. Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37. 27. Kemp, Ellis, and Maloney, “Standing By to Help,” 764–70. 28. Ward and Phetteplace, “Staffing by Design,” 193–207. 29. Gerlich and Whatley, “Using the READ Scale for Staffing Strategies,” 26–30; Lisa Vassady, Alyssa Archer, and Eric Ackermann, “READ-ing Our Way to Success: Using the READ Scale to Successfully Train Reference Student Assistants in the Referral Model,” Journal of Library Administration 55, no. 7 (2015): 535–48, https://doi.or g/10.1080/01930826.2015.1076309. 30. Kemp, Ellis, and Maloney, “Standing By to Help,” 764–70. 31. Cabaniss, “An Assessment of the University of Washington’s Chat Reference Services,” 85–96. 32. Mariana Lapidus et al., “Measuring the Quality of Reference Services Provided by Paraprofessionals at an Academic Library,” Journal of Academic Librarianship 46, no. 5 (Sept. 2020), https://doi.org/10.1016/j.acalib.2020.102198. 33. Keyes and Dworak, “Staffing Chat Reference with Undergraduate Student Assistants at an Academic Library,” 469–78. 34. Cabaniss, “An Assessment of the University of Washington’s Chat Reference Services,” 85–96; Keyes and Dworak, “Staffing Chat Reference with Undergraduate Student Assistants at an Academic Library,” 469–78; Lapidus et al., “Measuring the Quality of Reference Services Provided by Paraprofessionals at an Academic Library.” 35. “Guidelines for Behavioral Performance of Reference and Information Service Providers,” Reference and User Services Association, www.ala.org/rusa/resources/guidelines/guidelinesbehavioral [accessed 19 June 2020]. 36. Lee, “Do Virtual Reference Librarians Dream of Digital Reference Questions?” 95–110. 37. Dempsey, “Chat Reference Referral Strategies,” 674–93; Marie L. Radford et al., “‘Are We Getting Warmer?’ Query Clarification in Live Chat Virtual Reference,” Reference & User Services Quarterly 50, no. 3 (Spring 2011): 259–79, https://doi.org/10.5860/rusq.50n3.259. 38. Radford et al., “Are We Getting Warmer?” 259–79. 39. Judith Logan, Kathryn Barrett, and Sabina Pagotto, “Dissatisfaction in Chat Reference Users: A Transcript Analysis Study,” College & Research Libraries 80, no. 7 (Nov. 2019): 925–44, https://doi.org/10.5860/crl.80.7.925. 40. Lee, “Do Virtual Reference Librarians Dream of Digital Reference Questions?” 95–110. 41. Paula R. Dempsey, “‘Are You a Computer?’ Opening Exchanges in Virtual Reference Shape the Potential for Teaching,” College & Research Libraries 77, no. 4 (Jul. 2016): 455–68, https://doi.org/10.5860/crl.77.4.455; Logan, Barrett, and Pagotto, “Dissatisfaction in Chat Reference Users,” 925–44. 42. Christina M. Desai and Stephanie J. Graves, “Cyberspace or Face-to-Face: The Teachable Moment and Changing Reference Mediums,” Reference & User Services Quarterly 47, no. 3 (Spring 2008): 242–55, https://doi. org/10.5860/rusq.47n3.242. 43. Paula R. Dempsey, “Resource Delivery and Teaching in Live Chat Reference: Comparing Two Libraries,” College & Research Libraries 78, no. 7 (Nov. 2017): 898–919, https://doi.org/10.5860/crl.78.7.898. 44. Sandy Hervieux and Nikki Tummon, “Let’s Chat: The Art of Virtual Reference Instruction,” Reference Services Review 46, no. 4 (2018): 529–42, https://doi.org/10.1108/RSR-07-2018-0060; Keyes and Dworak, “Staffing Chat Reference with Undergraduate Student Assistants at an Academic Library,” 469–78. 45. Mawhinney and Kochkina, “Is the Medium the Message?” 56–73. 46. Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37; Kemp, Ellis, and Maloney, “Stand- ing By to Help,” 764–70; Logan, Barrett, and Pagotto, “Dissatisfaction in Chat Reference Users,” 925–44; Linda Rich and Vera Lux, “Reaching Additional Users with Proactive Chat,” Reference Librarian 59, no. 1 (2018): 23–34, https://doi.org/10.1080/02763877.2017.1352556. 47. “About McGill: 2019 Factbook,” McGill University, https://www.mcgill.ca/about/quickfacts [accessed 19 June 2020]. 48. Mawhinney, “User Preferences Related to Virtual Reference Services in an Academic Library,” 1–8. 49. “About the Library,” McGill Library, https://www.mcgill.ca/library/about [accessed 19 June 2020]. 50. Krista Bowers Sharpe and Christina Norton, “Examining Our Past, Considering Our Future: A Study of Email Reference, 2000–2015,” Internet Reference Services Quarterly 22, no. 4 (2018): 133–65, https://doi.org/10.1080 /10875301.2018.1455617; Logan, Barrett, and Pagotto, “Dissatisfaction in Chat Reference Users,” 925–44; Warner et al., “Proactive Chat in Research Databases,” 1–7. 51. Mawhinney, “User Preferences Related to Virtual Reference Services in an Academic Library,” 1–8. 52. Mawhinney, “User Preferences Related to Virtual Reference Services in an Academic Library,” 1–8. 53. Lynn Silipigni Connaway, Timothy J. Dickey, and Marie L. Radford, “‘If It Is Too Inconvenient, I’m Not Going After It’: Convenience as a Critical Factor in Information-Seeking Behaviors,” Library & Information Science https://doi.org/10.5860/rusq.56n1 https://doi.org/10.1108/RSR-06-2019-0041 https://doi.org/10.1080/01930826.2015.1076309 https://doi.org/10.1080/01930826.2015.1076309 https://doi.org/10.1016/j.acalib.2020.102198 http://www.ala.org/rusa/resources/guidelines/guidelinesbehavioral https://doi.org/10.5860/rusq.50n3.259 https://doi.org/10.5860/crl.80.7.925 https://doi.org/10.5860/crl.77.4.455 https://doi.org/10.5860/rusq.47n3.242 https://doi.org/10.5860/rusq.47n3.242 https://doi.org/10.5860/crl.78.7.898 https://doi.org/10.1108/RSR-07-2018-0060 https://doi.org/10.1080/02763877.2017.1352556 https://www.mcgill.ca/about/quickfacts https://www.mcgill.ca/library/about https://doi.org/10.1080/10875301.2018.1455617 https://doi.org/10.1080/10875301.2018.1455617 Dissonance between Perceptions and Use of Virtual Reference Methods 525 Research 33, no. 3 (Jul. 2011): 179–90, https://doi.org/10.1016/j.lisr.2010.12.002. 54. Stieve and Wallace, “Chatting While You Work,” 587–99; Lee, “Do Virtual Reference Librarians Dream of Digital Reference Questions?” 95–110; Mawhinney and Kochkina, “Is the Medium the Message?” 56–73. 55. Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37. 56. Maloney and Kemp, “Changes in Reference Question Complexity Following the Implementation of a Proactive Chat System,” 959–74. 57. Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37; Stieve and Wallace, “Chatting While You Work,” 587–99. 58. Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37. 59. Armann-Keown, Cooke, and Matheson, “Digging Deeper into Virtual Reference Transcripts,” 656–72; Bowers Sharpe and Norton, “Examining Our Past, Considering Our Future,” 133–65; Kern, “What Are They Asking?”; Lee, “Do Virtual Reference Librarians Dream of Digital Reference Questions?” 95–110; Côté, Kochkina, and Mawhinney, “Do You Want to Chat?” 36–46. 60. Stieve and Wallace, “Chatting While You Work,” 587–99. 61. Virginia Braun and Victoria Clarke, Successful Qualitative Research: A Practical Guide for Beginners (London, UK: SAGE, 2013). 62. Greenberg and Bar-Ilan, “Ask a Librarian,” 139–46. 63. Mawhinney, “User Preferences Related to Virtual Reference Services in an Academic Library,” 1–8. 64. Fennewald, “Same Questions, Different Venue,” 21–35. 65. Mawhinney, “User Preferences Related to Virtual Reference Services in an Academic Library,” 1–8. 66. Mawhinney, “User Preferences Related to Virtual Reference Services in an Academic Library,” 1–8. 67. Lee, “Do Virtual Reference Librarians Dream of Digital Reference Questions?” 95–110. 68. Gerlich and Berard, “Testing the Viability of the READ Scale,” 116–37. 69. Doreen Bradley et al., “Advancing the Reference Narrative: Assessing Student Learning in Research Consultations,” Evidence Based Library and Information Practice 15, no. 1 (2020): 4–19, https://doi.org/10.18438/ eblip29634; Lorie A. Kloda and Alison J. Moore, “Evaluating Reference Consultations in the Academic Library,” in Proceedings of the Library Assessment Conference: Building Effective, Sustainable, Practical Assessment, eds. Sue Baughman et al. (Washington, DC, 2016), 626–32, https://www.libraryassessment.org/wp-content/uploads/ bm~doc/proceedings-2016.pdf; Trina J. Magi and Patricia E. Mardeusz, “Why Some Students Continue to Value Individual, Face-to-Face Research Consultations in a Technology-Rich World,” College & Research Libraries 74, no. 6 (Nov. 2013): 605–18, https://doi.org/10.5860/crl12-363. 70. “McGill Commitment,” McGill University, https://commitment.mcgill.ca [accessed 19 June 2020]. 71. Stieve and Wallace, “Chatting While You Work,” 587–99. 72. Kemp, Ellis, and Maloney, “Standing By to Help,” 764–70; Rich and Lux, “Reaching Additional Users with Proactive Chat,” 23–34; Jie Zhang and Nevin Mayer, “Proactive Chat Reference: Getting in the Users’ Space,” College & Research Libraries News 75, no. 4 (Apr. 2014): 202–05, https://doi.org/10.5860/crln.75.4.9107; Warner et al., “Proactive Chat in Research Databases,” 1–7. 73. Dempsey, “Resource Delivery and Teaching in Live Chat Reference,” 898–919. https://doi.org/10.1016/j.lisr.2010.12.002 https://doi.org/10.18438/eblip29634 https://doi.org/10.18438/eblip29634 https://www.libraryassessment.org/wp-content/uploads/bm~doc/proceedings-2016.pdf https://www.libraryassessment.org/wp-content/uploads/bm~doc/proceedings-2016.pdf https://doi.org/10.5860/crl12-363 https://commitment.mcgill.ca https://doi.org/10.5860/crln.75.4.9107