25 Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes Catherine Meals* Many academic libraries have implemented personal librarian programs, seeking to improve relationships with students and reduce library anxiety. This pilot study, in which a personal librarian was assigned to sections of a general education course, aimed to expand upon the personal librarianship literature by assessing whether a personal librarian influences student academic outcomes and information literacy confidence. Results from the pilot suggest that the presence of and engagement with the personal librarian had minimal and not statistically significant effect on academic outcomes, but engagement with the personal librarian somewhat correlated with growth in information literacy confidence. Introduction Personal librarian programs are a proactive form of academic librarianship that exemplify the growth of library services beyond the physical library. In such programs, librarians reach out to and act as primary contacts for the library for a specific population of students, generally with the aim of improving students’ familiarity and comfort with academic libraries, academic and research skills, and relationships with librarians. In this pilot study, a personal librarian was assigned to certain sections of a research-intensive general education class at the University of the District of Columbia (UDC), a public, urban, land-grant, historically Black institution. The study sought to evaluate whether, and to what extent, the presence and student utiliza- tion of personal librarians affect information literacy–related academic outcomes and student confidence with information literacy skills. Specifically, the study aimed to answer the following research questions: • Does the presence of a personal librarian in a class affect information literacy–related academic outcomes? • Does the level of engagement with a personal librarian affect information literacy–related academic outcomes? • Does the presence of a personal librarian in a class affect student confidence in applying information literacy skills to coursework? * Catherine Meals is Reference & Assessment Librarian at the University of the District of Columbia; email: catherine.meals@udc.edu. Thank you to Faith Rusk for her support in conducting this study. Thanks also to the coordinators of the Institute for Research Design in Librarianship, the 2019 IRDL cohort, and Nina Exner, who provided valuable training, mentorship, and support. ©2022 Catherine Meals, Attribution-NonCommercial (https:// creativecommons.org/licenses/by-nc/4.0/) CC BY-NC. 26 College & Research Libraries January 2022 • Does the level of engagement with a personal librarian affect student confidence in ap- plying information literacy skills to coursework? UDC librarians learned of the personal librarianship concept and became interested in evaluating it as a potential avenue for establishing a greater presence in the institution’s academic life, building stronger relationships with students, promoting the library’s role in student success, and improving students’ information literacy skills. While librarians have long taught information literacy classes for UDC’s general education classes, the university does not currently require library instruction for any classes. Anecdotal feedback from profes- sors has suggested that library instruction, incorporating creative, active learning activities, correlates to increased student use of library resources and stronger research practices, but librarians had not yet conducted a formal assessment of either the impact of library instruc- tion or interaction with librarians on information literacy–related academic outcomes and student confidence. The researchers designed the personal librarianship intervention to support the univer- sity’s Equity Imperative strategic plan, one of whose aims is increasing the quality of general education classes.1 Further, this pilot represented an opportunity for cross-department aca- demic assessment. The study used a recently developed rubric, designed by general education department faculty for use in evaluating student final papers, as a mechanism for assessing student academic outcomes. Literature Review Personal librarian (PL) programs, in which librarians regularly reach out to an assigned group of students to serve as a one-on-one contact for library information and research support, have existed in academic libraries at least since 1984, when Sam Houston State University in Texas launched its program.2 In the decades since, dozens of academic libraries have piloted or implemented such programs. Each program is unique, reflecting an institution’s specific needs and capacity, but they share an overarching goal of creating meaningful librarian-student relationships “that allow students to have the confidence and resources to be successful in the skill sets that librarians particularly seek to instill in them.”3 That is, PL programs are meant to meet students’ intertwined affective and academic needs.4 In the affective realm, PL programs have articulated goals of putting a “friendly face” on university libraries,5 “mak[ing] complex library systems seem less intimidating and more accessible,”6 making library staff seem more approachable,7 and reducing library anxiety and strengthening student confidence in using the library and its resources.8 Further, in establishing personal relationships with students, such programs may encourage “librarians [to] become part of a student’s support network,” which can directly influence academic performance.9 By mitigating emotional barriers to library use, librarians can better help students access and effectively use high-quality resources and information10 and develop their research skills.11 In its proactive nature, personal librarianship represents academic librarians’ evolution into providers of services beyond the physical space of the library, as well as their heightened visibility on campus through partnerships with faculty.12 They amplify present library offer- ings and support through proactive outreach to students13 and provide an important variety of opportunities and means through which students can engage with library services and staff.14 Indeed, Joe Eshleman argues that “aspects of a personal librarian program could be considered a more direct, singular, and purposeful flavor of library instruction.”15 Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 27 PL programs have targeted specific student populations, including freshman students,16 transfer students,17 adult learners,18 and distance, online, or limited residency students.19 Some programs target more specific populations, such as the University of Alberta’s program focus- ing on freshmen who identify as Aboriginal/Native, specifically supporting them by working to “decolonize the university by attempting to remove the barriers present as a result of settler colonialism.20 The program described by Erica England and Leo S. Lo worked specifically with a cohort of students in a low-residency doctoral program.21 PL programs share many characteristics and goals with embedded librarian programs; like personal librarians, embedded librarians typically aim to reduce library anxiety and improve students’ information literacy skills,22 while amplifying the role of the library in academic success.23 Since each personal or embedded librarian program is unique in order to meet specific institutional and academic imperatives, and there appears to be no standard definition of personal or embedded librarianship, the choice of terminology is relatively flex- ible. However, what distinction there is between personal and embedded librarianship often exists in branding and the populations targeted by such programs. While PL programs often target specific populations of students, embedded librarianship often connotes a librarian’s unbranded involvement in a specific class, where the instructor may function as an interme- diary between students and the librarian. Embedded librarians may be prominently listed on syllabi,24 act as co-instructors,25 or have influence in assignment design and assessment.26 Owing to that intensive level of involvement, embedded librarianship may be more time- consuming.27 A common element of PL programs is regular outreach to students with information about library services, research advice, and invitations for personal consultations. Many programs regularly send emails, at a variety of time intervals, to students during the semester.28 Other programs reaching out to freshmen have mailed materials to students before the beginning of the semester.29 Lizah Ismail reached out to students through the university’s course manage- ment system.30 Some programs have included in-person events to encourage contact between students and librarians.31 Initial outreach to students often introduces personal librarians to students, sometimes including information on librarians’ hobbies and interests to make the librarians seem more accessible or approachable to students.32 Alfred University’s PL program approached intro- ductions creatively, creating Magic the Gathering cards for each PL.33 Outreach during the semester has included content on using the library, beginning a re- search process, finding sources, and sharing research tips.34 Reflecting the affective intentions of many PL programs, Jérôme Melançon and Nancy Goebel’s program outreach sought to mirror the emotional and academic rhythm of a semester, including emotional supports the library offered during stressful times, such as pet therapy.35 Programs frequently also strongly encourage individual research consultations with the personal librarian.36 An approach in which personal librarians teach library instruction for their assigned students provides li- brarians an opportunity to have in-person contact with students and explain how they can facilitate students’ connection to the library and academic support.37 Institutions assessing the impact of their PL programs have primarily evaluated the amount of interaction with PLs and student, faculty, and personal librarian perceptions of the programs. Libraries have measured quantity of interactions through metrics including click rates on emails, student replies to emails, and number of one-on-one research appointments 28 College & Research Libraries January 2022 made by students via emails.38 In some cases, the assessment has occurred on a more subjec- tive level: At the University of North Carolina, “the success of the program and the decision to continue it have relied heavily on anecdotal evidence from the participating students, li- brarians and campus partners, rather than on systematic assessment data.”39 Such anecdotal evidence included stories about the type and quality of student interactions. 40 Many program coordinators have conducted surveys of students and faculty to determine awareness of and familiarity with the program,41 feelings of connection and comfort with the library and likelihood of recommending library services to other students,42 their use of the personal librarian,43 satisfaction with the program,44 and perceptions of whether the program affected their current and future use of library services.45 Survey responses were positive, with students reporting that they appreciated the availability of the PL46 and felt more comfortable with the library and its staff.47 The study aimed to expand the personal librarian literature in two ways. UDC has a distinct status as a public, urban, land-grant, historically Black institution with a substantial “nontraditional” student population, and researchers hoped to contribute insights from a type of institution that is less frequently represented in the LIS literature. In addition, there appears to be no literature on the academic impact of PL programs. To date, published litera- ture on personal librarian programs has focused on program implementation, and program assessment described in the literature has largely focused on program utilization and student and faculty satisfaction. The study intended to expand on the existing literature by assessing the relationship between personal librarian programs and student academic and affective outcomes. What literature appears to exist on the academic impact of proactive librarian outreach is on embedded librarian programs, where librarians have worked with a specific class. In Amanda Shannon and Vaughn Shannon’s embedded librarian program, the librar- ian taught in-class information literacy sessions, collaborated with the course’s professor on assignment design and instructional materials, and provided consultations and several informal classroom visits.48 The authors, noting that prior evaluation of similar programs had largely relied on surveys and student self-evaluations, undertook a quantitative analysis of source use in student writing, ultimately finding that, when compared to a course in the previous semester without an embedded librarian, “repeated visits stressing information literacy may significantly affect the number of sources used, the quality of sources used, and the overall quality of the papers students write and professors read.”49 By contrast, Alexis Teagarden and Michael Carlozzi found no significant difference in information literacy out- comes between students in a class where a librarian taught a one-shot information literacy session and another class with an embedded librarian, as measured by pre- and post-tests. In this experimental study, the embedded librarian taught three additional information literacy sessions.50 Methods In fall 2019, professors in 9 of the 10 in-person sections of the general education class, known as Discovery Writing, agreed to allow their students to be invited to participate in the study. The researchers selected Discovery Writing for the pilot for three reasons: first, all students, including the many transfer students at UDC, are required to take it; second, many of its learning outcomes directly relate to information literacy (see appendix A); and third, it is one of the courses in which librarians have regularly taught information literacy instruction. Four Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 29 of the 10 sections were randomly assigned the personal librarian. The instructors of these four sections were each teaching a second section of the class that was not assigned a personal librarian. Student rosters were collected after the conclusion of the university’s add/drop period to identify eligible students. Of the 222 students enrolled in Discovery Writing at the conclusion of add/drop, 83 (37%) consented to participate in the study. The UDC Institutional Review Board reviewed and approved the study. Personal Librarian Intervention UDC librarians designed a personal librarian intervention that drew on features of PL pro- grams described in the literature. In the class sections assigned a personal librarian, the PL 1) taught two information literacy sessions to deliver information literacy skills; 2) sent monthly emails containing research tips to students in their class sections to reinforce skills, serve as a support that all students could refer to as needed, and offer information to students who did not attend the in-person information literacy sessions; and 3) encouraged and was available for one-on-one student research support appointments, to provide a supplemental, optional, in-person resource that could provide both academic and affective support. In cases of walk- in, unscheduled, or spontaneous student-initiated reference consultations, UDC librarians attempted to ensure that the personal librarian responded to students enrolled in personal librarian sections. The researchers anticipated that the information literacy instruction, emails addressed to students by name, and appointments would act as the “personal” nature of the intervention. Importantly, researchers envisioned the outreach and support of the personal librarian as a supplemental intervention on top of current library services. Students and faculty in class sections that were not assigned a personal librarian had access to all services offered and marketed by the library, including information literacy instruction at the discretion of faculty and engagement with librarians at the discretion of individual students. Data Collected and Studied Participating students completed beginning- and end-of-semester 5-point Likert-type scale surveys (see appendix B) on their confidence with information literacy skills to evaluate growth in students’ self-described levels of their information literacy confidence during the semester. Researchers chose to examine student confidence, as it is a critical component of information literacy: learners with low self-efficacy may be less likely to attempt to apply information literacy skills in their academic work or do so successfully.51 The researchers developed a survey with questions about confidence with the specific information literacy skills that the personal librarian taught in information literacy sessions and discussed in monthly emails to students. An independent samples t-test was used to determine whether mean net growth in information literacy confidence among students in course sections assigned a personal librar- ian was higher to a statistically significant extent than that of students in sections without a personal librarian. A point-biserial correlation was calculated to determine any relationship between being in a class section assigned a personal librarian and net information literacy confidence gain. Academic outcomes were measured by student scores on information literacy–related sections of a rubric (see appendix C), developed by the general education faculty and re- flecting aspects of the Association of American Colleges & Universities Written Commu- 30 College & Research Libraries January 2022 nication VALUE rubric,52 that has been used since fall 2018 to grade all final papers in the general education class. These sections—Use of Sources, Format and Structure, and Read- ing Ability and Synthesis—aligned with the personal librarian’s instruction and materials on locating, evaluating, integrating, and citing sources. In Discovery Writing, instructors of different sections of the course in which the student is enrolled blindly grade student work products; that is, no instructor evaluated their own students’ work. An independent samples t-test was used to determine whether mean rubric scores among students in course sections assigned a personal librarian were higher to a statistically significant extent than those of students in sections without a personal librarian. A point-biserial correlation was calculated to determine whether there was a correlation between having been assigned a personal librarian and rubric scores. For students in class sections assigned a personal librarian, engagement with the personal librarian was measured by attendance at information literacy sessions taught by the personal librarian (collected on normal class sign-in sheets), whether a student individually met with the personal librarian, and self-reported engagement with resources shared through emails from the personal librarian. Researchers selected these measurements with consideration for student privacy. Attendance at information literacy sessions and meetings with the personal librarian are relatively public activities: class attendance takes place in a public setting, and an appointment requires meeting in a public space with a librarian. Measurement of engage- ment with personal librarian emails relied on student self-reported engagement, as other options for measuring engagement (such as tracking email clicks) rely on tracking behavior that does not take place in the public sphere and for which students would have a reasonable expectation of privacy. Pearson correlation coefficients were calculated to determine any rela- tionships among rubric scores, net gain in information literacy confidence, and the measures of engagement with the personal librarian. Results Academic Outcomes: PL vs. Non-PL Class Sections Discovery Writing instructors scored student final papers in the class using a 0–25 scale rubric. Rubric scores (n = 72; 41 for students in personal librarian class sections and 31 for students not in personal librarian class sections) were analyzed for the three of four sections on the rubric that related to information literacy skills: Use of Sources, Format/Structure, and Read- ing Ability/Synthesis. Mean rubric scores for students in personal librarian sections were higher for each of the three sections, but the differences were not statistically significant, according to an indepen- dent samples t-test (p > 0.05): TABLE 1 Mean Rubric Scores and Independent Samples t-test Results, Personal Librarian vs. Nonpersonal Librarian Course Sections Average Rubric Score: Personal Librarian n Average Rubric Score: No Personal Librarian n p Use of Sources 20.01 41 18.77 31 0.256 Format/Structure 21.00 41 20.39 31 0.392 Reading Ability/Synthesis 21.28 41 20.19 31 0.229 Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 31 A point-biserial correlation was also calculated for each section of the rubric to identify whether student rubric scores correlated to the presence of the personal librarian in a course section. Weak positive correlations were found between the presence of the personal librarian and rubric scores: Information Literacy Confidence: PL vs. Non-PL Class Sections Beginning- and end-of-semester 5-point Likert-type scale surveys on information literacy confidence were scored by summing survey answers (maximum score = 25). Students’ net gain in confidence during the semester was calculated. Mean confidence gain was higher for students in personal librarian class sections than for students not in personal librarian sections, but the differences were not statistically significant according to an independent samples t-test (p > 0.05): A point-biserial correlation was also calculated for each section of the rubric to identify any relationship between being in a class section assigned a personal librarian and net confi- dence gain. Negligible positive correlations were found between the presence of the personal librarian and rubric scores: Academic Outcomes: Engagement with Personal Librarian For students in class sections assigned a personal librarian, Pearson correlation coefficients were calculated to identify any relationships between rubric scores on final papers and three TABLE 2 Mean Rubric Scores and Point-Biserial Correlation Results, Personal Librarian vs. Nonpersonal Librarian Course Sections Average Rubric Score: Personal Librarian n Average Rubric Score: No Personal Librarian n r pb Use of Sources 20.01 41 18.77 31 0.136 Format/Structure 21.00 41 20.39 31 0.102 Reading Ability/Synthesis 21.28 41 20.19 31 0.143 TABLE 3 Mean Net Gain in Information Literacy Confidence and Independent Samples t-test Results, Personal Librarian vs. Nonpersonal Librarian Course Sections Average Net Confidence Gain: Personal Librarian n Average Net Confidence Gain: No Personal Librarian n p 2.25 32 1.15 26 0.535 TABLE 4 Mean Net Gain in Information Literacy Confidence and Point-Biserial Correlation Results, Personal Librarian vs. Nonpersonal Librarian Course Sections Average Net Confidence Gain: Personal Librarian n Average Net Confidence Gain: No Personal Librarian n r pb 2.25 32 1.15 26 0.083 32 College & Research Libraries January 2022 measurements of engagement with the personal librarian: 1) the number of information lit- eracy sessions attended; 2) self-reported engagement with personal librarian emails; and 3) whether a student engaged in a consultation with the personal librarian. Negligible to weak positive correlations were found between the number of information literacy sessions attended and rubric scores. Each section assigned a personal librarian had two information literacy sessions. Thirty students attended both sessions, 10 students attended one session, and one student attended neither session. Negligible correlations were found between self-reported engagement with personal li- brarian emails and rubric scores. Self-reported engagement with personal librarian materials was measured by the 5-point Likert-type scale sum of student responses to two questions on the end-of-semester survey (see appendix B; maximum score = 10) regarding how often the student read emails from the personal librarian and how often the student used the tips and resources provided in the emails. Negligible to weak correlations were found between having had at least one consultation with the personal librarian and rubric scores. Information Literacy Confidence: Engagement with the Personal Librarian For students in class sections assigned a personal librarian, Pearson correlation coefficients were calculated to identify any relationships between net gain in information literacy con- TABLE 5 Pearson Correlation Coefficients: Attendance at Information Literacy Sessions and Rubric Scores n Pearson’s r p Attendance + Use of Sources 41 0.26 0.10 Attendance + Format/Structure 41 0.06 0.71 Attendance + Reading Ability/Synthesis 41 0.12 0.46 TABLE 6 Pearson Correlation Coefficients: Self-Reported Engagement with Personal Librarian Emails and Rubric Scores n Pearson’s r p Self-Reported Engagement + Use of Sources 29 0.01 0.96 Self-Reported Engagement + Format/Structure 29 –0.12 0.53 Self-Reported Engagement + Reading Ability/Synthesis 29 –0.06 0.76 TABLE 7 Pearson Correlation Coefficients: Consultation with the Personal Librarian and Rubric Scores n Pearson’s r p Consultation + Use of Sources 41 0.012 0.942 Consultation + Format/Structure 41 0.060 0.710 Consultation + Reading Ability/Synthesis 41 0.024 0.880 Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 33 fidence, as measured by scores on beginning- and end-of-semester 5-point Likert-type scale surveys on information literacy confidence, and 1) the number of information literacy ses- sions attended; 2) self-reported engagement with personal librarian emails; and 3) whether a student engaged in a consultation with the personal librarian. A weak negative correlation was found between the number of information literacy ses- sions attended and net gain in information literacy confidence. A weak positive correlation was found between the self-reported engagement with per- sonal librarian emails and net gain in information literacy confidence, and the correlation was not statistically significant (p > 0.05). Self-reported engagement with personal librarian materi- als was measured by the 5-point Likert-type scale sum of student responses to two questions on the end-of-semester survey (see appendix B; maximum score = 10) regarding how often the student read emails from the personal librarian and how often the student used the tips and resources provided in the emails. A weak positive correlation was found between having had at least one consultation with the personal librarian and net gain in information literacy confidence, and the correlation was not statistically significant (p > 0.05). Discussion Challenges and Limitations A variety of challenges in study implementation may help contextualize the results from the first semester of the PL program at UDC. The most salient limitation was sample size: Just 37 percent (n = 83) of students enrolled in Discovery Writing at the beginning of the semester agreed to participate in the study through an electronic consent form. While a majority of participating students completed the final paper required for the class and the end-of-semes- ter survey, not all did so, reducing the number of work products and surveys available for TABLE 8 Pearson Correlation Coefficient: Attendance at Information Literacy Sessions and Mean Net Gain in Information Literacy Confidence n Pearson’s r p Attendance + net confidence gain 29 –0.283 0.137 TABLE 9 Pearson Correlation Coefficient: Self-Reported Engagement with Personal Librarian Emails and Mean Net Gain in Information Literacy Confidence n Pearson’s r p Self-Reported Engagement + Net Confidence Gain 31 0.235 0.202 TABLE 10 Pearson Correlation Coefficient: Consultation with the Personal Librarian and Mean Net Gain in Information Literacy Confidence n Pearson’s r p Consultation + Net Confidence Gain 32 0.290 0.107 34 College & Research Libraries January 2022 analysis. In the following semester, researchers administered the consent form in person and on paper, which dramatically increased the participation rate; 68 percent of eligible students agreed to participate. Most significantly beyond the sample size, all sections of the class that participated in the study scheduled at least one information literacy instruction session. (A librarian other than the personal librarian taught sessions for sections that were not assigned a personal librar- ian.) This is understandable, as faculty members sought to make student support comparable across class sections, and is undoubtedly a positive development, as UDC librarians seek to reach as many students as possible and expand the library’s information literacy program. But for the purposes of the study, this development meant that information literacy instruction was ultimately not a distinguishing feature of the personal librarian intervention. As a result, the primary distinguishing feature of the personal librarian intervention was the branding of the “personal librarian” and provision of resources through monthly emails to students with research tips. However, given that aggregate click rates for emails ranged from 29 to 52 per- cent, and 47 percent of students in sections assigned a personal librarian reported they read the emails “never,” “rarely,” or “occasionally,” email alone may not be a significant enough intervention. Data collection limitations may have impacted the results. First, an unknown number of students enrolled in the class at the beginning of the semester either dropped out or were dropped from the class by the university after midterms due to tuition payment arrears. Second, attendance data for information literacy sessions were collected on sign-in sheets, which sometimes included students who were late to class or left early; any students listed as having attended the IL session who did not attend the full session may not have reaped the full benefits of instruction. Third, while librarians made all efforts to record consultations for students enrolled in the study, it is certainly possible that some were missed. Fourth, students’ self-reporting of their engagement with personal librarian emails may not accurately reflect their actual use of these resources. Fifth, data collection did not include information on any additional information literacy or writing support that students may have received through other campus services, such as peer tutoring, appointments with the campus Writing Center, or information literacy instruction in other courses. In addition, the rubric used to grade final papers is relatively new, having first been used in fall 2018. As a result, the rubric may not yet be fully normed and the interrater reliability of the rubric scores may be constrained. Further, some instructors who graded student papers had not yet participated in rubric calibration meetings at the time of grading, and one grader submitted overall scores for students but did not specify scores for individual sections of the rubric, reducing the number of individual rubric scores available for analysis. Differences in student academic and confidence outcomes could also reflect previous information literacy instruction or variations in teaching approaches among course instructors. Finally, this pilot did not incorporate an examination of how student and librarian social identities may affect outcomes in such programs, particularly how overlaps or differences in identities impact on student interactions with the personal librarian. This is an avenue of investigation the researchers hope to pursue in the future. While students and librarians have multiple interconnected identities that shape how they perceive and operate within institutions and society more broadly, differences in racial identities may be most relevant for study due to the pervasive whiteness in librarianship and is likely to be highly relevant at Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 35 historically Black institutions, where racial identity is foundational to institutional identity, and especially relevant at UDC, where public services librarians are all white but only 5 per- cent of the student body is white.53 Study Insights and Interpretation Despite the limitations described above, the pilot study resulted in some benefits to students and the library, and the implementation of the personal librarian intervention was a positive experience for the library, such that librarians plan to continue to offer the program in later semesters, with modifications informed by the results of the pilot. Encouragingly, the study appeared to result in a noteworthy—though anecdotal, as there was no baseline data against which to make comparisons—increase in one-on-one student appointments with a librarian. Of the 83 Discovery Writing students participating in the study, at least 17 (20%) participated in one or more consultations with a librarian. That total included 18 percent (n = 7) of students not enrolled in personal librarian class sections and 22 percent (n = 10) of students enrolled in personal librarian sections, indicating that the appointments may be linked to having had information literacy instruction rather than the presence of the personal librarian. These one-on-one appointments afforded librarians a better understanding of the general education class, the assignments given to students, and student life more broadly. Though these appointments could occasionally be time-consuming, the total time commitment in large part reflected the personal librarian’s individual tendency to have lengthy meetings of up to 90 minutes with students. Other UDC librarians report that their typical research consultations last 20–30 minutes. The personal librarian also reported more recognition by students on campus. As an example, hours after the personal librarian sent her first monthly email to students, one of the students who received the email saw the personal librarian at the reference desk, said, “Hi, personal librarian!” and introduced him- self. The pilot study also provided opportunities to build relationships with faculty teaching general education classes. Of the five faculty members whose course sections participated in the study, three had never previously scheduled library instruction with UDC librarians. Further, since participating faculty members scheduled instruction for all sections of the courses they taught, whether they were assigned a personal librarian or not, librarians were involved in at least some capacity in 9 of the 10 in-person sections of the course taught during the pilot semester, potentially opening the door to librarians and formal information literacy instruction becoming more systematically integrated into general education courses in the coming years. Finally, in conducting the study, librarians also demonstrated their interest in, support for, and possible contributions to Scholarship of Teaching and Learning that the university encourages. This study was a pilot with only one semester of data, but reflections from the experience will inform the future of personal librarians at UDC and may be instructive for additional research and the development of PL programs at other institutions. At UDC, academic perfor- mance, as measured by mean rubric scores on the final paper, was higher for students with a personal librarian than for those without, but the difference was not statistically significant. Nonetheless, given that fall 2019 was the first semester of the program, the fact that average scores in personal librarian sections were slightly higher is encouraging. Further research on the academic impact of PL programs, either at UDC or other institutions, will help clarify whether there is an impact and, if so, the extent of the impact. In future iterations of the 36 College & Research Libraries January 2022 personal librarian program, librarians may seek to assess student work products themselves to provide an additional perspective on performance with information literacy skills. While the use of the general education department’s rubric represented a worthwhile experiment in cross-department collaboration, easier data collection for the researchers, and a means of assessing academic performance from the perspective of teaching faculty, the researchers considered that instruction librarians, whose professional training and expertise is in infor- mation literacy, might assess information literacy–related aspects of student work products differently from general education faculty, who, while conversant in information literacy, have primary expertise in writing instruction and their own subject areas. In addition, the appropriate timing and setting for a personal librarian intervention is ripe for additional investigation, and UDC librarians plan to consider these factors in the de- velopment of future versions of the personal librarian intervention. Rather than in a general education class such as the one where this intervention was piloted, where there may be less need for intensive academic support, perhaps the most meaningful academic impact of a per- sonal librarian could occur in an upper-level class with increased research demands. While providing a personal librarian later in a student’s academic career may miss an opportunity to expose students to library resources and research skills that they can draw on early in their education, support from a PL during a longer-term or more complex project may result in deeper librarian-student collaboration and repeated opportunities for students to reinforce or refine information literacy skills. However, if further research suggests that there is little or no meaningful academic impact in the class where a personal librarian is assigned, it is possible that the primary academic benefit of the PL program lies in raising awareness of the library and its resources so that students know where to seek support for their academic suc- cess when research needs do arise. In that case, personal librarians may be most appropriate in introductory classes, and additional research may examine whether there are longer-term impacts of having had a personal librarian in introductory courses, such as success in higher- level courses or persistence or retention. As with the rubric scores, the differences in net gain in information literacy confidence between course sections with and without a personal librarian were not statistically signifi- cant, but the fact that the average confidence gain was slightly higher is encouraging for the first semester of the program. Additional data from future semesters may help identify any relationship between information literacy confidence and the presence of a personal librarian. Interestingly, there was a weak negative correlation between attendance at information literacy sessions taught by the personal librarian and net gain in information literacy confidence. The rapid introduction of myriad skills and concepts that may be new to students may result in a decrease in confidence if, as a result of the instruction, students learn what they do not know and identify skills they do not yet have. This correlation may, then, reflect the known limita- tions of one- or two-shot information literacy instruction54 and suggest a need for either ad- ditional teaching opportunities to reinforce the content of the instruction sessions or adjusted pedagogical approaches if librarians can teach only one or two sessions. Among students in course sections assigned a personal librarian, no notable correlation was found between aver- age rubric scores and self-reported engagement with emails from the personal librarian, or between rubric scores and having had at least one consultation with the personal librarian, but there was a weak positive correlation between attendance at information literacy sessions and average scores on the Use of Sources portion of the rubric, which assesses use and proper Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 37 citation of sources. In UDC librarians’ experience, students often report significant difficulty in appropriately summarizing, paraphrasing, and quoting sources in academic writing. The information literacy instruction taught by the personal librarian featured an active learning activity on using sources that, in past sessions, both students and faculty have anecdotally reported as very helpful; it is possible that this activity was the most impactful of those in the instruction sessions and that redesigned active learning activities or different pedagogical approaches on other information literacy topics could support growth in other areas. The strongest correlations in the study—between weak and moderate—were found between gains in information literacy confidence and either self-reported engagement with personal librarian emails or having had at least one consultation with the personal librarian. These correlations suggest that supplemental interactions with the personal librarian, either through email or in person, have more of an impact on information literacy confidence than instruction alone, reinforcing discussions in existing literature about the role of PL programs in relationship-building between librarians and students.55 If instruction alone does not improve confidence, further engagement with a librarian may serve as a natural follow-up on introduc- tory instruction for boosting student confidence, helping students fill in the gap between what they do and do not know that they identified as a result of instruction. A personal librarian may be a particularly useful support for students who are struggling, as her or his presence is a clear, recurring encouragement to ask for help. To be sure, one-on-one librarian-student interaction is easier at smaller or well-staffed institutions, and this approach would be difficult to replicate at larger or less-resourced institutions. This challenge reflects the need to adapt personal librarian programs to an institution’s specific context; larger institutions may need to narrow the scope of the population to be served or the intensity of the intervention. Given the intention of personal librarianship to support both affective and academic needs, as well as indications from the pilot semester of the program that contact with the personal librarian correlates to improvements in information literacy confidence, additional methods of outreach beyond instruction and email may prove beneficial by amplifying the “personal” branding and nature of the intervention. Recognizing that interaction with monthly emails from the personal librarian, as measured by aggregate click rates and student self-reported use of the emails, was more limited than originally anticipated, UDC librarians adjusted student outreach when they continued the personal librarian study in spring 2020. While the monthly emails continued, the personal librarian added monthly in-person pop-in visits to sections as- signed a personal librarian prior to the COVID-19 public health emergency and sent a recorded video pop-in via the university learning management system during remote instruction. As another means of “personalizing” the intervention, UDC librarians hope to time emails to students, instruction, and other outreach in accordance with major assignments or projects on the course syllabus and solicit feedback from students and faculty about their experiences with the personal librarian to make continuous improvements. While “personalizing” per- sonal librarians appears important, it may look different on different campuses, depending on student population, patterns of student life, or preferred communication methods. 38 College & Research Libraries January 2022 A P P E N D I X A Information Literacy–Related Course Outcomes • Demonstrate ability to read and evaluate scholarly research and criticism from a vari- ety of academic disciplines in the arts and humanities, social sciences, natural sciences, mathematics, and other disciplines. • Analyze textual material to evaluate expertise of authorship, validity of support, and implications of claims. • Use texts to compare, synthesize, and analyze information. • Demonstrate writing strategies (prewrite, outline, draft, revise, and cite) in a scholarly argument that is logically valid, rhetorically persuasive, and appropriate to the subject and the audience addressed; summarizing, paraphrasing, and quoting from academic sources. • Demonstrate research skills, integrate their ideas with those of others, and apply the conventions of attribution and citation correctly. Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 39 A P P E N D I X B Information Literacy Confidence Survey This survey will take only one minute! It is meant to measure your confidence with research skills at the end of the semester. Please answer all questions. 1. How confident are you in using library resources? □ Very Unconfident □ Somewhat Unconfident □ Neutral □ Somewhat Confident □ Very Confident 2. How confident are you in finding sources to use in a research paper? □ Very Unconfident □ Somewhat Unconfident □ Neutral □ Somewhat Confident □ Very Confident 3. How confident are you in integrating sources into your research paper writing? □ Very Unconfident □ Somewhat Unconfident □ Neutral □ Somewhat Confident □ Very Confident 4. How confident are you in evaluating possible sources to use in a research paper to select the best ones to use? □ Very Unconfident □ Somewhat Unconfident □ Neutral □ Somewhat Confident □ Very Confident 5. How confident are you in writing citations for the sources you use in a research paper? □ Very Unconfident □ Somewhat Unconfident □ Neutral □ Somewhat Confident □ Very Confident Survey questions included on the end-of-semester survey for students enrolled in sections assigned a personal librarian: 6. This semester, your personal librarian sent you monthly emails about library resources and research. How often did you read the emails? □ Never □ Rarely □ Occasionally □ Often □ Always 40 College & Research Libraries January 2022 7. If you read the emails from your personal librarian, how often did you use the tips or resources provided in the emails? □ Never □ Rarely □ Occasionally □ Often □ Always Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 41 A P P E N D I X C Rubric for Scoring Student Papers Highly Proficient Competent Novice No Evidence Use of Sources 20–25 points The artifact shows high proficiency in the use of primary and secondary sources; demonstrating summaries, paraphrases, and/ or quotes with appropriate in-text citations. 15–19 points The artifact shows adequate proficiency in the use of primary and secondary sources; demonstrating summaries, paraphrases, and/ or quotes with appropriate in-text citations. 10–14 points The artifact shows low proficiency in the use of primary and secondary sources; demonstrating summaries, paraphrases, and/ or quotes with appropriate in-text citations. 0–9 points There is little-to-no proficiency in the use of primary and secondary sources; demonstrating summaries, paraphrases, and/ or quotes with appropriate in-text citations. Format/Structure 20–25 points The artifact shows high proficiency in format (such as title, introduction, body/ paragraphing, and conclusion, and Works Cited/ Reference List). 15–19 points The artifact shows adequate proficiency in format (such as title, introduction, body/ paragraphing, and conclusion, and Works Cited/ Reference List). 10–14 points The artifact shows low proficiency in format (such as title, introduction, body/ paragraphing, and conclusion, and Works Cited/ Reference List). 0–9 points There is little-to- no proficiency in format (such as title, introduction, body/ paragraphing, and conclusion, and Works Cited/ Reference List). Usage (Grammar) 20–25 points The artifact shows high proficiency in the use of Standard Written English (SWE) with 0–4 errors in usage (such as subject/ verb agreement, fragments, and the like) and mechanics (such as spelling, punctuation, and other mechanics). 15–19 points The artifact shows adequate proficiency in the use of Standard Written English (SWE) with 5–9 errors in usage (such as subject/ verb agreement, fragments, and the like) and mechanics (such as spelling, punctuation, and other mechanics). 10–14 points The artifact shows low proficiency in the use of Standard Written English (SWE) with 10–15 errors in usage (such as subject/ verb agreement, fragments, and the like) and mechanics (such as spelling, punctuation, and other mechanics). 0–9 points The artifact shows little-to- no proficiency in the use of Standard Written English (SWE) with more than 15 errors in usage (such as subject/ verb agreement, fragments, and the like) and mechanics (such as spelling, punctuation, and other mechanics). 42 College & Research Libraries January 2022 Highly Proficient Competent Novice No Evidence Reading Ability/ Synthesis 20–25 points The artifact shows high proficiency in having selected, verified, and synthesized sources from a variety of disciplines; with analysis and interpretation of sources in a clear and sustained argument. 15–19 points The artifact shows adequate proficiency in having selected, verified, and synthesized sources from a variety of disciplines; with analysis and interpretation of sources in a clear and sustained argument. 10–14 points The artifact shows low proficiency in having selected, verified, and synthesized sources from a variety of disciplines; with analysis and interpretation of sources in a clear and sustained argument. 0–9 points The artifact shows little-to- no proficiency in having selected, verified, and synthesized sources from a variety of disciplines; with analysis and interpretation of sources in a clear and sustained argument. Notes 1. “The Equity Imperative: A Strategic Plan to Regenerate the University of the District of Columbia as a Public Higher Education Model of Urban Student Success, 2019–2022,” University of the District of Columbia, http://docs.udc.edu/president/UDC-Equity-Imperative-FULL-v11-2-1-19-Singles-SM.pdf [accessed 6 July 2020]. 2. Richard Moniz, “The Origin of the Personal Librarian Concept,” in Personal Librarian: Enhancing the Student Experience, eds. Richard Moniz and Jean Moats (Chicago, IL: ALA Editions, 2014), 8–17. 3. Moniz, “The Origin of the Personal Librarian Concept,” 14–15. 4. Helene LaFrance and Shannon B. Kealey, “A Boutique Personal Librarian Program for Transfer Students,” Reference Services Review 45, no. 2 (2017): 332–45, https://doi.org/10.1108/RSR-10-2016-0066; Amanda MacDonald and Suchi Mohanty, “Personal Librarian Program for Transfer Students: An Overview,” Reference Services Review 45, no. 2 (2017): 346–54, http://dx.doi.org/10.1108/RSR-10-2016-0071; Moniz, “The Origin of the Personal Librarian Concept.” 5. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students,” 350. 6. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students,” 347. 7. Jérôme Melançon and Nancy Goebel, “Personal Librarians for Aboriginal Students: A Programmatic Assessment,” College & Research Libraries 77, no. 2 (2016): 184–96, https://doi.org/10.5860/crl.77.2.184. 8. Joe Eshleman, “Information Literacy and the Personal Librarian,” in Personal Librarian: Enhancing the Stu- dent Experience, eds. Richard Moniz and Jean Moats (Chicago, IL: ALA Editions, 2014), 30–43; MacDonald and Mohanty, “Personal Librarian Program for Transfer Students”; Moniz, “The Origin of the Personal Librarian Concept.” 9. Steven J. Bell, “Retention Matters, But It’s Not the Only Thing That Counts,” Information Outlook 18, no. 1 (2014): 12. 10. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students.” 11. Eshleman, “Information Literacy and the Personal Librarian.” 12. Rebekah Kilzer, “Reference as Service, Reference as Place: A View of Reference in the Academic Library,” Reference Librarian 52, no. 4 (2011): 291–99, https://doi.org/10.1080/02763877.2011.588539. 13. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students.” 14. Eshleman, “Information Literacy and the Personal Librarian.” 15. Eshleman, “Information Literacy and the Personal Librarian,” 37. 16. “Personal Librarian Program,” Drexel University Libraries, https://www.library.drexel.edu/news-and- events/programs-and-initiatives/personal-librarian/ [accessed 6 July 2020]; “Duke University Libraries Residence Hall Librarian Program,” Duke University Libraries, https://library.duke.edu/residence-hall-librarian [accessed 6 July 2020]; Kilzer, “Reference as Service, Reference as Place”; Brian Sullivan, Eva Sclippa, and Trevor Riley, “Librarians, the Gathering,” Library Journal 141, no. 3 (2016): 18–19. Evaluating the Impact of Personal Librarians on Academic and Affective Outcomes 43 17. LaFrance and Kealey, “A Boutique Personal Librarian Program for Transfer Students”; MacDonald and Mohanty, “Personal Librarian Program for Transfer Students.” 18. Lizah Ismail, “Getting Personal: Reaching Out to Adult Learners through a Course Management System,” Reference Librarian 52, no. 3 (2011): 244–62, https://doi.org/10.1080/02763877.2011.556993. 19. Erica England and Leo S. Lo, “Becoming a Librarian BFF: Three Tips to Connect with Your Students,” College & Research Libraries News 78, no. 9 (2017): 510–15; “Online Learners Library Guide: Personal Librarian Program,” University of West Florida University Libraries, https://libguides.uwf.edu/online/personallibrarian [accessed 6 July 2020]. 20. Melançon and Goebel, “Personal Librarians for Aboriginal Students,” 186. 21. England and Lo, “Becoming a Librarian BFF.” 22. Sara J. Beutter Manus, “Librarian in the Classroom: An Embedded Approach to Music Information Lit- eracy for First-Year Undergraduates,” Notes 66, no. 2 (2009): 249–61. 23. Amanda Shannon and Vaughn Shannon, “Librarians in the Midst: Improving Student Research through Collaborative Instruction,” Journal of Political Science Education 12, no. 4 (2016): 457–70. 24. Shannon and Shannon, “Librarians in the Midst.” 25. Beutter Manus, “Librarian in the Classroom”; Trey Lemley, “Virtual Embedded Librarianship Program: A Personal View,” Journal of the Medical Library Association 104, no. 3 (2016): 232–34. 26. Beutter Manus, “Librarian in the Classroom”; Shannon and Shannon, “Librarians in the Midst.” 27. Beutter Manus, “Librarian in the Classroom”; Lemley, “Virtual Embedded Librarianship Program.” 28. Brooke E. Duffy, Lisa Thornell, and Kelleen Maluski, “Librarians Connect to Self-Reflect: Collaboration among Personal Librarian Coordinators at Different Institutions,” Marketing Libraries Journal 2, no. 1 (2018): 75–80; Eshleman, “Information Literacy and the Personal Librarian”; Melançon and Goebel, “Personal Librarians for Aboriginal Students.” 29. Kilzer, “Reference as Service, Reference as Place”; Sullivan, Sclippa, and Riley, “Librarians, the Gather- ing.” 30. Ismail, “Getting Personal.” 31. Duffy, Thornell, and Maluski, “Librarians Connect to Self-Reflect”; Sullivan, Sclippa, and Riley, “Librar- ians, the Gathering.” 32. Cynthia L. Henry, Carrye K. Syma, and Kimberly K. Vardeman, “Reaching Out: Connecting Students to Their Personal Librarian,” Reference Services Review 40, no. 3 (2012): 396–407, https://doi.org/10.1108/00907321211254661. 33. Sullivan, Sclippa, and Riley, “Librarians, the Gathering.” 34. Melançon and Goebel, “Personal Librarians for Aboriginal Students.” 35. Melançon and Goebel, “Personal Librarians for Aboriginal Students.” 36. England and Lo, “Becoming a Librarian BFF.” 37. Eshleman, “Information Literacy and the Personal Librarian.” 38. Duffy, Thornell, and Maluski, “Librarians Connect to Self-Reflect”; LaFrance and Kealey, “A Boutique Personal Librarian Program for Transfer Students”; Melançon and Goebel, “Personal Librarians for Aboriginal Students.” 39. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students,” 352. 40. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students,” 352. 41. Sullivan, Sclippa, and Riley, “Librarians, the Gathering.” 42. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students”; Melançon and Goebel, “Personal Librarians for Aboriginal Students.” 43. Ismail, “Getting Personal”; Judy M. Spak and Janis G. Glover, “The Personal Librarian Program: An Evaluation of a Cushing/Whitney Medical Library Outreach Initiative,” Medical Reference Services Quarterly 26, no. 4 (2007): 15–25; Sullivan, Sclippa, and Riley, “Librarians, the Gathering.” 44. Spak and Glover, “The Personal Librarian Program.” 45. Melançon and Goebel, “Personal Librarians for Aboriginal Students”; Sullivan, Sclippa, and Riley, “Li- brarians, the Gathering.” 46. Ismail, “Getting Personal”; Sullivan, Sclippa, and Riley, “Librarians, the Gathering.” 47. MacDonald and Mohanty, “Personal Librarian Program for Transfer Students”; Melançon and Goebel, “Personal Librarians for Aboriginal Students.” 48. Shannon and Shannon, “Librarians in the Midst.” 49. Shannon and Shannon, “Librarians in the Midst,” 465. 50. Alexis Teagarden and Michael Carlozzi, “Time Enough? Experimental Findings on Embedded Librarian- ship,” WPA: Writing Program Administration 41, no. 1 (2017): 12–32. 51. Serap Kurbanoglu, “Self-Efficacy: An Alternative Approach to the Evaluation of Information Literacy,” 44 College & Research Libraries January 2022 in QQML2009: Qualitative and Quantitative Methods in Libraries, International Conference, Chania, Crete, Greece, May 26–29, 2009, https://pdfs.semanticscholar.org/9eb7/8666f8c7174aafa3207cd273f4137fb18ab6.pdf [accessed 30 November 2020]. 52. “Written Communication VALUE Rubric,” American Association of Colleges & Universities, https://www. aacu.org/sites/default/files/files/VALUE/WrittenCommunication.pdf [accessed 30 November 2020]. 53. “College Navigator: University of the District of Columbia: Enrollment,” National Center for Education Statistics, https://nces.ed.gov/collegenavigator/?q=131399&id=131399#enrolmt [accessed 6 July 2020]. 54. Heidi Julien, Melissa Gross, and Don Latham, “Survey of Information Literacy Instructional Practices in U.S. Academic Libraries,” College & Research Libraries 79, no. 2 (2018): 179, https://doi.org/10.5860/crl.79.2.179. 55. Eshleman, “Information Literacy and the Personal Librarian”; MacDonald and Mohanty, “Personal Li- brarian Program for Transfer Students”; Melançon and Goebel, “Personal Librarians for Aboriginal Students”; Moniz, “The Origin of the Personal Librarian Concept.”