Identifying and Articulating Library Connections to Student Success 227 Identifying and Articulating Library Connections to Student Success Lisa Massengale, Pattie Piotrowski, and Devin Savage Lisa Massengale is the Head of Research and Instruction, Pattie Piotrowski is the Assistant Dean for Public Services, and Devin Savage is the Assistant Dean for Assessment and Scholarly Communication, all at Paul V. Galvin Library, Illinois Institute of Technology; e-mail: lmasseng@iit.edu, piotrowski@iit.edu, and dsav- age@iit.edu. Acknowledgements: The authors would like to thank our project team of Tayyab Arshad, Carol Emmons, and Betsi Pinkus-Huizenga. Attribution: ©2016 Lisa Massengale, Pattie Piotrowski, and Devin Savage. Attribution-NonCommercial 3.0 (http://creativecommons.org/licenses/by-nc/3.0/) CC BY-NC 3.0. Engaging in ongoing assessment is key to libraries demonstrating their value to their institutions. This study is an initial step in a STEM library’s long-term goal of measuring the library’s connection to, and impact on, student academic success markers such as retention and persistence. Initial results showed that any library usage was always accompanied by a slightly higher achievement in GPA for the user. Results will serve as benchmarks for further study. Introduction In order to articulate their value to their institutions, libraries must move beyond compiling and reporting statistics and cannot make assumptions regarding their con- tributions or relevance to the academic community. By actively developing a culture of assessment, libraries can prepare to move forward in this endeavor. As Amos Lakos and Shelley Phipps state: Libraries must to be able to measure their outcomes and systematically make technology, budget allocation, service, and policy decisions based on a range of data—needs assessment data, customer evaluation data, stakeholder expectation data, and internal process and organizational effectiveness data. Pressure to offer value-added service is mounting in intensity, and the rate of change is relentless.1 As library assessment has increasingly become a central part of institutional prac- tice, Galvin Library used the opportunity of the Assessment in Action (AiA) award to begin a systematic process of measuring the library’s connection to, and impact on, student success.2 The library utilized an action research methodology as a means of not just collecting data to be counted, but also engaging in assessment and analysis of that data to craft the stories of library value for our stakeholders. Engaging in action research requires an evolving and ongoing process as results lead to a reformation and improvement of the next steps in the research process. The data collected should raise additional questions that need to be answered and should also prompt a review of the initial research questions. It is a quantifiable way to get to outcomes that can be useful, and not just a system of inputs and outputs. Engaging in action research creates the doi:10.5860/crl.77.2.227 228 College & Research Libraries March 2016 space for enriched conversations with our users and the larger institution.3 Informa- tion we compile, conversations with users, and continuous analysis mean that our value will no longer be viewed as traditional and supportive as previously assumed, but we will be able to articulate to patrons and administrators alike that our value is data-driven and needs-based. For the purposes of this study, the authors partnered with the director of the cam- pus tutoring center who also serves as chair of the Retention Task Force (RTF), the Director of Housing, and the university’s Director of Assessment to study the library’s contribution to university efforts to improve student retention to 90 percent, as well as a review of the library’s provision of excellent service to its students. The study was designed to examine whether intensity of library usage affects student success and to build upon previous assessments of library user demographics. Continuous improvement has been a gradual process for Galvin Library over the last decade. On the qualitative side, Galvin Library has participated in LibQUAL+, an internationally normed library service and quality assessment instrument developed by Texas A&M University (TAMU) and the Association for Research Libraries (ARL), four times since 2004. Galvin Library has continued its commitment to assessment and ongoing transformation, running the LibQUAL+ survey a fifth time in the spring of 2015. Using data from the most current and from previous LibQUAL+ studies, we know that undergraduates’ primary complaint about the library was a dearth of available group study spaces, whereas graduate students suggested that more quiet spaces and comprehensive access to more full-text articles were their primary needs. Findings from service improvement–oriented assessment projects in 2009 and 2011 served as a less comprehensive forerunner in analyzing the gate counts and indicated strong majorities of international students and graduate students entering the library. Results from those studies affected how we think about the users of the Paul V. Galvin Library and led to changes in staffing and collections. The main library for the Illinois Institute of Technology (IIT) is often thought of as a building used primarily by un- dergraduates, but in the 2011 project, the graduate students in particular were shown to outnumber the undergraduates by about a 2:1 ratio. In addition to the internal data generated, the library has also been able to use com- ments generated through two campus-wide studies known as Students Speak, in which service quality is studied across various campus units. Included in the 2011 and 2014 surveys, the library was able to utilize the comments to enhance and improve spaces, services, staffing, and collections. However, the authors consider all of the previous internal assessment efforts to be part of an emergent research process, and as some surveys or statistical measures are repeated, the assessment methodology may change dependent on findings. Another source of student input that has proven valuable is the Student Library Advocates (SLA), a group of fewer than 20 students who meet three to four times per semester with library administration. At the March 10, 2015, SLA meeting, they worked through design exercises focused on library spaces. Led by library administrators, the design exercises were a series of questions that reviewed the current landscape, pre- sented alternative study areas, and surveyed the group on their study preferences. The students had many suggestions for improved signage and other information-sharing mechanisms. And in agreement with our findings from the other assessment projects, namely the Spring 2015 LibQUAL+ survey and a home-grown student space survey, students reiterated the themes of too few available group study spaces for undergradu- ates, whereas more quiet study spaces and access to more online full-text articles were articulated as the graduate students’ primary needs. Particularly valuable was the insight provided by students on the reasons for their behavior, like the commuting student who Identifying and Articulating Library Connections to Student Success 229 valued the long hours that the library is open so that she can just study on campus a few days out of the week. Another insight came from the student who invites his friends to study in groups at the library even if they are in other programs (confirming the “parallel study” trend noted in the library spaces assessment project’s seating sweeps), and even just hearing about the difficulty of finding power outlets during midterms or finals can be helpful for our planning and preparations during these high-volume times. The exercises were also intended to expose the library advocates to trends oc- curring in academic library environments, at both peer and aspirant institutions. This group was formerly an advisory board and is still used occasionally as a self-selected focus group for feedback. The SLA was informed that the library would be engaging in the AiA study, and their response is published in the Results section of this article. Working with this specific student group demonstrates well the democratic nature of action research because involving participants in the research process honors their experience and contributes to changes in the library to better meet student needs. It helps to ensure that the action research process is emergent and responsive, just as analyzing data will help create the next steps of the research process. In this way, action research gives students a voice in their own educational experience. This study set out to establish connections between academic success outcomes such as student retention, persistence, and grade point average (GPA) by demonstrating that Galvin Library contributes to student learning and success through such services as research help, instruction, use of online resources, and interlibrary loan. An assess- ment of library services supports strategic planning, improved processes, and a greater understanding of our users’ needs. The study gathered data generated internally and externally in order to anticipate student expectations, expand services, develop col- lections, and improve access. Analyzing library user data will help the library validate what is believed anecdotally: that using library services has a positive effect on student retention and persistence. There are limited studies that emphasize the impact of the library on the institutional goals of retention and persistence. The University of New Mexico undertook a study that examined the impact academic libraries have on student persistence by focusing on the ratio of library professionals staff to students.4 A study at Ryerson University notably found a .32–.34 increase in GPA among library users, with correlating trends in achievement for light and heavy users of library services.5 The University of Minnesota examined library impact on retention and academic success of first year undergraduate students.6 This examination of user achievement and persistence based upon datasets created through the incidence of individual interactions with the library reflects the larger trend of data usage in the academic library field. This study at Galvin Library adds new variables and, perhaps more importantly, a private STEM university set- ting to studies done at larger public universities like the University of Minnesota and Ryerson. Outside of North America, the United Kingdom Library Impact Data Project performed at the University of Huddersfield demonstrated a statistically significant relationship between library usage and level of degree results.7 Using library data to reveal associations between library usage and student success is a useful element in establishing a complete picture of the library’s value to the institution.8 Methodology As the AiA team approached this project, a goal was set for building upon the work that had been done in previous internal studies, while also creating a dataset that could both act as a benchmark and inform further analysis. Data generated internally was collected in February 2015, and external university-generated data was collected and analyzed in May of 2015. Creating an environment for continuous improvement while allowing 230 College & Research Libraries March 2016 for immediate action and documenting evidence for future articulations of value was a lofty but worthwhile guiding principle on this project. The team first worked with library colleagues to identify service touch points and users for each service: • students physically entering the building • students receiving instruction from a librarian • students visiting the Research Help Office (RHO) • students checking out laptops • students using interlibrary loan • students reserving study rooms • students placing requests for 3-D printer usage • students accessing our online resources The team was able to secure touchpads to collect student ID numbers at the entrance and also installed one in Research Help Office (RHO), but the ID numbers of students attending instruction sessions and borrowing laptops had to be collected via pen and paper. Requests for interlibrary loan, 3-D printing, and study rooms automatically captured student ID numbers in the online forms created for those services. However, to gather the data on which students were accessing the online resources, the Library Technology Department had to turn on EZProxy authentication for all resources in order to capture their NetIDs (whereas IIT Libraries usually provide seamless on- campus access) and then utilize IIT’s Office of Technology Services to provide a list of student ID numbers from the NetIDs. Library Technology was able to support the installation of the touchpads and worked with the Head of Research and Instruction to provide training. Scanning student IDs in order to enter the library is usually done only between the hours of 6:00 P.M. and 7:30 A.M. Posting a librarian at the entry desk to ask the students to scan their cards upon entry had been found helpful in a previous project, so it was successfully adapted for this project. In addition to gaining near-total compliance, the librarian was able to answer any student questions or concerns about the project on the spot, as some students were truly interested in the research topic. Since the number of entries was still lower than the initial benchmark goal, the library team was able to fill out the dataset by coordinating with Campus Access Card and Parking Services to obtain student entry records that are automatically collected when card entry is used during overnight and weekend hours. External Data The library team collaborated with our AiA campus partners to discuss the availability of data sources for students who utilized the campus’s Academic Resource Center, which provides tutoring services, and to identify students who live on campus in dorms or fraternity/sorority housing. From the Retention Task Force the team was able to examine data from the campus’s early warning system, which compiles data and identifies students struggling academically in an effort to provide additional academic assistance and better ensure student success. The early warning system uses GPA, class attendance, extra-curricular activities, and advisor or instructor recommendations to identify students, but currently the early warning system does not include library data. Students who appear in the early warning system are strongly encouraged to take a General Learning Strategies course. Collaborating with campus partners allows the library to take advantage of the extensive amounts of analytics that the university collects in order to assess the influence of student academic practices.9 Results The researchers set out with a long-term goal of measuring the library’s connection to, and impact on, student academic success markers such as retention and persistence. Identifying and Articulating Library Connections to Student Success 231 The results of this first research study were as expected in some areas, and yet also a complete surprise in others. While the library is not yet able to measure the relationship of library services such as instruction, research consultations, interlibrary loan, and use of online resources to persistence, this study did attempt to look at the relationship to achievement. The initial picture looked rather rosy, as every instance of library usage was always accompanied by higher-than-average (mean) GPA. However, it quickly became apparent that the average library student user was, more often than not, actually a graduate student, and graduate students had far better GPAs than undergraduates. When controlling for this variable, the mean GPA of the library-user was usually within a small range (.01–.10) above the total mean, indicating the relationship is weak and more discrete data is needed. It has become apparent that there are additional factors that need to be examined, due to the relative omnipresence of the library and library instruction in student life at IIT. For example, it became clear that the instruction sessions captured not only undergraduate students, but also graduate students, and even more problem- atically (for a controlled research perspective) classes for undergraduates who were involved in an “intervention course” once they had been identified as potential retention risks. The library’s involvement in this successful program is quite valuable, but the way that we gathered the data made it difficult to separate these groups of students. Still, there appears to be an opportunity for further avenues for comparison—such as comparing the retention risk of students who participate in library instruction against that of those that don’t. Our initial baseline was established using the total student dataset from our study and comparing it to the total number of undergraduates at IIT. The cumulative GPA for the entire undergraduate population is 3.02, and library users during the two weeks of the study had GPAs slightly higher than the overall population at 3.09. However, the GPA from our total undergraduate dataset was also 2.99 (see table 1). This data does appear to indicate a small increase in the percentage of undergraduate to graduate users in library use. Whereas previous user studies showed that we had more graduate student users by a 2:1 ratio, in this study the graduate students were greater by only a 1.3:1 ratio. While some narrowing was to be expected by gathering data from what we would have expected to be undergraduate-focused activities such as instruction sessions, 3-D printing, and laptop checkouts, there are other variables to consider. Possible explanations of this increase in the four-year period since the 2011 gate study could be tied to the modest increases in the student body due to the university’s strategic priority of building the undergraduate students to a community TABLE 1 Mean GPA Student Group Number of Students Mean GPA Total Student Dataset 2,413 3.09 Total Undergraduates in Dataset 1,849 2.99 Total Graduates in Dataset 561 3.43 Library Users 644 3.29 Library Undergraduate Students 276 3.09 Library Graduate Students 368 3.44 232 College & Research Libraries March 2016 of 5,000 by 2018, as well as the draw of the library’s unique undergraduate textbook collection, which contains a copy of almost all undergraduate textbooks. Circulation transactions for reserves materials were little more than 6,800 in 2008 before efforts were made to include textbooks, but in 2011 reserve textbook transactions rose to more than 14,700, and by 2013 transactions had jumped to almost 18,000. In addition to serving a stated need of students, the textbook collection is also used as a recruiting tool by undergraduate admissions. Because our research is developmental and responds to what we learn through our process, adding the textbook collection as a data point in the next study is planned. The importance of our textbook collection, on reserve for a two-hour loan period, might be indicated from its high circulation numbers, especially when compared to our general collection (approximately 35,000 against 33,000 over the last two years). In fact, for undergraduate students, our textbook collection circulates over twice as much as the general collection. Furthermore, given the high prices of STEM textbooks and that research on Open Educational Resources has shown that easier student access to curricular materials has correlated with improved outcomes, we think that this might be a data point of particular interest.10 When the users were sorted by the services they used and their GPAs examined, library use of any kind indicated a consistent, if somewhat small, increase of that group’s GPA. The GPAs of those students who reserved rooms in the Group Study Suite showed the highest achievement in correlation, while just simply entering the library correlates to a higher GPA (see table 2). Although the correlation in achievement with utilizing the RHO, attending a librarian-led in- struction session, or using interlibrary loan was slight, there also was an interesting increase in GPA for the 64 undergraduates in our sample who used the library more than once (at 3.14). While there was a slight increase for those who used more than one service over those who used one service multiple times (3.15 to 3.13), it was a small enough increase even without considering the small sample size. Graduate GPA remained steady at 3.43 for the 85 students who used library services multiple times or used multiple services. Considered by itself, it was not a huge increase over the total under- graduate library user GPA (3.09), but on top of the modest increase for undergraduate mean GPA (3.02), this is a significant variable that deserves further study despite its small sample size. This project also aimed to examine the relationships of library activities to each other, specifically looking at Galvin’s online users. Were they just using the electronic resourc- es, or were they availing themselves of other services? Perhaps the most unsurprising connection found was the most significant overlap of student exposure to instruction sessions and electronic resource usage. Library instruction, at least at IIT, has often been heavily focused on student exposure to and usage of electronic resources. However, this data point also seems to support our efforts to increase instruction sessions and student exposure to librarian consultation services. As the library moves away from a service model that has librarians stationed at a desk in our Research Help Office and we seek richer interactions with our patrons, we have been heavily promoting research consultations with subject specialists who can utilize the RHO for such meetings. In TABLE 2 User Group GPA User Groups GPA IIT University Average 3.02 Group Study 3.39 Laptop Use 3.34 Library Entry 3.30 Online User (EZProxy) 3.24 3-D Printers 3.17 Research Help Office 3.13 Instruction 3.13 Interlibrary Loan (ILL) 3.11 Identifying and Articulating Library Connections to Student Success 233 spring of 2016 we have plans to operate a pilot program that would strategically place librarians, for short shifts during busy times, at our circulation service desk on the lower level, which is the only entry point for the library. Often there are information exchanges at that desk that are more reference- or research-related than just circula- tion questions. Staffing this service point and marketing consultations may be a more effective method of reaching students. We do intend to study data generated during this spring semester pilot to compare with data gathered when services are offered at more traditional reference desk. While it was fully expected that the users of the Galvin Library’s online collections would reflect higher use in some areas such as interlibrary loan, which would likely be related to the level of re- search of the user, it was the overlap between the online users who also physically enter the building that was most surprising (see table 3 and figure 1). The prevailing thinking at Galvin Library was that students who used the electronic resources would be less likely to come into the building, and so a separation between online users and library entry was expected. Given that online users and library entry was one of the bigger overlaps, it appears that many students do both. TABLE 3 User Overlap with Online Users (EZProxy) ILL 84.2% Instruction 72.4% Research Help Office 33.3% Library Entry 16.9% 3D Printers 16.7% Laptop Use 15.4% Group Study Rooms 15.1% FIGURE 1 Library Use Relationship Node Map 234 College & Research Libraries March 2016 Conclusions There are a myriad of future research steps that were suggested by the analysis and that will be used to reformulate the study. First, we will establish a data point for use of the textbook collection. This is an obvious draw for getting students in the door, but what is the relationship between students using the textbook collection and academic success? Are our efforts decreasing cost or other barriers to student access to curricular material? Second, we will conduct a more thorough examination of the relationship between various student segments and the library. For example, do the graduate students come to the library because of the nature of their coursework (the fact that they’re graduate students engaging in deeper research), or do they come because they’re international students and are more likely to use the library for study space? If so, we might examine what it is about the space that lends itself to student engagement. Third, we will delve deeper in our work with the Office of Institutional Information in order to differentiate results on a smaller scale, especially with respect to instruction. It is suspected that it would be valuable to break out the at-risk students in retention-oriented classes and compare them against the greater population of at-risk students, as well as against other undergraduates who received library instruction. Fourth, we will be adding additional variables for repeat users of library services to measure library exposure and its effect on persistence. The authors met with the Student Library Advocates in November of 2015 in order to share results and receive feedback. Given their perspective, they can help ascribe meaning and interpret the findings through a valuable personal lens. That this representative body of students has regular opportunities to be informed and to engage with senior library administration continues to lead us to new questions and avenues of mutual benefit for our student users and the library. Hearing the results, the students were interested in the use of additional methods of measuring success other than GPA, such as working an on-campus job, research or scholarly output, or even linking to job or academic placement after graduation. The students engaged in discussing how the library pushed information out to them, and they suggested librar- ians create a departmental e-mail distribution list, promotion of the library hashtag, and the presentation of library announcements graphically. The Advocates asked if the library could be a place where they could receive professional writing assistance. Currently, writing assistance is offered by minimally trained peer tutors, and Career Services has a résumé review service offered twice a week. Given what we know now, Galvin Library has taken some early actions and will be implementing various new service models; librarian consultation hours will be provided in student housing during the evening when such consultation is tradition- ally not available, modules will be revamped for students in the General Learning Strategies classes, and librarians will be strategically placed at service desks outside of the RHO. Communication to faculty will be targeted to increase both class instruction and student consultations by using an established library liaison program. In concert with feedback from assessment done for both internal and external surveys, listening to the voice of students via surveys and the SLA group, the library will continue to stress quality interactions and enhance or develop new services and programs that are measurably effective. Notes 1. Amos Lakos and Shelley E. Phipps, “Creating a Culture of Assessment: A Catalyst for Organizational Change,” portal: Libraries and the Academy 4, no. 3 (2004): 346. 2. Meredith Gorran Farkas, Lisa Janicke Hinchliffe, and Amy Harris Houk, “Bridges and Barriers: Factors Influencing a Culture of Assessment in Academic Libraries,” College & Research Identifying and Articulating Library Connections to Student Success 235 Libraries 76, no. 2 (2015): 150–69. 3. Craig Calhoun, Foreword, in Engaging Contradictions: Theory, Politics, and Methods of Activist Scholarship, ed. Charles R. Hale (Oakland: University of California Press, 2008), xiii–xxvi. 4. Mark Emmons and Frances C. Wilkinson, “The Academic Library Impact on Student Persistence,” College & Research Libraries 72, no. 2 (2011): 128–49. 5. Dana Thomas and Weina Wang, “Evaluating Library Contribution to Student Success” (presentation, Library Assessment Conference, Seattle, WA, August 4–6, 2014). 6. Krista M. Soria, Jan Fransen, and Shane Nackerud, “Library Use and Undergraduate Stu- dent Outcomes: New Evidence for Students’ Retention and Academic Success.” portal: Libraries and the Academy 13, no. 2 (2013): 147–64. 7. Graham Stone and Bryony Ramsden, “Library Impact Data Project: Looking for the Link between Library Usage and Student Attainment,” College & Research Libraries 74, no. 6 (2013): 546–59. 8. Shane Nackerud, Jan Fransen, Kate Peterson, and Kristen Mastel, “Analyzing Demograph- ics: Assessing Library Use across the Institution,” portal: Libraries and the Academy 13, no. 2 (2013): 131–45. 9. Bernd Becker, “Learning Analytics: Insights into the Natural Learning Behavior of Our Students,” Behavioral & Social Sciences Librarian 32 (2013): 63–67. 10. Andrew Feldstein, Mirta Martin, Amy Hudson, Kiara Warren, John Hilton III, and David Wiley, “Open Textbooks and Increased Student Access and Outcomes,” European Journal of Open, Distance and E-Learning, no. 2 (2012), http://www.eurodl.org/?p=archives&year=2012&halfyear=2 &article=533.