Editorial

Outcomes Assessment in Select Papers from the 2012 Library Assessment Conference

Martha Kyrillidou

Senior Director, Statistics and Service Quality Programs

Association of Research Libraries

Washington, District of Columbia, United States of America

2015 Kyrillidou. This is an Open Access article distributed under the terms of the Creative Commons-Attribution-Noncommercial-Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

We continue the tradition of featuring select articles from the Library Assessment Conference in EBLIP. We first featured a few from the 2010 event (Kyrillidou & Jaggars, 2013), demonstrating our commitment to open access publishing. We continue in this issue by having five of the six papers presented here go through peer review, with a sixth that focuses on the process of the research project included as commentary since the findings of this work have already been published in another venue.

It is with great pleasure that we present these six articles, some of them with slightly different and more precise titles compared to the original submissions included in the 2012 Library Assessment Conference Proceedings (Hiller, Kyrillidou, Pappalardo, Self, & Yeager, 2012). These articles feature a diverse array of topics and methods, a trend that can be observed over the last ten years that assessment has been documenting through the conference proceedings (Hiller, Kyrillidou, & Oakleaf, 2014), demonstrating that assessment speaks to all parts and processes of a library organization.

Other papers presented at the 2012 Library Assessment Conference that we invited to consider for publication in EBLIP are already in the process of being published in other peer reviewed journals (e.g., Plum & Franklin, 2015; Lupton & Davidson, 2015). Seeing conference papers successfully published in this and other peer reviewed journals speaks to the quality of the work presented at the conference itself, which boasts ‘effective, sustainable, and practical’ assessment as its tagline.

The feature articles in this issue of EBLIP speak to the two key aspects of the academic mission: research and teaching. Three papers speak to the relation of libraries and research. Rawls writes from an institutional perspective, relating expenditures on electronic resources to scholarly productivity. Gessner, Jaggars, Rutner, and Tancheva write from the perspective of improving library services for humanities doctoral students, while McGowan and Namachchivaya write from the perspective of sponsored research pursued by library staff and linked to organizational strategies. The other three articles (Stemmer and Mahan; Askew; and Donahue) focus on student learning. The authors offer ways to understand and implement different approaches for outcomes assessment, which has become the golden standard in practical and tangible ways for libraries as their future survival is tied to demonstrating the value and benefits they deliver to their constituencies.

Rawls’ article brings five years of data related to libraries and universities, as well as citation data from the Web of Science, to understand the relationship between institutional characteristics and investments and research productivity. He tested a variety of models to examine whether:

the number of journal articles produced by the faculty journal article output of each institution correlated with their libraries’ investment in electronic materials. This inquiry is based on the concept that the speed of access and convenience of use offered by electronic library materials creates efficiencies that should increase research productivity by saving the researcher’s time. Thus the expectation is that institutions investing more in electronic materials should generate more journal articles over a given period.

Rawls’ analysis captures the emerging effect and relationship of research article production and electronic journal spending and has implications for disciplinary perspectives. Electronic journal spending has risen dramatically over the last decade and he documents the relationship between spending levels and scholarly productivity, with a positive and statistically significant correlation. He notes that “expenditures for [electronic] materials have a positive and statistically significant correlation with journal article production,” finding that an increase of 511 additional journal articles produced “from 2005 through 2010 for each additional $1,000,000 spent on electronic materials on average per year.” Rawls’s study captures a baseline prototype:

a university that attracts an average of $1 billion per year in revenue, employs 3,500 faculty members, 100 professional librarians, spends $200 million on research, and spends $5 million apiece on both electronic and nonelectronic library materials is predicted to produce 1,801 articles each year.

We hope to see more refinement of these models in future years as well as a deeper understanding of the relationship between library staff and increased article productivity.

Gessner, Jaggars, Rutner and Tancheva discusses in detail the extensive collaborative ethnographic research methods Cornell and Columbia undertook to complete a study that helped improve services for humanities doctoral students. The results of the study have been published elsewhere (see Gessner, Jaggars, Rutner, And Tancheva, 2011), so the feature published here outlines in detail the methodological rigor behind the qualitative methods utilized, and this is primarily why this piece appears as a commentary. In particular, this work:

examines the processes taken to design and administer a collaborative ethnographic study of humanities doctoral students within an inter-institutional, collaborative framework. Project organization and management, including the creation of instruments and analysis of results across two local research teams and institutional cultures is discussed. Effective communications, among and between project teams, and time management were identified as critical factors for success. Benefits resulting from the project included an improved understanding of the needs of a key user group, a heightened interest in user assessment and data-driven decision making among staff within the partner organizations, and a deeper engagement with important academic administrators on both campuses.

The study utilized focus groups and in depth interviews and utilized a suite of collaborative tools available to Columbia and Cornell, and the authors emphasize the importance of clear, flexible and engaging communication. According to the authors, results from this research:

were used on both campuses to improve services and launch new initiatives targeted at this user population. Results were used at Cornell to plan and implement a pilot immersion program for humanities graduate students and at Columbia as impetus to relocate the graduate student teaching center within the library, among several other initiatives at both universities.

Namachchivaya and McGowan present a mixed methods analysis of seven years of sponsored research projects at the University of Illinois Library at Urbana-Champaign. They analyzed proposals and funding trends in a quantitative way and provided context based on in-depth interviews with principal investigators. The authors were seeking to understand:

the research trends and themes over that period. The analysis was aimed at identifying future research trends and corresponding support opportunities. Goals included developing institutional research themes that intersect with funding priorities; demystifying grant writing and project management through professional development programs, increasing communication about grant successes; and bringing new faculty and academic staff into these processes.

The authors report that their research has been valuable for the library’s institutional practices and its grant writing initiatives, concluding that “the most important outcome of the assessment was that it revealed the need for the library to support grant efforts as an integral component of the research process.”

In Stemmer and Mahan we see one of the emerging studies that link library use to student outcomes. They explored a) whether library use influences freshman retention, b) whether it is linked to four-year graduate rates, and c) whether it affects the cumulative GPA for freshmen and seniors. In all cases they found positive relations indicating the importance of library as place for freshmen and the importance of developing good study habits as students progress through a four-year higher education program. As the factors that influence student outcomes differ depending on what stage of their program students find themselves, questions are raised of how this information can be used by libraries to offer interventions and improvements in the students’ learning trajectories. The authors have plans to continue this line of inquiry, supplementing a rich set of studies that are taking place in this area.

Regarding developing specific intervention services, Donahue reports on an evaluation study of a peer2peer service at the University of New Hampshire Manchester for approximately ten years. This service is a collaboration between the library and the college’s Center for Academic Enrichment (CAE). A critical component of this collaboration is the incorporation of peer writing tutors trained in basic library research skills who work side-by-side with instruction librarians in the classroom over the course of a semester. The study uses a mixed methods approach and provides valuable baseline evidence for informing teaching and learning practices. Furthermore, it has led to exploring future approaches for deepening the understanding of peer2peer mentoring during a first-year writing course for imparting critical information literacy skills.

Finally, Askew discusses the implementation of assessment of roaming reference services. She reviews results from a pilot study at Florida International University through inputs, outputs, qualitative data, and organizational perspectives offered by the roaming librarians. She emphasizes the need to iteratively improve new services and the critical importance of taking advantage of new mobile technologies to offer roaming services throughout the campus, even outside library buildings.

These curated feature articles speak to the maturation of library assessment in shifting towards outcomes perspectives, from the summative to the formative, by engaging mixed methods, and by addressing the need to relate to research and learning outcomes. Learning and research are ultimately intertwined, and depend on past experiences, knowledge, and the perspectives our students and faculty bring as they approach the information rich environments they are constantly exposed to.

References

Gessner, C. G., Jaggars, D. E., Rutner, J., & Tancheva, K. (2011). Supporting humanities doctoral student success: A collaborative project between Cornell University Library and Columbia University Libraries. Retrieved from the Council on Library and Information Resources website: http://www.clir.org/pubs/ruminations/02cornellcolumbia/report.html/report.pdf

Hiller, S., Kyrillidou, M., & Oakleaf, M. (2014). The Library Assessment Conference – Past, present and near future! Journal of Academic Librarianship, 40(3-4), 410-412. http://dx.doi.org/10.1016/j.acalib.2014.05.013

Hiller, S., Kyrillidou, M., Pappalardo, A., Self, J., & Yeager, A. (2012). Proceedings of the 2012 Library Assessment Conference: Building Effective, Sustainable and Practical Assessment. Washington, DC. Association of Research Libraries.

Kyrillidou, M., & Jaggars, D. (2013). Current themes in academic library assessment: Select papers from the 2010 Library Assessment Conference. Evidence Based Library and Information Practice, 8(2), 4-8. http://ejournals.library.ualberta.ca/index.php/EBLIP/index

Lupton, A., & Davidson, C. (2013). Assessing the value of e-resources to York University faculty using the MINES for Libraries protocol: An evolving landscape. Journal of Web Librarianship, 7(4), 422-433. http://dx.doi.org/10.1080/19322909.2013.839849

Plum, T., & Franklin, B. (2015). What is different about e-books?: A MINES for Libraries® analysis of academic and health sciences research libraries’ e-book usage. portal: Libraries and the Academy, 15(1), 93-121. http://dx.doi.org/10.1353/pla.2015.0007