Editorial

Integrating 2014 Library Assessment Conference Proceedings and Peer Review

Martha Kyrillidou

Principal, QualityMetrics LLC

Research Associate, University of Illinois at Urbana-Champaign

Part-time Instructor, Kent State University

2016 Kyrillidou. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

This is the third editorial I am writing for EBLIP. In the prior two occasions I focused my editorial only on those articles from the 2010 and the 2012 Library Assessment Conferences that we invited and published through EBLIP. In 2010 I co-authored the editorial with Damon Jaggars as the effort we had initiated was going to be a special issue of the Journal of Library Administration originally but with debates over the way open access was treated we decided to move the corpus of articles to a venue that had a strong open access mission.

Since then I continued to work with EBLIP in featuring articles from the 2012 conference. This issue features work from the 2014 conference. In this issue my editorial will cover not only the two commentaries (Carlsson, and Doshi, Scharf, and Fox) and the four feature articles that originally appeared in the 2014 Conference Proceedings (Neurohr and Bailey; Reed, McFarland and Croft; Baumgart, Carillo, and Schmidli; and, Ziegenfuss and Borrelli) but also the two original submissions to EBLIP by Applegate as well as Chew, Schoenborn, Stemper and Lilyard.

It is really important and would like to encourage all authors submitting conference proceedings to consider publishing their work in the formal journal literature beyond the proceedings. Going through the peer review submission workflow allows further refinement of ideas and growth of our own professional thinking. Publishing in an open access venue is also an important consideration. Over the years of working for the Association of Research Libraries (ARL), I often felt the tension of publishing in open access platforms while also gaining the reputation necessary that the peer review process can afford us and this is typically for work that was above and beyond what I produced for ARL as work for hire. Our field has an increasing number of venues that are open access, so I am hoping that most of our literature will be available as open access in the future. It is an important consideration for the growth of library assessment and evidence based library and information practice that journals like EBLIP have facilitated and enabled.

The richness of our library assessment work is emerging through four key themes in the current issue: assessment across boundaries (inter- and intra- institutional), organizational improvements based on data, innovative qualitative methods that develop emergent understanding of key aspects of our environment, and articles that focus on the scholarly communication cycle with its emerging emphasis on online profiles for researchers and disciplinary differences of e-journals.

Assessment Across Boundaries: Collaborating Across Institutions and Within the Parent Institution

Two of the pieces presented in this issue discuss strategic assessment elements crossing organizational boundaries. Focusing on student learning outcomes assessment, Ziegenfuss and Borrelli describe an impressive effort within the Greater Western Library Alliance and focusing on a more than a decade long strategy development effort, Carlsson describes the strategy and assessment integration at Gothenburg University in Sweden.

“Exploring the Complexity of Student Learning Outcome Assessment Practices Across Multiple Libraries” is a collaborative qualitative research project, initiated by the Greater Western Library Alliance (GWLA), to explore how librarians were involved in the designing, implementing, assessing, and disseminating student learning outcomes (SLOs) in GWLA member academic libraries. The original objective of the research was to identify library evaluation/assessment practices at the different libraries to share and discuss by consortia members at a GWLA-sponsored Student Learning Assessment Symposium in 2013. However, findings raised new questions and areas to explore beyond student learning assessment, and additional research was continued by two of the GWLA collaborators after the Symposium. The purpose of this second phase of research was to explore the intersection of library and institutional contexts and academic library assessment practices.

“Library Assessment and Quality Assurance - Creating a Staff-Driven and User-Focused Development Process” describes the ways assessment and strategy is linked at Gothenburg University. “The process has both bottom-up and top-down features designed to generate strong staff involvement and long-term strategic stability.” Dating back to 2001 the quality cycle initiated by the university filtered into the library by 2003.The paper describes how the library operates with the quality cycle building feedback mechanisms for improving activities. The quality cycle is viewed as part of the strategic cycle that is an annual process of visioning and refreshing strategy at the library level. An environmental scan provides new intelligence every year on the main areas that need to be the focus of the strategy. “The definition of a yearly process, into which data and previous findings can be funneled, has shown to be a powerful driving force for implementing meaningful change.”

Organizational Improvements Based on Data

Whether it’s about improving reference services or developing advisory structures that help advocate sound improvements in the library, the next two articles have important organizational improvement lessons to share with us.

“The Role of Student Advisory Boards in Assessment” describes the successful deployment of student advisory boards at the University of Central Florida, Georgia Tech and the University of Louisville. These institutions received valuable assessment information through the student advisory board. “The input has been used to trigger additional evaluation and assessment of programs, services, or resources as an indicator of areas that need a quick fix or a longer-term solution, and to focus on trends in campus life that affect use of the library. In all three libraries, insight gained from board members has resulted in positive improvements. Board members can be surveyed between meetings. They can act as a test group for a survey or for questions and topics for focus group and other qualitative research. Minutes and feedback to board members ensure their continued engagement with the library.” Board members become active advocates for the library on campus who can help close the assessment loop by supporting the programs and enhancements … “through invaluable word-of-mouth publicity and through support of campus funding initiatives.” As the authors caution us “boards are not a shortcut to obtaining qualitative assessment information. If done well, everything takes time: recruiting the right students, preparing agendas, distributing minutes. It is only one tool in an assessment portfolio.”

“Iterative Chat Transcript Analysis: Making Meaning from Existing Data” examines patron satisfaction with reference services analyzing an existing corpus of chat transcripts. Having conducted a similar analysis in 2010, the authors also compared librarian behaviors over time. Drawing from the library literature, the authors identified a set of librarian behaviors closely associated with patron satisfaction. These behaviors include listening to and understanding patrons’ needs, inviting patrons to use the service again, and providing instruction or completing a search for patrons. The analysis shows that librarian behaviors have changed over time, pointing to what campus librarians are doing well, and that implementation of best practices at a campus level after the 2010 analysis may have increased these positive behaviors.

Emergent Understandings

Two of the pieces focus on implementation of interesting qualitative approaches – deploying photovoice and content analysis of a variety of evidence sources help us gain emergent understanding of the needs of special user groups and special skills we need to develop as assessment professionals.

“Using Photo-Elicitation with Native American Students to Explore Perceptions of the Physical Library” describes Native American students’ perceptions of the Edmon Low Library at Oklahoma State University (OSU). The study sought to understand how Native American students perceived the role of the academic library in their lives, and which elements of the library students depicted and described as holding meaning for them. Photo-elicitation, a form of visual research and a participatory research method, was the primary method chosen to explore students’ perceptions of the library. Students followed a photo prompt for taking at least fifteen pictures of the library, then participated in two separate interviews with the primary researcher. Participants also completed a demographic/questionnaire form, answered semi-structured questions, and ranked the photos they took. Exploring how individual students who identify as Native American perceive the university library enhanced our understanding of how libraries in Predominantly White Institutions (PWIs) can best serve and support students. This study provided insight into the method of photo-elicitation interviews. This research also provided practical benefits for student participants through increased library knowledge.

“Educating Assessors: Preparing Librarians with Micro and Macro Skills” examined the fit between libraries’ needs for evaluation skills, and library education and professional development opportunities. Many library position descriptions and many areas of library science education focus on professional skills and activities, such as delivering information literacy, designing programs, and managing resources. Only some positions, some parts of positions, and some areas of education specifically address assessment/evaluation skills. The growth of the Library Assessment Conference, the establishment of the ARL-ASSESS listserv, and other evidence indicates that assessment skills are increasingly important. Examining different bodies of evidence such as core competencies, training and course requirements, the authors found that while one-third of job postings made some mention of evaluation responsibilities, less than 10% of conference or continuing education offerings addressed assessment skills. In addition, management as a topic is a widespread requirement in MLS programs (78%), while research (58%) and assessment (15%) far less common. Overall, there seems to be more need for assessment/evaluation skills than there are structured offerings to educate people in developing those skills.

Serving Scholarship

The last two pieces focus on serving the scholarly needs of faculty and students through the development of online profiles and altmetrics and through the use of e-journal metrics.

“Laying the Groundwork for a New Library Service: Scholar-Practitioner & Graduate Student Attitudes Toward Altmetrics and the Curation of Online Profiles” assesses the knowledge base and needs of our academic communities in order to support the creation and maintenance of scholarly online profiles. Participants were queried about use, issues, and attitudes toward scholarly profile and altmetric tools, as well as the role librarians could play in assisting with the curation of online reputation. While all participants had Googled themselves, few were strategic about their online scholarly identity. Participants affirmed the perception that altmetrics can be of value in helping to craft a story of the value of their research and its diverse outputs. Librarians are well-placed to assist scholar-practitioners who wish to curate an online profile or use altmetrics tools. Areas of assistance include: personalized support, establishment of goals, orientation to specific tools, orientation to altmetrics and scholarly promotion landscape, preparing users for potential difficulties, discussing copyright implications, Open Access education, and guidance with packaging content for different venues and audiences.

“E-Journal Metrics for Collection Management: Exploring Disciplinary Usage Differences in Scopus and Web of Science” shares a few important conclusions with us: Collecting and correlating authorship and citation data allows patterns of use to emerge, resulting in a more accurate picture of use activity than the commonly used cost-per-use method. To find the best information on authoring activity by local faculty for subscribed journals, use Scopus. To find the best information on citing activity by faculty peers for subscribed titles use Thomson Reuters’ customized Local Journal Use Reports (LJUR), or limit a Web of Science search to local institution. The Eigenfactor and SNIP journal quality metrics results can better inform selection decisions, and are publicly available. Given the trend toward more centralized collection development, it is still critical to obtain liaison input no matter what datasets are used for decision making. This evidence of value can be used to defend any local library “tax” that academic departments pay as well as promote services to help faculty demonstrate their research impact.

I hope this corpus of articles serves as an inspiration to all of you to continue to innovate in library assessment and evidence based library and information practice and demonstrate the value of our services and libraries to our users through tangible contributions to their improved outcomes and increased impact.

Reference

Hiller, S., Kyrillidou, M., and Oakleaf, M. (2014). The Library Assessment Conference – Past, Present and Near Future! Journal of Academic Librarianship, 3(4), 410–412.