150 Bridges and Barriers: Factors Influencing a Culture of Assessment in Academic Libraries Meredith Gorran Farkas, Lisa Janicke Hinchliffe, and Amy Harris Houk Meredith Gorran Farkas is Faculty Librarian at Portland Community College, e-mail: meredith.farkas@ pcc.edu; Lisa Janicke Hinchliffe is Coordinator of Instruction and Information Literacy Services at Uni- versity of Illinois, Urbana-Champaign, e-mail: ljanicke@illinois.edu; Amy Harris Houk is Information Literacy Program Coordinator and Reference Librarian at University of North Carolina, Greensboro, e- mail: a_harri2@uncg.edu. © 2015 Meredith Gorran Farkas, Lisa Janicke Hinchliffe, and Amy Harris Houk, Attribution-NonCommercial (http://creativecommons.org/licenses/by-nc/3.0/) CC BY-NC. In an environment in which libraries need to demonstrate value, illustrat- ing how the library contributes to student learning is critical. Gathering and analyzing data to tell the library’s story as well as identify areas for improvement require commitment, time, effort, and resources—all components of a culture of assessment. This paper presents the results of a survey designed to understand what factors facilitate the develop- ment of a culture of assessment of student learning in academic libraries and what factors may hinder it. Unlike previous research in this area, which has focused on case studies and surveys with nonrepresentative samples, the authors conducted a systematic survey of academic libraries at four-year institutions in the United States and achieved a 42 percent response rate. The results suggest certain factors are highly associated with a culture of assessment and provide guidance to administrators and front-line librarians working to build such a culture. n Lakos and Phipps’ seminal work on creating an assessment culture in academic libraries, they included a definition of a culture of assessment from a presentation given by Lakos, Phipps, and Wilson, which is often quoted: A Culture of Assessment is an organizational environment in which decisions are based on facts, research, and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for customers and stakeholders. A Culture of Assessment exists in organizations where staff care to know what results they produce and how those results relate to customers’ expectations. Organizational mission, values, structures, and systems support behavior that is performance and learning focused.1 By this definition, a culture of assessment is conceived of at the organizational level and is largely fostered by the leadership. The description also suggests that, in some doi:10.5860/crl.76.2.150 crl14-601 Factors Influencing a Culture of Assessment in Academic Libraries 151 cases, an assessment culture requires a cultural shift in organizations. In his article “Contra Assessment Culture,” Ennis argues that “‘assessment culture’ is code for not just doing assessment but liking it.”2 This requires an internalization of the value of assessment, which does not necessarily come from doing assessment alone. For the purposes of this study, the authors define a culture of assessment of student learning as one where assessment is a regular part of institutional practice. It is a core part of what the library does, just like materials acquisition or reference service. Conse- quently, it is well-supported and integrated through everything the library does. When a new service is planned for, part of the planning will include examining existing data and determining how the new service will be assessed. Assessment is not something the organization does because of external pressures from the institution or accrediting bodies; it is something the organization does because librarians and staff want to better understand library users and how they can improve services and collections to meet user needs. When assessment becomes the norm and is no longer seen as something extra, it has become part of the culture. Dozens of articles have been written on what it takes to build a culture of assess- ment in libraries or higher education. While the authors of such articles do not always agree on what is required, there are certain themes that are echoed in a large number of these works: a learning culture that is user-focused; administration that values as- sessment in both word and deed; support in the form of time and/or funding; clear and agreed-upon expectations for assessment; having assessment work built into reward structures; faculty and staff buying into the value of assessment and feeling a sense of ownership of assessment; an environment where faculty and staff trust each other and believe that assessment results will not be used against them; a commitment to faculty or staff training in assessment and/or the creation of learning communities around assessment; technologies to support data collection and analysis; a focus on assessment for learning and improvement rather than accountability; the use of as- sessment results in individual and organizational improvement; and the availability of assessment data to all stakeholders.3 Most articles written on building a culture of assessment are case studies that have explored the paths that individual libraries, departments, and institutions have taken to create a culture of assessment in higher education. Some have also been written by fac- ulty and administrators with extensive assessment experience. While case studies and anecdotes can provide helpful insights and ideas, they are not a substitute for research that looks more systematically at assessment practices and barriers in the profession. Few quantitative or qualitative research studies on what facilitates or hinders the creation of an assessment culture have been undertaken by libraries or units within higher education. Oakleaf and Hinchliffe surveyed instruction librarians to determine how many assess student information literacy skills and whether or not libraries that do assessment use the results to improve practice.4 They distributed the survey on two information literacy instruction listservs and received responses from 437 librarians. They found that 76 percent of librarians surveyed assessed information literacy skills and, of those, 68 percent used the results. Those who did not use their assessment results cited lack of time, lack of knowledge about assessment, lack of support, and a lack of clear expectations as barriers to doing so. Some also did not feel confident that the assessments they conducted meaningfully assessed student learning. While not explicitly about creating a culture of assessment, this study uncovered many of the same barriers to building a culture of assessment that are suggested in the broader higher education assessment literature. A few studies from organizations within academic librarianship provide valuable insights into assessment practices and issues. Troll Covey explored the assessment 152 College & Research Libraries March 2015 needs, practices, and concerns of libraries in her role as a Digital Library Federation (DLF) distinguished fellow.5 She conducted interviews with 71 individuals at 24 DLF libraries and found that librarians were concerned with how to focus on collecting meaningful data, best understand their users, develop in-house skills in assessment, develop realistic assessment plans, and make assessment a core activity in their library. In 2007, Wright and White, as members of the Association of Research Libraries (ARL), surveyed ARL member libraries and developed a SPEC Kit that examined the state of library assessment in ARL libraries.6 They found that assessment activities had increased significantly among member libraries, though how assessment activities were coordinated organizationally varied widely. They found that 46 percent of re- sponding libraries had an assessment plan but that only 19 percent believed their staff had the necessary assessment skills and expertise and 11 percent believed their staff development programs in assessment were adequate. Sixty-eight percent reported that assessment is a library priority, and 79 percent reported that their library administra- tors are committed to supporting assessment. This indicates a significant gap between administrative priorities and activities that would ensure assessment practices. While the DLF and ARL studies are quite helpful in creating a picture of the library assessment landscape, they primarily represent the practices and concerns of academic research libraries with large budgets and staff. Additional research on creating a culture of assessment in libraries explores factors that facilitate success. Hiller, Kyrillidou, and Self reported on what they had learned from working with 24 ARL libraries in the ARL-sponsored project “Making Library Assessment Work: Practical Approaches for Developing and Sustaining Effective As- sessment.” From their work with the participating institutions, they concluded that the two most critical determinants for successful assessment were library leadership that promoted, supported and used assessment, and an organizational culture that was customer-centered and motivated to improve library services. If those two were lacking, it was unlikely that a library could perform useful and sus- tained assessment.7 Ndoye and Parker conducted a similar study of the factors that help an institu- tion of higher education build and sustain a culture of assessment.8 They found that leadership, faculty involvement, resources (such as money and time), and the use of assessment data were the most frequently cited facilitating factors among both institu- tions with established cultures of assessment and institutions that were in the process of establishing cultures of assessment. While these surveys, case studies, and observations provide valuable insights for those looking to build an assessment culture, none of them have surveyed or explored assessment through a systematic sample. Oakleaf and Hinchliffe distributed their survey on two information literacy instruction listservs, including one composed of instruction librarians who had received training in student learning assessment. Ndoye and Parker distributed their survey to two national assessment listservs. It is likely that those who subscribe to assessment listservs are more interested in the topic than the average member of academe. Hiller, Kyrillidou, and Self collected information from libraries that paid to participate in an assessment development program. Again, this self-selected group is far from representative of academic libraries as a whole. The authors of this study sought to better understand what factors facilitate and hinder academic libraries in building a culture of assessment, with a particular focus on instructional assessment. When library directors ask their instruction or assessment coordinators what it takes to build a culture of assessment, it is frustrating that the Factors Influencing a Culture of Assessment in Academic Libraries 153 profession lacks any wide-scale and systematic study of this topic. The goal of this study was to produce a more systematic analysis with a sample that was truly repre- sentative of academic librarianship so that reliable conclusions could be drawn from the data from academic libraries of any size or configuration. Methods Unlike most previous surveys that examined assessment culture-building in libraries or higher education, the present study was designed to systematically survey individual libraries rather than librarians. The survey was distributed to academic librarians at all four-year institutions in the United States and gathered information about institu- tional characteristics, the presence or absence of an assessment culture in the library, and what factors have facilitated or hindered the library organization in building an assessment culture. To ensure that only a single response was submitted per institution, the authors sought to e-mail individual library directors rather than putting out a call for respon- dents on a listserv. Each library director received a unique link to the survey that could only be used once, which protected against multiple individuals from a single institution completing the survey. In the recruitment e-mail, each library director was asked to forward the survey to the most appropriate individual at the library who could com- plete the survey, ideally, someone leading the library’s instructional assessment efforts. To develop a list of contact information for academic library directors at four-year institutions in the United States, spreadsheets with all of the institutions listed under the Carnegie Basic Classifications of Baccalaureate, Master’s, Doctoral, and Research were downloaded from the Carnegie Foundation for the Advancement of Teaching website.9 This represented a total of 1,835 institutions, and their breakdown by Bac- calaureate (BA), Master’s (MA), and Doctoral/Research (PhD) can be seen in table 1. The name and e-mail address for each library director were identified using library websites and other publicly available websites. If a director’s name or contact informa- tion could not be found, a generic e-mail address for the library was used. In some cases, neither the director’s contact information nor any contact information for the library could be found. In the majority of cases, these were for-profit institutions at the BA/MA level that either did not have a library or had no evidence on the Web that a library existed. Some of the institutions were listed more than once even though they had only one library, or a single library served two separate institutions. The authors also excluded their own institutions from the sample. In the end, recruitment e-mails were sent out to a total of 1,604 institutions, as shown in table 1. TABLE 1 Responses by Basic Carnegie Classification Total Institutions on Carnegie List by Classification Total E-mails Sent to Library Directors by Classification Responses Received by Classification Received, Percent of Total Sent BA 810 (44%) 645 (40%) 236 (35%) 37% MA 727 (40%) 671 (42%) 263 (39%) 39% PhD 298 (16%) 288 (18%) 155 (23%) 54% Other/No response N/A N/A 21 (3%) N/A Total 1,835 (100%) 1,604 (100%) 675 (100%) 42% 154 College & Research Libraries March 2015 The survey, which was deployed using the Qualtrics web-based survey software, contained a total of 63 questions. Through the use of skip logic, respondents saw ques- tions based upon their response to previous items and thus no respondent answered more than 47 of the questions. This allowed the authors to tailor their questions based on whether the library had or did not have a culture of assessment. The first group of thirteen questions, which every respondent answered, asked about institutional characteristics, such as Carnegie status, regional accrediting organization, faculty status, and the existence of specific assessment-focused roles or committees. Most important, in this section respondents were asked to report whether or not their library had a culture of assessment, which was defined in the survey as “where as- sessment is a regular part of your institutional practice.” The questions common to all survey respondents were followed by yes/no ques- tions regarding the presence or absence of sixteen facilitating factors and whether their presence helped facilitate a culture of assessment or the absence hindered. The full list of facilitating factors was generated based on factors identified as facilitating the creation of an assessment culture in the library literature. The questions related to each facilitating factor are listed below. • Is there a campuswide assessment initiative at your college or university? • Is the library involved in the campuswide assessment initiative? • Are there clear expectations for assessment in your library? • Is assessment a priority of library administration? • Has your library or library instruction program adopted learning outcomes? • Does your library have an assessment plan? • Is there a shared understanding of the purpose of assessment in your library? • Are library faculty/staff adequately supported in their assessment work? • Is education and training related to assessment work offered or supported? • Does your library have the necessary skills in-house to develop and conduct meaningful assessments and analyze the results? • Does your library have access to systems/technologies that support your as- sessment work? • Does your library leadership offer explicit support (time, funding, and the like) to get library faculty/staff involved in assessment? • Are assessment data available to interested parties? • Does the library leadership use assessment data systematically in decision making and planning? • Do librarians use assessment data to improve their practice? • Is your library’s culture user-focused? After a respondent answered whether a particular factor was present, the survey asked if the presence facilitated the culture of assessment or if the absence was a bar- rier to developing a culture of assessment. Finally, there were three open-ended questions of which respondents saw two. If they responded that their library had a culture of assessment they were asked to name “which of the elements mentioned in this survey (or others) do you think have contributed most to your library developing a culture of assessment and why.” If they responded that their library did not have a culture of assessment, they were asked which elements prevented their library from achieving a culture of assessment. All respondents also had the option of responding to a final open-ended question asking if there was anything else they would like to disclose about their library’s instructional assessment program. Results The survey received a total of 675 responses, only six of which were incomplete. Two Factors Influencing a Culture of Assessment in Academic Libraries 155 of the incomplete responses skipped only the open-ended questions and three of them only answered the first question. The authors decided to retain the incomplete responses since they all answered, at a minimum, the question asking whether they would describe their library as having a culture of assessment. The survey received responses from 42 percent of the institutions contacted and thus a margin of error of only 3.79 percent with a confidence level of 99 percent can be claimed. Table 1 also shows the total number of responses received by Basic Carnegie Clas- sification. The survey received responses from between 37 and 54 percent of institu- tions in each of the Carnegie classifications. In analyzing the responses received, PhD-granting institutions are slightly overrepresented and BA-granting institutions are somewhat underrepresented. However, the responses within each subgroup are well within the thresholds for representativeness. For the BA-granting institutions’ subpopulation, with a confidence level of 95 percent, there is a margin of error of 5 percent. For the MA-granting institutions, with a confidence level of 95 percent, there is a margin of error of 4.72 percent. For PhD-granting institutions, with a confidence level of 95 percent, there is a margin of error of 5.36 percent. Each respondent was asked to indicate whether their institution was public or private and whether it was for-profit or not-for-profit. Forty-one percent (277) of responding institutions were classified as public and 59 percent (395) as private. Public institutions were slightly overrepresented among respondents versus their numbers in the total population. This may be partially due to the number of private institutions, particu- larly BA-granting, for which the authors could not find library contact information. Twelve percent of institutions on the original Carnegie Foundation lists were for-profit institutions, but they only represent 4 percent of total survey respondents. With only 29 respondents from for-profit institutions, it would be difficult to do any analysis around this specific subpopulation. Due to the increasing focus of accrediting organizations on assessment, the survey asked respondents to identify their regional accrediting organization. The results are summarized in table 2. The widely varying representation can be explained by the widely varying sizes of the accrediting bodies. For example, the Northwest Commission on Colleges and Universities (NWCCU) currently accredits fewer than 200 institutions of higher education, while the Southern Association of Colleges and Schools (SACS) accredits more than 800. With the exceptions of Middle States, which is overrepresented, and North Central, which is underrepresented, the responses received from each accrediting body is quite proportional to their representation in the population. TABLE 2 To What Regional Accrediting Organization Does Your Institution Belong? Accrediting Organization Response Percentage Middle States Association of Colleges and Schools 145 22% New England Association of Schools and Colleges (NEASC) 56 8% Northwest Commission on Colleges and Universities (NWCCU) 35 5% North Central Association of Colleges and Schools (NCA) 213 32% Southern Association of Colleges and Schools (SACS) 171 25% Western Association of Schools and Colleges (WASC) 36 5% None of the above 16 2% 156 College & Research Libraries March 2015 Tenure and faculty status are sometimes cited in the literature as both facilitat- ing factors and hindrances in building a culture of assessment, so the authors asked respondents what status the majority of their librarians have. 10 A total of 65 percent (n=456) of respondents have faculty status of some sort, while 35 percent (n=233) do not. Of those with faculty status, 54 percent (n=245) are tenure track and 46 percent (n=191) are non–tenure-track faculty. The numbers in each status category—tenure track faculty (37%), non–tenure-track faculty (29%), and staff (35%)—are quite close to each other. Since the sample is representative of the population of academic libraries in the United States, it can be inferred that the numbers are similar in the general population. Respondents were also asked specifically for their role at their institution, since the authors asked each library director to have the most appropriate individual at the institution complete the survey. Respondents ranged from entry-level reference and instruction librarians to library administrators, but the majority of respondents had some role in library public services. The most common respondents were library ad- ministrators at 43 percent (n=287) and instruction or information literacy coordinators at 38 percent (n=256). The remaining 19 percent of respondents were heads of public services (8%, n=53), public services librarians (5%, n=32), other leadership positions within public services (3%, n=23), assessment coordinators (2%, n=15), and positions outside public services (1%, n=6). The authors hypothesized that the existence of positions and committees dedicated to instruction and assessment would help foster a culture of assessment and indicate a type of administrative/organizational commitment to assessment. Respondents were asked about the presence of specific positions and committees in their institutions. The most common position responding institutions have is a coordinator or head of instruction, a position that 63 percent (n=424) of institutions report having. Only 14 percent (n=94) report having an assessment coordinator or head of assessment and a full 34 percent (n=231) do not have either position in their library. Similarly, the com- mittee that is most common in academic libraries is one focused on instruction (44%, n=299). The existence of assessment committees is more common, at 23 percent (n=156), than the existence of assessment coordinators. Some 10 percent (n=69) of responding institutions have a committee specifically dedicated to instructional assessment. How- ever, nearly half of all institutions (46%, n=308) do not have committees dedicated to instruction or assessment, much less both. Given that many of the libraries represented in the survey are quite small, this may not be an indication of a lack of interest in as- sessment, but in a lack of staffing. State of Assessment in U.S. Academic Libraries While the authors were seeking to examine the factors that facilitate or hinder libraries in achieving a culture of assessment, the information collected in the survey provided an interesting portrait of the state of assessment in academic libraries at four-year institutions in the United States. Given the representativeness of the sample, it can be assumed that the results described are quite similar to those one would find in the entire population of U.S. academic libraries at four-year institutions. The first question asked was about whether the respondent would describe his or her library as having a culture of assessment and described this as an institution “where assessment is a regular part of your institutional practice.” As shown in figure 1, 59 percent (n=400) of respondents indicated that the library currently had a culture of assessment, while 41 percent (n=275) did not. The numbers differed when looked at by Carnegie classification, though not significantly. Libraries in 57 percent (n=88) of doctoral/research institutions, 62 percent (n=165) of master’s colleges, and 56 percent (n=132) of baccalaureate colleges reported having a culture of assessment. Factors Influencing a Culture of Assessment in Academic Libraries 157 The survey asked respon- dents to indicate whether certain facilitating factors existed in their library or the larger academic institution. The frequency of responses for each of the facilitating factors is available in table 3, but a few of the results stand out. The vast majority of academic libraries (85%, n=567) were part of institu- tions where a campuswide assessment initiative existed and the majority of those (79%, n=447) also were in- volved in the campuswide initiative. A full 94 percent (n=629) of respondents indi- cated that their library’s cul- ture is user-focused and 78 percent (n=520) agreed that assessment is a priority of the library administration. Other facilitating factors that were present in a high percentage of libraries are “libraries use assessment data to improve practice” (71%, n=473), “assessment data are available to interested parties” (74%, n=495), “library has access to systems/technologies that sup- port assessment work” (69%, n=459), and “education and training related to assessment are offered or supported” (67%, n=445). These percentages are significantly higher than the percentage of respondents that indicated they have a culture of assessment, so it is unlikely that the presence of any of these factors alone can guarantee that a library will attain a culture of assessment. There were also two facilitating factors that were well below the percentage of libraries that indicated having a culture of assessment. Interestingly, both related to having a clear sense of direction for the assessment program. Only 46 percent (n=305) of respondents indicated that the library has clear expectations for assessment and only 41 percent (n=271) indicated that the library has an assessment plan. It is notable that so many libraries lack a basic sense of direction for their assessment programs. Significant Associations with a Culture of Assessment To determine the relationship between a culture of assessment and the other char- acteristics or facilitating factors analyzed in this study, the authors conducted tests of statistical significance. Qualtrics makes it easy to visualize relationships between variables with cross-tabulations and provides a Chi-square value, degrees of freedom and, ultimately, a p-value for each cross-tabulation. To confirm what was found via Qualtrics analysis, the authors also independently conducted Fisher’s exact test to determine an exact p-value using GraphPad.11 The authors found that both tests pro- duced consistent results. It is as interesting to see what is significantly associated with having or not having a culture of assessment as it is to see what is not. As described in the previous section, there was some variation in the numbers of BA-, MA-, and PhD-granting institutions FIGURE 1 Would You Describe Your Library as Having a Culture of Assessment, Where Assessment is a Regular Part of Your Institutional Practice? Yes 59% No 41% 158 College & Research Libraries March 2015 that reported having a culture of assessment, but the differences were not statistically significant (p=0.27). There also was no significant difference between public and private institutions in terms of whether they had a culture of assessment (p=0.82). Faculty status also was not found to be significantly associated with having a culture of assessment. The authors looked at faculty status in two different ways and found that in neither case were there significant associations. When looking at librarians who are tenure track versus those who are not, a p-value of 0.40 was found. For librarians who have faculty status versus those who are staff, the p-value was 0.63. Faculty status was also not sig- nificantly associated with being involved in a campuswide assessment initiative (p=0.77). One of the characteristics that is most significantly associated with a culture of as- sessment is impossible for a library to change—the regional accrediting association for its institution. Displayed in table 4, the percentage of libraries, organized by accredit- ing association, indicating they had a culture of assessment ranged from 44 percent among the Western Association of Schools and Colleges (WASC) to 73 percent among the Southern Association of Colleges and Schools (SACS). Four of the six regional accrediting agencies fell below the average reported frequency of a culture of assess- ment (59%): New England Association of Schools and Colleges (54%), North Central Association of Colleges and Schools (52%), Northwest Commission on Colleges and Universities (49%), and Western Association of Schools and Colleges (44%). TABLE 3 Responses to Facilitating Factors Questions Facilitating Factor Yes No Library culture is user-focused 629 (94%) 39 (6%) Campuswide assessment initiative exists 570 (85%) 101 (15%) Library is involved in campuswide assessment initiative 450 (79%) 116 (21%) Assessment is a priority of library administration 523 (78%) 148 (22%) Assessment data is available to interested parties 498 (74%) 173 (26%) Librarians use assessment data to improve practice 473 (71%) 195 (29%) Library has access to systems/technologies that support assessment work 459 (69%) 209 (31%) Education and training related to assessment are offered or supported 445 (67%) 223 (33%) Library has adopted learning outcomes 399 (60%) 269 (40%) Library leadership uses assessment data systematically in decision making 391 (59%) 277 (41%) Library leadership offers explicit support to get faculty/ staff involved in assessment 390 (58%) 281 (32%) Library staff/faculty are adequately supported in their assessment work 373 (56%) 295 (44%) Library has necessary skills in-house to develop and conduct meaningful assessments and analyze the results 374 (56%) 294 (44%) Shared understanding of the purpose of assessment in the library 363 (54%) 305 (46%) Clear expectations for assessment in the library 308 (46%) 363 (54%) Library has an assessment plan 271 (41%) 688 (59%) Factors Influencing a Culture of Assessment in Academic Libraries 159 All of the facilitating factors in the survey were significantly associated with having a culture of assessment, most of which had a p-value of <0.001. This indicates that the presence or absence of the facilitating factors is strongly associated with the presence or absence of a culture of assessment. What this does not tell us is the directionality of that influence. Does the culture of assessment exist because of the facilitating factor or does the facilitating factor exist because a culture of assessment already exists? Figure 2 shows how many libraries report having a culture of assessment that also reported the presence or absence of each facilitating factor. From this graph, it is clear that sig- nificantly more libraries reporting the presence of facilitating factors also report having a culture of assessment. With some facilitating factors, very few responding libraries that report their absence have achieved a culture of assessment. Those include libraries whose culture is not user focused, where assessment is not an administrative priority, which are not involved in a campuswide assessment initiative, or whose institution does not have an assessment initiative. The authors also looked at the differences between libraries that had roles and com- mittees related to instruction and assessment versus libraries that did not. A total of 78 percent (n=73) of libraries with assessment coordinators reported having a culture of assessment as did 64 percent (n=271) of institutions with an instruction coordinator. Only 50 percent of libraries with neither role reported having a culture of assessment versus 64 percent (283) that reported having at least one of the positions. This is a statistically significant difference (p<0.001). There were also significant differences (p<0.001) found between libraries that have committees devoted to instruction and/or assessment and libraries that do not have such committees. Respondents who reported having instructional assessment com- mittees had the highest percentage of institutions with a culture of assessment at 83 percent (n=57), followed by assessment committees with 79 percent (n=123) and in- structional committees with 68 percent (n=202). Of the libraries reporting not having committees dedicated to instruction or assessment, only 48 percent (n=149) reported TABLE 4 Percentage of Institutions Reporting a Culture of Assessment by Regional Accrediting Association Accrediting Association Yes, My Library Has a Culture Of Assessment No, My Library Does Not Have a Culture of Assessment Middle States Association of Colleges and Schools 63% (n=91 ) 37% (n=54) New England Association of Schools and Colleges (NEASC) 54% (n=30) 46% (n=26) Northwest Commission on Colleges and Universities (NWCCU) 49% (n=17) 51% (n=18) North Central Association of Colleges and Schools (NCA) 52% (n=111) 48% (n=102) Southern Association of Colleges and Schools (SACS) 73% (n=125) 27% (n=46) Western Association of Schools and Colleges (WASC) 44% (n=16) 56% (n=20) None of the above 50% (n=8) 50% (n=8) 160 College & Research Libraries March 2015 having a culture of assessment versus 69 percent (n=251) for those that have at least one kind of instruction or assessment committee. Interestingly, there were also significant associations found between libraries with instruction or assessment committees and the library being involved in a campuswide assessment initiative (p<0.001). Fully 85 percent (n=262) of libraries with a committee focused on instruction or assessment are also involved in their campuswide assessment initiatives versus only 72 percent (n=190) for those libraries without a committee. The same association was not found between positions dedicated to instruction and/or assessment and library involvement in the campuswide assessment initiative (p=0.84). Most Significant Facilitating and Hindering Factors for Building a Culture of Assessment Since each of the facilitating factors were found to be significantly associated with a culture of assessment, the authors used several methods of analysis to explore which of them might most significantly facilitate or hinder the creation of a culture of as- sessment. First, the cross-tabulations between the facilitating factors and the culture of assessment questions were analyzed to see which facilitating factors were associated with the highest percentages of libraries that had achieved a culture of assessment. The top five facilitating factors according to that criterion are in the first column of table 5 FIGURE 2 How Many Libraries Report Having a Culture of Assessment Based on Reported Presence or Absence of Facilitating Factors 0 200 400 600 Campuswide assessment initiative Involved in campuswide assessment User-focused culture Clear expectations for as sessment Shared understanding of assessment Adopted learning outcomes Has assessment plan Priority of library administration Leadership offers explicit support Staff/faculty are adequately supported Education offered/supported Has assessment skills in-house Access to systems/technologies Assessment data is available Leadership uses assessment data Librarians use assessment data Facilitating Factors and Assessment Culture Facilitating factor is present Faciliating factor is absent Factors Influencing a Culture of Assessment in Academic Libraries 161 along with the percentage that also reported having a culture of assessment. All five were focused on the commitment of library leadership to assessment, their support of assessment work, and how well the value of and expectations for assessment have been communicated. The survey contained two open-ended questions that asked librarians to state what factors they believed had most facilitated or hindered them in their efforts to build a culture of assessment. Each respondent saw and responded to only one of the two questions, depending on their answer to the question about whether or not they have a culture of assessment. The responses were exported to Excel, coded, and analyzed for frequency of themes. The top five facilitating factors from the open-ended question, along with their frequencies, are listed in the second column of table 5. Two of the three most common responses were related to factors largely outside the library’s control: whether there is an institutionwide emphasis on assessment and accreditation requirements. The two are intimately connected, as the impact of the importance of assessment in accreditation is not limited to the library. Many respon- dents highlighted that interconnectedness, making statements such as: “Middle States emphasis has impacted institution expectations for assessment and outcomes measure- ment.” Others wrote about how accreditation itself was a motivator for the library to build an assessment culture: “Applying for WASC accreditation has contributed significantly in developing the library’s culture of assessment. The institution’s leaders are now aware of developing library support and utilizing a culture of assessment that WASC recognizes.” Middle States and WASC are leaders in assessing student learning and also in emphasizing information literacy as a general learning outcome; however, such emphasis cannot be assumed to be permanent, so the influence of these factors on library culture of assessment could change over time. Many also mentioned the campuswide focus on assessment as a driving factor in moving the library toward an assessment culture: “Since our institution requires an- nual assessment plans/reports, our Library has developed the ‘routine’ of assessment.” In a few cases, it was the library’s culture of assessment that impacted the institution: “The library has been involved in assessment much longer than the rest of campus… our assessment process dates back to the 1950s. Emphasis from the Higher Learning Commission on assessment for re-certification (re-accreditation) has moved the rest of the University in that direction and made the library an acknowledged process leader.” The other three top facilitating factors mentioned are internal to the library. Library administrative interest in and support for assessment was commonly acknowledged as the driving force in the library’s assessment efforts. Many wrote about their ad- ministrator’s interest in assessment, his or her assessment expertise, involvement in assessment across campus, and commitment to supporting assessment in the library. One respondent described what the right leadership brings to a growing assessment program: Although we do not have all the answers yet and often feel we are stumbling along, the leadership in the library and on campus is one that fosters professional devel- opment and supports efforts to improve both assessment methods and learning. We continually strive to do a better job, even if our efforts are often imperfect. The proper leadership and the opportunities for professional development are key. In terms of an organizational culture that facilitates an assessment culture, many mentioned a user-focused culture, but some also used the term “culture of curiosity.” One respondent argued that assessment is natural in a user-focused culture, because “if you begin with thinking about how users will want to use your resources and ser- 162 College & Research Libraries March 2015 vices, then it becomes natural to assess how well those resources and services match user needs.” In terms of access to education and trainings, many mentioned support for attending assessment-focused conferences or an Immersion program offered by the Association of College and Research Libraries, in-house trainings, and general sup- port for professional development. As a lack of expertise or training was mentioned frequently by those at institutions without a culture of assessment as a key barrier, it is not surprising that the converse is a strong facilitator. For each of the yes/no questions about a culture of assessment factor, respondents were asked whether the presence of each factor helped to facilitate the creation of an assessment culture or whether its absence was a hindrance. Those that were perceived by the greatest number of respondents as a facilitating factor are listed in the third column of table 5 along with the percentage of respondents who reported that it was a facilitating factor for their library. These results are similar to those in column 1 in that they are focused on the commitment of library leadership to assessment and having clear expectations for assessment. Column 3 does not include anything about support for librarians engaging in assessment, but it does include librarians’ use of assessment data as an important facilitating factor. The same characteristics were analyzed to determine the most significant hindering factors shown in table 6. Column 1 shows those facilitating factors whose absence is TABLE 5 Most Significant Facilitating Factors Presence associated with a culture of assessment (yes/ no questions) Presence associated with a culture of assessment (open-ended questions) Perceived to facilitate creation of a culture of assessment Clear expectations for assessment in the library (86% have a culture of assessment) Institutionwide emphasis on assessment (67) Clear expectations for assessment in the library (92% report it as a facilitating factor) Library has an assessment plan (85% have a culture of assessment) Library administrative interest or support (56) Library leadership uses assessment data systematically in decision making (91% report it as a facilitating factor) Library leadership uses assessment data systematically in decision making (78% have a culture of assessment) Accreditation (33) Library has an assessment plan (89% report it as a facilitating factor) Shared understanding of the purpose of assessment in the library (77% have a culture of assessment) Organizational culture (22) Librarians use assessment data to improve practice (88% report it as a facilitating factor) Library staff/faculty are adequately supported in their assessment work (77% have a culture of assessment) Access to professional development/training/ education (15) Assessment is a priority of library administration (85% report it as a facilitating factor) Factors Influencing a Culture of Assessment in Academic Libraries 163 associated with the greatest number of libraries without a culture of assessment. Some of these—assessment not being a priority of administration, not having a user-focused culture, and leadership not using assessment data—were seen in their positive form as facilitating factors in table 6. The two that were not mentioned above—the library’s not being involved in the campuswide assessment initiative and the library’s not mak- ing assessment data available to individual parties—relate directly to the themes of institutional focus and use of data, which are threaded throughout the list of primary facilitating factors. Column 2 includes the five factors indicated by respondents as most responsible for keeping their library from achieving a culture of assessment. The two most frequently mentioned, a lack of staffing and time, were often stated in the same sentence, as a lack of sufficient staffing can also cause a lack of time. Some of the librarians who mentioned either of these two issues were solo librarians or the only professional li- brarian on staff. Others described how “budget cuts have decimated the library staff” or how inadequate their staffing is. Some of those who mentioned time as a factor also TABLE 6 Most Significant Hindering Factors Absence associated with lack of a culture of assessment (yes/no questions) Absence associated with lack of a culture of assessment (open-ended questions) Perceived to hinder creation of a culture of assessment Assessment is not priority of administration (81% don’t have a culture of assessment) Lack of sufficient staffing (54) Library does not have necessary skills in-house to develop and conduct meaningful assessments and analyze the results (86% report it as a hindering factor) Library culture is not user- focused (80% don’t have a culture of assessment) Lack of time (43) Library staff/faculty are not adequately supported in their assessment work (86% report it as a hindering factor) Library is not involved in the campuswide assessment initiative (72% don’t have a culture of assessment) Lack of library administrative leadership or support (42) There is no shared understanding of the purpose of assessment in the library (84% report it as a hindering factor) Assessment data is not available to interested parties (70% don’t have a culture of assessment) Lack of expertise/lack of access to training (39) Library does not have access to systems/technologies that support assessment work (75% report it as a hindering factor) Library leadership does not use assessment data systematically in decision making (68% don’t have a culture of assessment) Institution doesn’t prioritize assessment (29) Library culture is not user- focused (74% report it as a hindering factor) 164 College & Research Libraries March 2015 alluded to the fact that assessment simply was not a priority: “Our staff is small and very busy—there is a clear perception that assessment takes time and that we know from our own experience the things that assessment is telling us.” Others have made efforts to conduct assessments; but, without sufficient time, they cannot close the loop: “I have a pile of assessment data from the fall term that I’ve not even looked at because there hasn’t been the time.” Just as in the open-ended comments about facilitating factors, institutional and library leadership was mentioned quite frequently in the comments about the most significant hindrances. One respondent explained that “it is too easy to ignore assess- ment when it is not a required part of instructional structure.” In some cases, the institu- tion simply did not have a campuswide culture of assessment; but, in other cases, the library was left out of the campuswide assessment push. In some libraries, it was their own administrators who did not prioritize assessment: “The administration doesn’t seem to care and doesn’t ever ask for real assessment data beyond the numbers ALA or some accrediting body asks for.” A lack of commitment from those in leadership positions was seen as discouraging to those seeking to build an assessment culture. A lack of in-house expertise and training was mentioned frequently as a barrier for librarians who want to get started with assessment and who are already trying to do meaningful assessment work. One respondent mentioned that he or she conducted “a survey and gathered data about the information literacy skills and abilities of our students, but there is no one to help me analyze it or help me figure out what conclu- sions can be drawn from the data.” Others feel that their lack of training has left them not knowing where to start. Another respondent saw this paralysis as “the perception that one has to be a statistician to carry out the analysis of the data.” The third column includes the five factors perceived by the greatest percentage of respondents as hindering them in building a culture of assessment. All but one of these have been mentioned before, centering around themes of lack of support for assessment, lack of a shared understanding of assessment, and lack of a user-focused culture. The one that had not been included as a top hindering or facilitating factor before is the lack of systems or technologies that support assessment work. While one might assume that the absence of the things that facilitate the creation of a culture of assessment would hinder the creation of a culture of assessment, this was shown not always to be the case, as least as perceived by respondents. For example, while 91 percent of libraries who report that their leadership will use assessment data systematically in decision making and planning also report that it has facilitated movement toward a culture of assessment, only 64 percent of those whose leadership do not use data systematically report that it has hindered them in moving toward a culture of assessment. Similarly, 81 percent of libraries that are involved in their campus’s assessment initiative report that it has helped them move toward a culture of assessment, while only 55 percent of those whose libraries are not involved in their campus’s assessment initiative report that it has been a hindrance. Those factors with the closest agreement between its presence facilitating and its absence hindering in- clude whether the library has a shared understanding of the purpose of assessment (85% presence facilitates, 84% absence hinders), whether there is adequate support of library faculty and staff in their assessment work (85% presence facilitates, 86% absence hinders), and whether the library’s culture is user focused (75% presence facilitates, 74% absence hinders). These lists of facilitating and hindering factors do not constitute a recipe for building a culture of assessment, but they do provide useful insight into what is associated with and perceived as most helpful and detrimental in building a culture of assessment. For libraries working to build a culture of assessment, considering whether they have Factors Influencing a Culture of Assessment in Academic Libraries 165 the facilitating factors mentioned in table 5 and lack the factors mentioned in table 6 would constitute valuable first steps toward building a culture of assessment. Discussion The authors sought to better understand what factors facilitate the creation of a cul- ture of assessment; while there was no silver bullet found in the analysis, there are certainly some valuable conclusions that can be drawn. Every one of the facilitating factors asked about in the survey was significantly associated with having a culture of assessment, and no single one of them, or even a grouping of them, seems to provide a guarantee that a library will attain a culture of assessment. Through analysis, some factors came to the fore as being most frequently associated with having a culture of assessment or were more frequently mentioned as being a perceived facilitating factor. Library administration and whether or not they prioritize assessment is mentioned in four places across figures 7 and 8. The use or lack thereof of assessment data by library administrators, the presence or absence of a user-focused library culture, and the sup- port or lack of support for assessment work are mentioned three times each. Whether the library has the necessary skills in-house, whether there are clear expectations for assessment, whether there is a shared understanding of the purpose of assessment, and whether there is an assessment plan are each mentioned twice. What does seem clear from the results is that some sort of administrative priori- tization and support of assessment is necessary in building a culture of assessment, though it may not be sufficient. At libraries where assessment is a priority of library administration, 71 percent report having a culture of assessment versus 19 percent at libraries that do not report assessment being an administrative priority. Administrative prioritization of assessment is associated with significantly higher frequencies of all of the facilitating factors, but most striking is that the frequency of libraries with clear expectations for assessment is more than five times higher at libraries where assess- ment is an administrative priority (56%) than at those where it is not (10%). Also, only 22 percent of administrators who do not prioritize assessment use assessment data in decision making versus 69 percent who do prioritize assessment. Library administrators play a crucial role in setting the expectations and priorities of the library. If an administrator does not value assessment work, he or she will be less likely to use assessment results, provide explicit support for assessment, or hire people with assessment expertise. Many of the libraries that did not have a culture of assessment mentioned a lack of time or staff among the primary reasons why they could not achieve such a culture. If library administration values assessment, they are simply more likely to ensure that faculty and staff have the time to do meaningful assessment work. This is borne out in the results, as 67 percent of respondents whose administrators prioritize assessment offer explicit support to get library faculty and staff involved in assessment work versus 27 percent of those who do not. Of the 260 respondents who reported that their administrators prioritize assessment work, use assessment data in decision making, and adequately support their faculty and staff in their assessment work, 86 percent report having an assessment culture. This suggests that libraries where administrators are consistent in word and deed regarding their prioritization of assessment are more likely to have a culture of assessment. However, administrators are not the only individuals in the library who are needed to nurture an assessment culture. Library assessment initiatives often require active leadership and support from below to secure library faculty and staff buy-in. The survey results indicate that a greater percentage of libraries that have positions or committees dedicated to instruction and/or assessment have a culture of assessment versus those that do not. When looking at the existence of these positions or committees along with 166 College & Research Libraries March 2015 the existence of administrative prioritization of assessment, it becomes clearer that leadership from below can even help build an assessment culture in the absence of administrative support. In libraries where administrators prioritize assessment, more libraries with instruction and/or assessment coordinators reported having a culture of assessment (73%) than institutions without these positions (65%). In the absence of administrative prioritization of assessment, more libraries with instruction and/or as- sessment coordinators reported having a culture of assessment (24%) than institutions without these positions (13%). The support for having committees dedicated to instruc- tion and/or assessment is even stronger. In the absence of administrative prioritization of assessment, 29 percent of libraries with at least one of these committees report having a culture of assessment versus 14 percent of libraries that do not have instruction and/ or assessment committees. It is certainly ideal to have leadership from above and below in building a culture of assessment, but the results suggest that leadership from below can effectively move the library forward on assessment in some cases. The two facilitating factors that the lowest numbers of respondents reported hav- ing were clear expectations for assessment (46%) and an assessment plan (41%). Yet, based on the analysis, these are two of the most important factors in building an as- sessment culture. Of the 203 libraries that had both clear expectations for assessment and an assessment plan, 92 percent also reported having a culture of assessment. Of the 294 libraries that did not have either of those two facilitating factors, only 30 percent reported having a culture of assessment. Even when looking only at libraries whose administrators prioritize assessment, have instruction and/or assessment coordina- tors, and instruction and/or assessment committees, 44 percent of libraries that do not have clear expectations of assessment or a plan report having a culture of assessment versus 95 percent that do. This indicates that a library can have supportive leadership and the right positions and committees, but without a clearly articulated plan and ex- pectations, an assessment culture may not be achieved. Everyone in the organization needs to understand what is expected of them regarding assessment; simply stating its importance is rarely sufficient. Another cluster of factors that appears to be strongly associated with whether or not a library has a culture of assessment is the availability and use of assessment data. As Oakleaf and Hinchliffe found, only about two-thirds of instruction librarians who conduct assessments ever use the results, so this closing of the loop is a common problem in libraries. 12 Of the 283 libraries that report making assessment data avail- able to interested parties and whose administrators and library staff use assessment data, 85 percent report having a culture of assessment. Of the 70 that report a lack of those three facilitating factors, only 11 percent report having a culture of assessment. Assessment data are only valuable if they are used and thus, it is not surprising that assessment would not become part of a library’s culture if assessment data are not used to improve the library. The perceived support for librarians and staff in their assessment efforts was seen as important in building an assessment culture, but its presence or absence did not have as significant an impact on the rates of reporting a culture of assessment as the factors listed above. Of those respondents who reported that faculty and staff are ad- equately supported in their assessment efforts, that education and training regarding assessment are supported, and that explicit support to get faculty and staff engaged with assessment is offered, 79 percent also reported having a culture of assessment. Of those respondents who reported the absence of all three facilitating factors, only 35 percent had a culture of assessment. Support is important, but it does not appear to be as important to librarians in building a culture of assessment as has been indicated in the literature to be for faculty in higher education.13 Factors Influencing a Culture of Assessment in Academic Libraries 167 One factor that is strongly associated with having a culture of assessment is the least within the library’s control. The requirements and culture of the regional accrediting association has a major impact on the library’s assessment program or lack thereof. The numbers of libraries within SACS and Middle States that reported having a culture of assessment was significantly higher than those within other regional accrediting organizations. Given that it is unlikely that the library is causing the accrediting body to value assessment, it can be inferred that the emphasis the accrediting organization has placed on assessment has impacted library assessment practice. Those associations that had come more recently to an assessment emphasis, such as NEASC and WASC, have some of the lowest rates of libraries with an assessment culture. While the library can exert some influence over whether their institution has a campuswide assessment initiative, at most institutions this is not usually driven by the library. At libraries whose institutions do not have a campuswide assessment initiative, only 38 percent reported having an assessment culture versus 63 percent of those whose institutions do have a campuswide initiative. The lack of a campuswide assessment initiative is significantly associated with lower frequencies of each facili- tating factor, but its impact is most keenly felt on the library’s sense of direction with regard to assessment. Only 19 percent of those without a campuswide assessment initiative report having clear expectations for assessment and 17 percent report having an actual assessment plan, versus 51 percent and 45 percent respectively of libraries whose institutions have a campuswide assessment initiative. It appears that the most important thing a campuswide assessment initiative provides is a clear mandate for assessment at the library level. Faculty status has long been seen as important to library involvement in campuswide initiatives and decision making,14 so the authors expected to see a significant associa- tion between faculty status and a culture of assessment. In spite of the fact that several respondents in the open-ended questions listed faculty or tenure status as the primary factor moving them toward an assessment culture or keeping them from achieving an assessment culture, it was not found to be significantly associated with a culture of assessment. Even more surprising was that there also was not a significant association between faculty status and the library’s involvement in a campuswide assessment initiative (p=0.51 for tenure vs. non-tenure and p=0.77 for faculty status vs. staff). While faculty status may provide many benefits for librarians, this study suggests that greater involvement in campuswide assessment initiatives is not one of them. What cannot be concluded from these results is whether there is a causal relation- ship between the presence of any of the facilitating factors and a culture of assessment. There is a significant association between them; but whether one causes the other, or the directionality of any causation, cannot be determined in most cases. One cannot conclude from our results that an instructional assessment committee creates a culture of assessment, even though there is such a strong association between the two. Does having an instructional assessment committee make a library more likely to have a culture of assessment; or is it simply that institutions that prioritize assessment, and thus foster an assessment culture, happen to have an instructional assessment commit- tee? In spite of not being able to ascertain causality, just knowing that these facilitating factors are present much of the time when a library has a culture of assessment can help provide a sense of direction for a library seeking to build such a culture. Another limitation of this study is that what constitutes a “culture of assessment” is very much in the eye of the beholder. The survey asked respondents to determine whether their library has a culture of assessment and defined that as a state in which assessment is a regular part of the library’s practice. In spite of the definition, it be- came clear when reading the open-ended responses that librarians define a culture of 168 College & Research Libraries March 2015 assessment in very different ways. Some felt that, as long as their library was doing some assessment, they had a culture of assessment; others whose libraries were doing significantly more did not identify as having a culture of assessment. In some cases, libraries that were doing regular instructional assessment did not identify as having a culture of assessment because the entire library had not adopted such practices. Others in the same situation chose to identify their library as having a culture of assessment. This may have been alleviated to some extent by providing a scale that represents libraries moving toward a culture of assessment rather than offering only a binary notion of libraries either having or not having such a culture. Even then, however, subjective notions of what a culture of assessment is and where one’s library falls on the continuum will always be an issue in survey research. Similarly, singularity of perspective comes into play because only a single individual from each institution was able to respond to the survey. In many cases this was an administrator, sometimes it was a middle manager, and sometimes it was a front-line librarian in public services. It would not be surprising if the differences in positions created differences in perspectives and that different people in the same organization might answer differently. The authors asked library directors to identify an individual with a leadership role in instructional assessment and depended on their judgment to select the most appropriate person. It would be interesting in the future to look at assessment through the eyes of several members of each institution at different levels in the organization for a comparative perspective analysis. Ethnographies of organiza- tions that have, do not have, and are developing a culture of assessment would also provide a much richer picture of what it takes to build an assessment culture. Finally, this survey also represents a moment in time rather than a longitudinal perspective. Respondents with an assessment culture were not asked whether it came to be before or after facilitating factors were present. Likewise, no questions were asked of those without a culture of assessment about whether the library had one previously. Conclusion This study provides a systematic analysis of the state of the culture of assessment in academic libraries at four-year institutions in the United States. A culture of assessment was defined as one where assessment is a regular part of institutional practice. Because every one of the facilitating factors examined in the survey was significantly associated with having a culture of assessment and no single one of them, or even a grouping of them, seems to provide a guarantee that a library will attain a culture of assessment, the results do not offer prescriptive direction for developing a culture of assessment. The results do, however, bring clarity to understanding how different factors are related to one another and also offer the insight that administrators or front-line leadership can be influential in culture development. The librarian wishing to develop a culture of assessment in an academic library can pursue such culture development through campus engagement, administrative support, and ground-level activities. There is no singular path and so the librarian can choose those approaches that best fit within the circumstances of the library and the larger institution. Notes 1. Amos Lakos and Shelley E. Phipps, “Creating a Culture of Assessment: A Catalyst for Organizational Change,” portal: Libraries and the Academy 4, no. 3 (2004): 345–61. 2. Daniel J. Ennis, “Contra Assessment Culture,” Assessment Update 22, no. 2 (2010): 1–15. 3. List of facilitating factors was generated from a close reading of the following sources: Trudy W. Banta, “Moving Assessment Forward: Enabling Conditions and Stumbling Blocks,” Factors Influencing a Culture of Assessment in Academic Libraries 169 New Directions for Higher Education, no. 100 (1997): 79–91; Donald H. Bennion and Michael Har- ris, “Creating an Assessment Culture at Eastern Michigan University: A Decade of Progress,” Assessment Update 17, no. 2 (2005): 7–10; Brian Cox and Margie Jantti, “Discovering the Impact of Library Use and Student Performance,” Educause Review Online, 2012, available online at www. educause.edu/ero/article/discovering-impact-library-use-and-student-performance [accessed 2 June 2013]; Don Haviland, “Why Are Faculty Wary of Assessment?” Academic Leadership Journal 7, no. 3 (2009) available online at https://web.archive.org/web/20100423145724/http://www. academicleadership.org/ideas_with_merit/601.shtml [accessed 24 May 2013]; Jeffrey S. Hill, “Developing a Culture of Assessment: Insights from Theory and Experience,” Journal of Politi- cal Science Education 1, no. 1 (Feb. 9, 2005): 29–37; Steve Hiller, Martha Kyrillidou, and Jim Self, “When the Evidence Is Not Enough: Organizational Factors That Influence Effective and Success- ful Library Assessment,” Performance Measurement and Metrics 9, no. 3 (2008): 223–30; Nicholas Hunt-Bull, “Doing Assessment as if Teaching Matters: Changing the Assessment Culture in an Academic Division,” Assessment Update 19, no. 6 (2007): 1–15; Philip I. Kramer, “The Art of Mak- ing Assessment Anti-Venom: Injecting Assessment in Small Doses to Create a Faculty Culture of Assessment,” Assessment Update 21, no. 6 (2009): 8–11; Lakos and Phipps, “Creating a Culture of Assessment,” 345–61; Jack Magruder, Michael A. McManis, and Candace C. Young, “The Right Idea at the Right Time: Development of a Transformational Assessment Culture,” New Directions for Higher Education, no. 100 (1997): 17–29; Heat Marrs, “Perceptions of College Faculty Regarding Outcomes Assessment,” International Electronic Journal for Leadership in Learning 13, no. 2 (2009): 1–9; Charles J. McClain and Darrell W. Krueger, “Using Outcomes Assessment: A Case Study in Institutional Change,” in New Directions for Institutional Research, ed. Peter T. Ewell (San Francisco, Calif.: Jossey-Bass, 1985), 33–46; Michael F. Middaugh, “Closing the Loop: Linking Planning and Assessment,” Planning for Higher Education 37, no. 3 (2009): 5–14; Abdou Ndoye and Michele A. Parker, “Creating and Sustaining a Culture of Assessment,” Planning for Higher Education 38, no. 2 (2010): 28–39; Megan Oakleaf, Michelle S. Millet, and Leah Kraus, “All Together Now: Getting Faculty, Administrators, and Staff Engaged in Information Literacy Assessment,” portal: Libraries and the Academy 11, no. 3 (2011): 831–52; Megan Oakleaf and Lisa Janicke Hinchliffe, “Assessment Cycle or Circular File: Do Academic Librarians Use Information Literacy Assessment Data?” in Proceedings of the 2008 Library Assessment Conference Building Effective Sustainable Practical Assess- ment, ed. Steve Hiller et al. (Washington, D.C.: Association of Research Libraries, 2008), 159–64; Peggy Piascik and Eleanora Bird, “Creating and Sustaining a Culture of Assessment.,” American Journal of Pharmaceutical Education 72, no. 5 (2008): 97; Stephanie A Schlitz et al., “Developing a Culture of Assessment through a Faculty Learning Community: A Case Study,” International Journal of Teaching and Learning in Higher Education 21, no. 1 (2009): 133–47. 4. Oakleaf and Hinchliffe, “Assessment Cycle or Circular File,” 159–64. 5. Denise Troll Covey, “Academic Library Assessment: New Duties and Dilemmas,” New Library World 103, no. 4/5 (2002): 156–64. 6. Stephanie Wright and Lynda S. White, SPEC Kit 303, Library Assessment (Washington, D.C.: Association of Research Libraries, 2007). 7. Hiller, Kyrillidou, and Self, “When the Evidence Is Not Enough,” 223–30. 8. Ndoye and Parker, “Creating and Sustaining a Culture of Assessment,” 28–39. 9. Carnegie Classification of Institutions of Higher Education. Accessed 1 November 2012. http:// classifications.carnegiefoundation.org/. 10. Danielle Bodrero Hoggan, “Faculty Status for Librarians in Higher Education,” portal: Libraries and the Academy 3, no. 3 (2003): 431–45; Corey M. Johnson and Elizabeth Blakesley Lindsay, “Why We Do What We Do: Exploring Priorities within Public Services Librarianship,” portal: Libraries and the Academy 6 (2006): 347–69; Marrs, “Perceptions of College Faculty Regarding Outcomes Assessment,” 1–9; Steve McKinzie, “590: Local Notes—Tenure for Academic Librar- ians: Why It Has to Go,” Against the Grain 22, no. 4 (2010): 60; John Tagg, “Why Does the Faculty Resist Change?” Change: The Magazine of Higher Learning 44, no. 1 (2012): 6–15. 11. “Analyze a 2x2 contingency table.” GraphPad. Accessed 3 March 2013. http://graphpad. com/quickcalcs/contingency1/. 12. Oakleaf and Hinchliffe, “Assessment Cycle or Circular File,” 159–64. 13. Banta, “Moving Assessment Forward,” 79–91; Hunt-Bull, “Doing Assessment as if Teach- ing Matters,” 1–15; Lakos and Phipps, “Creating a Culture of Assessment,” 345–61; Ndoye and Parker, “Creating and Sustaining a Culture of Assessment,” 28–39. 14. Hoggan, “Faculty Status for Librarians in Higher Education,” 431–45.