User Experience Methods and Maturity in Academic Libraries ARTICLES User Experience Methods and Maturity in Academic Libraries Scott W. H. Young, Zoe Chao, and Adam Chandler INFORMATION TECHNOLOGY AND LIBRARIES | MARCH 2020 https://doi.org/10.6017/ital.v39i1.11787 Scott W. H. Young (swyoung@montana.edu) is UX and Assessment Librarian, Montana State University. Zoe Chao (chaoszuyu@gmail.com) is UX Designer, Truist Financial. Adam Chandler (alc28@cornell.edu) is Director of Automation, Assessment, and Post-Cataloging Services, Cornell University. ABSTRACT This article presents a mixed-methods study of the methods and maturity of user experience (UX) practice in academic libraries. The authors apply qualitative content analysis and quantitative statistical analysis to a research dataset derived from a survey of UX practitioners. Results reveal the type and extent of UX methods currently in use by practitioners in academic libraries. Themes extracted from the survey responses also reveal a set of factors that influence the development of UX maturity. Analysis and discussion focus on organizational characteristics that influence UX methods and maturity. The authors conclude by offering a library-focused maturity scale with recommended practices for advancing UX maturity in academic libraries. INTRODUCTION User experience (UX) is a design practice for creating tools and services from a user-centered perspective. Academic libraries have been practicing UX for some time, with UX methods having been incorporated across the profession. However, there has been a lack of empirical data showing the extent of UX methods in use or state of UX maturity in libraries. To help illuminate these areas, we distributed a survey to UX practitioners working in academic libraries that inquired into methods and maturity. We followed a mixed-methods approach involving both qualitative content analysis and quantitative statistical analysis to analyze the dataset. Our results reveal the most- and least-common UX methods currently in use in academic libraries. Results also demonstrate specific organizational characteristics that help and hinder UX maturity. We conclude by offering a set of strategies for reaching higher levels of UX maturity. BACKGROUND AND MOTIVATION: UX IN ACADEMIC LIBRARIES UX has been represented in the literature of library and information science for at least two decades, when “the human interaction involved in service use” was recognized as a factor affecting the value and impact of libraries.1 The practice of UX has expanded and evolved and is now a growing specialty in the librarianship profession.2 UX in libraries is motivated by a call to actively pay close attention to users’ unique and distinctive requirements, which allows libraries to more effectively design services for our communities.3 As a practice, UX is now beginning to be represented in graduate curricula, public services and research support, access services, space design, and web design.4 With its attunement to a set of practices and principles, UX can be viewed as a research and design methodology similar and related to other methodologies that focus on users, services, problem solving, participation, collaboration, and qualitative data analysis .5 Notably, UX is related to human-centered design, service design, and participatory design.6 mailto:swyoung@montana.edu mailto:chaoszuyu@gmail.com mailto:alc28@cornell.edu INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 2 Specific methods of UX practice are today wide-ranging. They include surveys, focus groups, interviews, contextual inquiry, journey mapping, usability testing, personas, card sorting, A/B testing, ecology maps, observations, ethnography, prototyping, and blueprinting.7 Some UX methods are incorporated into agile development processes.8 Though tools and techniques are available to library UX practitioners in abundance, the rate of adoption of these tools is less understood. In a notable contribution to this question, Pshock showed through a nation-wide survey that the most familiar UX methods among library practitioners included usability testing, surveys, and focus groups.9 The question of methods is related to the question of maturity—how advanced is library UX practice? In addition to the rate of adoption of methods and tools, several different UX maturity models have been advanced in recent years. Priester derives maturity from four factors: culture of innovation, infrastructure agility, acceptance of failure, and library user focus.10 In discussing UX capacity in libraries, MacDonald proposes a six-stage maturity model: unrecognized, recognized, considered, implemented, integrated, and institutionalized.11 Sharon defines maturity as a combination of staff resources and organizational buy-in.12 Similarly, Sheldon-Hess proposes a five-level scale of UX maturity, based primarily on the degree of implementation of UX practice and user-centered thinking in an organization.13 And even earlier, Nielsen proposed an eight-level scale of UX maturity, starting with a “hostility toward usability” and concluding with a “user- driven” organization.14 After reviewing a number of different maturity models, Anderson reports that the most common hierarchies include the following steps: (1) Absence/Unawareness of UX Research, (2) UX Research Awareness—Ad Hoc Research, (3) Adoption of UX research into projects, (4) Maturing of UX research into an organizational focus, (5) Integrated UX research across strategy, and (6) Complete UX research culture.15 The field of library UX shows a clear and compelling interest in UX maturity, and we can benefit from further empirical evidence that can help illuminate the current state and future progress toward UX maturity, including the rate of adoption of methods, resource allocation toward UX, and organizational buy-in. The research presented in this paper is motivated by the need to provide current and comprehensive data to answer questions related to UX maturity in academic libraries. METHODS Research Questions The research questions for this study are the following: • RQ1: How mature is UX practice within academic libraries? • RQ2: What factors influence UX maturity? To answer these questions, we distributed a survey to UX practitioners working in academic libraries. Survey responses were analyzed qualitatively using content analysis and quantitatively using statistical analysis. Survey Participants The team members sent out the survey on May 23, 2018, to library profession electronic discussion lists.16 Of the 87 received responses, 74 included an institution name. We identified size and setting classification using The Carnegie Classification of Institutions of Higher Education (see INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 3 table 1) for the institutions.17 Eight of them cannot be mapped to the Carnegie classification due to being outside the United States (n = 6) or of different scopes (one research lab and one information school). Six schools have more than one response, which are treated separately to represent the diversity of opinion and experience within an organization. Classification Response Count Percentage Four-year, large 4918 56 Four-year, medium 1019 11 University outside US 6 7 Four-year, small 5 6 Non-university 2 2 Four-year, very small 1 1 Two-year, very large 1 1 Unspecified 13 15 Table 1. Institutional profiles of survey respondents, with response counts. Materials and Procedure Our online survey was organized into two main parts. After an initial informed consent section, the survey investigated (1) Demographics and UX Methods and (2) UX Maturity. Demographics and UX Methods In the first main part of the survey, participants were asked to select among 20 different UX methods that “you personally use at least every year or two at your institution.” The list of methods is derived from the UX Research Cheat Sheet by Nielsen Norman Group.20 Participants were asked to complete an optional free-text response question: “Would you like to add a comment clarifying the way you completed [this question]?” UX Maturity In the second main part of the survey, participants were asked to identify the UX maturity stage that “properly describes the current UX status” in their organization. The stages were adapted from the eight-stage scale of UX maturity proposed by Nielsen Norman Group: • Stage 1: Hostility Toward Usability • Stage 2: Developer-Centered UX • Stage 3: Skunkworks UX • Stage 4: Dedicated UX Budget • Stage 5: Managed Usability • Stage 6: Systematic User-Centered Design Process • Stage 7: Integrated User-Centered Design INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 4 • Stage 8: User-Driven Organization We concluded the survey by asking participants to optionally “explain why you selected that stage” with a free-text response. Research Data Analysis Content Analysis We followed the methodology of content analysis.21 Each qualitative survey response functioned as a meaning unit, with meaning units sorted into themes and subthemes. Each article author coded units independently; themes were resolved through discussion among the author group. The process of coding via content analysis allowed us to identify overarching trends in UX practice and maturity. Results are further discussed below. Statistical Analysis Data preparation and statistical analysis were conducted using R version 3.4.1 (see table 2 for full R package). Base R was used for our statistical analysis. Other R packages utilized in the project are listed in the table below. R package name Version ggplot2 3.0.0 Tibble 2.1.1 dplyr 0.7.5 tidyr 0.8.1 stringr 1.4.0 readr 1.1.1 readxl 1.3.1 Table 2. R packages used in the analysis Data Preparation The following steps were taken in the data analysis: 1. Content analysis into themes (see above) 2. Normalize institution names. We received more than one response from a few institutions. For these, the responses were treated as separate responses that happened to have the same demographics. 3. For responses that included institution names, we added a total student population variable to the response using values derived from Wikipedia and the Carnegie Classification of Institutions of Higher Education. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 5 4. For variables we derived during the content analysis we coded them as 0 or 1 dummy variables, that is, 0 = not present, and 1 = present. Coding them in this way allows us to bring them into a multiple linear regression model. 5. Using an R script, we tested each response for the presence of the content analysis, 0 or 1. 6. Plots were created using the R ggplot2 library. 7. Linear regression models were conducted using the base R lm function. Research Dataset Dataset, survey instrument, and R code are available through Dryad at https://doi.org/10.5061/dryad.jwstqjq5d.22 Survey Respondents Eighty-seven participants responded to one or more components of the survey. See table 3 for a breakdown of survey responses. Survey Question Responses UX Methods multiple choice: “Please check the following UX methods that you personally use at least every year or two at your institution.” 81 UX Methods free-text response: “Would you like to add a comment clarifying the way you completed [the question related to UX methods]?” 20 UX Maturity Stage multiple choice: “Which of the following [maturity stages] do you think properly describes the current UX status in your organization?” 79 UX Maturity Stage free-text response: “Please explain why you selected that stage.” 54 Table 3. Survey Responses. RESULTS Our research results demonstrate that certain characteristics of a library organization are related to UX maturity. These characteristics include the type and extent of UX methods that are currently in use, as well as organizational factors such as leadership support, staffing, and collaboration. We further explicate below according to our two research questions. RQ1: How mature is user experience practice within academic libraries? Our survey also asked participants to identify which stage of the Nielsen Norman Group Maturity scale “properly describes the current UX status” in their organization. Our findings indicate that most libraries are in a low-to-middle range of maturity, with more than 75% of respondents placing their organization at either Stage 3, Stage 4, or Stage 5 (figure 1). https://doi.org/10.5061/dryad.jwstqjq5d INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 6 Figure 1. Histogram of responses by stage, showing that the majority of respondents placed their organization at either Stage 3, Stage 4, or Stage 5. RQ2: What Factors Influence UX Maturity? Overview of Statistical Analysis Results We use linear regression for two different applications in this study (see appendix A for glossary of terms related to statistical analysis). The process of creating a statistical model allows us to see, with varying degrees of confidence, the impact of different variables on UX maturity stage. The results of the linear regression help us to tease out the variables with the most predictive value. Using certain methods does not cause the library to be at a higher stage; rather, libraries that use certain methods tend to be at a higher stage, statistically. That is what is meant by “predictive” in this context. Linear regression provides a ground truth in what we think we are seeing in survey responses: A useful general principle in science is that when you don’t know the true form of a relationship, start with something simple. A linear equation is perhaps the simplest way to describe a relationship between two or more variables and still get reasonably accurate predictions.23 The other reason we are using the linear regression output is to inform a possible future version of a UX maturity survey instrument, one more finely tuned to libraries than the Nielsen instrument alone that we used in this iteration.24 We feel that our use of multiple linear regression is appropriate and helpful given the exploratory nature of our study. The complete output is available at https://doi.org/10.5061/dryad.jwstqjq5d. Size of Institution We used the institution’s student population, the number of full-time enrolled students, as our proxy for the size of the library. Our assumption being, larger enrollment generally means larger https://doi.org/10.5061/dryad.jwstqjq5d INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 7 number of library staff. There are different ways UX maturity level could be compared with student enrollment; because the range in our sample is very wide, from 1,098 to 200,000 students across the sample of institutions, we attempted to control for the vast differences in size between the smallest and largest by sorting, from smallest to largest, 1 to 69 (the total number of cases in our dataset with both a stage and population defined), then assigning rank to the institution as an additional demographic variable. We then created a simple linear regression model comparing maturity stage as a function of ranked size. The null hypothesis is that there is no relationship between ranked size of institution and stage. Stage is the response variable and ranked size of the institution is the explanatory variable. The adjusted R-squared relationship is 0.027. This means that only about 3% of the variance is accounted for by the ranked size of the institution. The probability, or “p-value,” of getting our observed result if the null hypothesis is true for this relationship is 0.095 (almost 10%). This exceeds the standard .05 confidence level commonly used in statistical analysis. Therefore the size of the institution is not a reliable predictor of UX maturity level in our sample, a counterintuitive finding. The full statistical summary is available in the appendix. Methods Currently in Use by Academic Libraries Our next RQ2 finding relates to the type and extent of UX methods that are currently in use in academic libraries. Our survey asked participants to select which UX methods “you personally use at least every year or two at your institution.” User surveys, usability testing, and user interviews stand out as the most commonly used. Figure 2 shows response counts for all of the methods in the survey. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 8 Figure 2. Number of respondents that selected each method in the survey, showing the type and extent of UX methods currently in use in academic libraries. We then examined the number of methods in use per institution compared to the reported maturity stage (figure 3). The number of methods used per institution illustrates a trend: more methods used at an institution generally means the institution is at a higher stag e of maturity. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 9 Figure 3. The number of methods used per institution, illustrating that more methods currently in use at an institution generally indicates a higher level of maturity. Another way of representing the same two variables (reported number of methods and maturity stage) is with a scatterplot and statistical test (figure 4). In this simple linear regression model we have two variables: the response variable is stage and the explanatory variable is number of UX methods used in the past two years. In plotting these two variables on a chart, we can draw a line that minimizes the distance between the line and all of the points on the plot. Like the chart above, the linear relationship between total methods and stage is clearly visible. The total number of methods practiced accounts for about 18% of the variance when predicting the correct maturity stage. (Recall from our discussion about ranked size of institution that rank accounts for less than 3% of the variation, and is not even statistically significant.) In this case, the p-value is far below the 0.05 threshold, meaning the likelihood that we are seeing a relationship by random chance is very low. Therefore total number of methods is predictive of stage. Generally, the more methods respondents chose, the higher the maturity stage. We can see from this data that the number of methods used is more predictive of maturity stage than institution size. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 10 Figure 4. Maturity stage compared against total number of methods used, showing the positive relationship between number of UX methods used and UX maturity stage. For a more granular view, figure 5 shows the relation of specific UX methods used in different UX research phases (as categorized in the survey question, with methods organized by discovery, exploration, listening, and testing) to reported maturity stage. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 11 Figure 5. Showing the relation of specific UX methods to reported maturity stage. Factors that Influence UX Methods: Recency, Formality, Regularity We then applied content analysis to the free-text questions of our survey. Following the question that asked participants to select among 20 different UX methods that “you personally use at least every year or two at your institution,” the free-text question asked, “Would you like to add a comment clarifying the way you completed [this question]?” Each of the 20 free-text responses to this question was counted and categorized as a “meaning unit.” Themes were extracted from the free-text survey responses. We identified 3 themes across 20 meaning units: formality, regularity, and recency (see table 4). Question - Would you like to add a comment clarifying the way you completed [this question related to UX methods]? Thematic Analysis Theme Definition Number of Meaning Units* Example Meaning Unit INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 12 Recency How new or developed a library’s UX practice 7 “I am fairly new here and we are still developing a process that is well-rounded.” Formality How formal or structured the UX practice is 9 “We are aware of many of the techniques mentioned, but we don’t have a formal process for implementing them.” Regularity How often or frequently UX is practiced 4 “Right now we are doing a workflow analysis of interlibrary loan, but once completed probably wouldn’t do that for another three to four years.” *Each free-text response was counted and categorized as a single meaning unit. Table 4. Qualitative Questions and Thematic Analysis for UX Methods responses (n = 20). Factors that Influence UX maturity: Leadership Support, Collaboration, UX Lead, UX Group, Growth, and Resources We also conducted a content analysis on the free-text responses to the survey question related to the UX maturity scale that asked participants to “explain why you selected that stage.” Each of the 54 free-text responses to this question was counted and categorized as a “meaning unit.” Themes were extracted from the free-text survey responses. We identified 7 themes across 54 meaning units: leadership support, collaboration, UX lead, UX group, growth, and resources, and strategic alignment (see table 5). Question - Please explain why you selected [the current UX status in your organization]? Thematic Analysis Theme Definition Number of Meaning Units* Example Meaning Unit Leadership support The degree to which UX work is seen, understood by, and supported by library leadership. 32 “Just last year, the UX team moved into administration so that we can tie our work to strategic planning for the organization.” INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 13 UX group The presence of a committee or working group that conducts or otherwise supports UX work. 31 “I also chair a Web Working Group which focuses on improving our website from a usability standpoint.” Collaboration The degree to which UX work is collaboratively shared by individuals and departments throughout the library 30 “I don't know if UX has become a necessarily planned activity across the whole organization. I am team of one, and though I’ve tried, I haven’t been able to add anyone else to form an official UX team as well.” UX lead Personnel assigned to UX work, especially a dedicated UX lead 30 “I have recently been hired to partially work with UX and another person has been appointed UX coordinator.” Growth The degree to which expansion occurs around staffing, resources, and organizational understanding of UX work. 13 “We . . . will soon be posting a position for a UX librarian.” Resources The amount of time and budgetary resources dedicated to UX. 10 “Budget is our biggest constraint when it comes to UX testing.” Strategic alignment The inclusion of UX or user- centeredness in strategic planning 2 “We do employ user research to determine where to target priorities and strategy. However, I do not think we have a robust process for iterative testing or participatory design yet.” * Each free-text response was counted and categorized as a single meaning unit. Table 5. Qualitative questions and thematic analysis for UX maturity responses (n = 54). INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 14 This data can be visualized to show the relationships between UX maturity stage and the coded thematic responses (figure 6). Figure 6. Coded responses versus selected stage (0 indicates no comment related to that theme), showing that a lack of leadership support is often cited as a reason for not advancing past Stage 3; the presence of dedicated staff in the form of a UX lead or a UX group is often cited as a reason for reaching Stage 5. Full UX Maturity Model: UX Maturity as a Function of UX Methods In building a full model for the purposes of quantitative data analysis, we are attempting to predict the maturity stage based on the many different variables that appear in our dataset. This statistical exercise is a heuristic tool that can help us understand the survey responses and to draw results from the dataset that reveal key characteristics of UX maturity in libraries. We approached building a full model using a modified backward stepwise approach. With this approach, we begin with the full range of variables and work backward step by step to focus only on those variables that combine to form a model that makes the best predictions about the response variable—the UX maturity stage—for each case. Through this process, those variables that are less predictive are removed from the model one by one until we can settle on a model that explains the most variance.25 The modified backward stepwise “step” function used to create our model required 18 iterations before settling on the best version. Using adjusted R-squared as our metric, our full model accounts for 62% of the variance for this dataset. Adjusted R-square is an appropriate measure because it allows us to include many variables but also includes a penalty for including too many variables (as a penalty, the adjusted R-square value will decrease). With this model, we can make reasonable estimates of the maturity stage that a survey respondent selected by knowing which methods they use combined with the coded explanation the respondent provided via the free-text survey questions. The coded responses (see table 4 and table 5) provided measurable insights into INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 15 the organizational context of our respondents’ institutions, and this allows us to analyze and predict their respective maturity levels. With this additional information, we have a model that represents the multiple dimensions available in the dataset (see appendix B for additional data analysis). In table 6, we show the relationship between specific UX methods and the UX maturity stages. We see here that journey mapping, for example, is a highly influential factor for UX maturity. Variable Estimated Influence on Maturity Stage P-value (Significance) Journey maps 1.7 0.001*** Design review 1.3 0.010* User interviews 0.9 0.047* Usability testing 0.8 0.158 Benchmark testing 0.7 0.067 Usability ug review 0.3 0.498 User stories 0.2 0.440 Requirements and constraints 0.2 0.514 User surveys -0.3 0.492 Diary camera studies -0.6 0.325 FAQ review -0.8 0.062 Prototype testing -0.8 0.076 Field studies -1.3 0.003** *p < .05 (statistically significant result), **p < .01 , ***p <.001 (a highly statistically significant result) Table 6. Relationship between UX method variables and predicted maturity stage. In table 7, we show the relationship between the coded responses from the free-text survey questions (presented in tables 4 and 5), and the UX maturity stages. We see through this analysis that variables such as “resources” are important for advancing maturity. Similarly, we see that a lack of “leadership support” has a strong negative effect on maturity. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 16 Variable Estimated Influence on Maturity Stage P-value (Significance) Resources: yes 2.9 0.014* Collaboration: yes 0.8 0.147 Growth: yes 0.2 0.561 Resources: no -0.2 0.615 UX lead: no -0.5 0.216 Leadership support: yes -0.7 0.177 UX lead: yes -0.9 0.038* UX group: no -0.9 0.022* Leadership support: no -1.0 0.009** Strategic alignment: no -2.8 0.012* *p < .05 (statistically significant result), **p < .01 , ***p <.001 (a highly statistically significant result) Table 7. Relationship between organizational variables and predicted maturity stage, in descending order of influence on maturity stage. A Statistical Example Case: Estimating UX Maturity To help the reader understand the statistical summary provided by our model, we take a close look at one case drawn from one actual survey participant. In this example case, the respondent’s institution is a four-year, large university. The intercept for this multiple regression model happens to be 4.1119. Intercept in a multiple regression model represents the mean response (stage) when all the predictors are all zero.26 It is a baseline. Our example institution has practiced the following methods, with their respective influence on UX maturity included in parentheses: • User interviews (+ 0.9521) • Usability testing (+ 0.7984) • Benchmark testing (+ 0.7124) • Usability bug review (+ 0.2692) • Field studies (- 1.3346) • Prototype testing (- 0.8454) • User surveys (- 0.3204) Additionally, this institution has the following organizational characteristics, with their respective influence on UX maturity included in parentheses: • Leadership support: yes (- 0.6842) INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 17 • Resources: no (- 0.2192) By adding these numbers together with the starting point (4.111), we can calculate that the predicted stage for this institution is 3.44. Actual stage selected by survey respondent: 3 Residual -0.44 The model predicts a stage of 3.440 for this large, four-year university library. The model’s predicted value for this library is 0.440 greater than the stage selected by this survey respondent. The leftover part, or error, is the residual. The attentive reader might at this point ask why the variable called Leadership support: yes has a negative estimate, - 0.6842. That certainly is counterintuitive. Other evidence and our own interpretation lead us to expect that Leadership support: yes should have a positive effect on the maturity. In this particular case, the negative estimate has a high p-value (0.177) and is thus unreliable and not significant to the model. Part of the unreliability stems from the relatively small number of institutions (n = 9) that were coded as Leadership support: yes. In contrast, responses that were coded as Leadership support: no (n = 23) produced an even lower negative estimate of -0.9911, with a very reliable and highly significant p-value of 0.009. This shows us that when leadership support is lacking, maturity reliably suffers. We discuss this and other organizational characteristics in more detail below. DISCUSSION In interpreting our results, we have identified four key areas that we wish to emphasize: the significance of leadership support, the importance of organization-wide collaboration, the role of applied UX methods, and the emerging theory and practice of UX and design in libraries. Leadership Support and Strategic Alignment A major theme evident in the results relates to leadership support and strategic alignment. As expressed by the survey respondents, leadership support is viewed as the degree to which UX work is seen, understood by, and supported by library leadership and organizational strategic planning. In particular, a lack of support and visioning from leadership exerts negative pressure on UX maturity. On the other hand, when UX is coordinated with leadership vision and situated into strategic planning, UX maturity was rated more highly. From a leadership perspective, UX maturity relies on an allocated budget and designated staff to move beyond an ad hoc approach and reach higher levels on the maturity scale. One might expect that the larger an institution, the more advanced the UX maturity stage. However, based on our data analysis, size of institution is not a significant factor in UX maturity. Therefore the resources provided to library UX activities may not be about how large institutions are, but rather if leadership acknowledge the importance of UX and provide official, particularly financial, support. Organizational Collaboration Another major theme was collaboration—the degree to which UX research is collaboratively shared by individuals and departments throughout the library. Higher levels of UX maturity are driven by a widespread understanding of UX within an organization, with user research data INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 18 integrated into decision-making across multiple touchpoints. Conversely, a lack of collaboration was a factor that hindered maturity. Many respondents shared similar experiences, telling us that other staff or departments within the organization are not ready to embrace the potential of UX data, methods, and insights. We recognize that cultivating UX is an organic process that can result in uneven growth of UX within an organization. Some units may be ready to move further and faster while others may hesitate to contribute or collaborate. Not every department will immediately see the relevance or value of UX work for their area. Accounting or human resources, for example, might consider UX as beyond the scope of their practice. Thinking inclusively and holistically from the perspective of user-centered service design, however, opens up new connections between UX and the work of all departments across the organization. UX can help center those users—even internal users—who interact with service points such as accounting or human resources in ways that can improve the service experience for all involved. Applied UX Methods Across the 20 methods that we included in the survey, our results indicate that the application of different UX methods varies widely in type and extent. Many methods are in use to varying degrees. As the methods relate to maturity, we find that a greater number of methods in use during the previous two years was indicative of a higher maturity rating. In short, more methods lead to more maturity. The five most common methods included usability testing, user surveys, user interviews, accessibility evaluation, and field studies. These methods are similar in their ease of implementation and their wide representation in the library literature, and due to their commonness, they are not strongly indicative of UX maturity, high or low. The five least common methods included journey maps, benchmark testing, design review, FAQ review, and diary/camera studies. In this grouping we see a set of UX methods that are not as well known or widely discussed, but which can paint a more complete picture of the user experience. Journey mapping in particular was strongly and positively influential on UX maturity in our statistical model. This result does not necessarily indicate that a library can boost UX maturity simply by creating a journey map. Rather, we interpret this to indicate that the method itself is reflective of a coordinated UX effort in the institution. Journey mapping aims to obtain a high-level overview of a user’s interactions with every touch point to accomplish a task. As such, the successful implementation of a journey map relies on cross-functional and cross-departmental input and interpretation. This result calls for greater collaboration toward greater UX maturity. UX as an Emerging Practice within Libraries Many respondents focused on the newness or the maturity of their library’s UX practice, and most responses connected a low methods usage to the newness of the practice. In these responses, we see that UX in libraries is still a new field, and the practice is emerging with variations across institutions in terms of methods and maturity. We note that institutional size was not a factor that influenced maturity—some smaller institutions reported mature UX practice while some larger institutions reported lower UX maturity. In this result, we see that the amount of possible resources matters less than the intentional application of those resources in support of UX work. As institutions begin to see the value of UX and dedicate increasingly more resources relative to their budgets, UX maturity increases. Our survey respondents shared a variety of experiences along this journey toward maturity. Many told us “I’m new here” and that their library doesn’t fully understand UX and isn’t yet ready to include UX research in decision-making or strategic planning, or that the institution doesn’t have a plan yet for how to integrate the UX librarian into library operations. Still others reported that librarians in other units or library administrators are INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 19 not required or encouraged to consult with the UX librarian or integrate UX research. In this way, many libraries continue a more traditional model of decision-making that does not regularly apply intentional methods to account for the voices of users. On the upper end of the maturity scale, on the other hand, we see a wide adoption of UX as a legitimate area of work varies across units and within leadership groups. In this way, some libraries have demonstrated more responsiveness to UX and have more successfully integrated UX practices into strategic and operational workflows. Through the survey responses, we see a three- step progression that marks the emergence of UX as a trusted and legitimate methodology for understanding user experiences and designing library services: recency, formality, and regularity (table 4). In the earlier stages of maturity, survey respondents emphasize the newness or recency of a group or person assigned to conduct UX work. From there, a UX practice emerges as increasingly more formal as more UX methods are introduced more often into different contexts. Finally, as a library reaches UX maturity, we see a frequent application of a wide variety of UX methods in all corners of the library and with many stakeholders, along with organizational decision-making that regularly includes UX research data. A UX Maturity Scale for Libraries To help in understanding of the UX maturity scale and the characteristics related to each of its stages, we have adapted the Nielsen Norman UX Maturity scale for a library context. Table 8 shows a set of organizational characteristics that correspond to the eight stages of UX maturity. The indicators in table 8 are presented as an approximate guideline for understanding and diagnosing UX maturity. Stage Key Indicators Stage 1–2 Apathy or hostility to UX practice; lack of resources and staff for UX Stage 3 Ad hoc UX practices within the organization; UX is practiced, but unofficially and without dedicated resources or staff; leadership does not fully understand or support UX Stage 4 Leadership beginning to understand and support UX; dedicated UX budget; UX is assigned fully or partly to a permanent position Stage 5 The UX lead or UX group collaborates with units across the organization and contributes UX data meaningfully to organizational and strategic decision-making Stage 6 UX research data is regularly included in projects and decision-making; a wide variety of methods are practiced regularly by multiple departments Stage 7–8 UX is practiced throughout the organization; decisions are made and resources are allocated only with UX insights as a guide Table 8. Key indicators for UX maturity in academic libraries. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 20 This scale reflects the research presented in this paper while building on related models and prior research (more granularity is available in Stages 2–6 because we received more survey responses representing those stages). We note that our research is consonant with prior work in this area. Priestner includes a greater focus on library users (in contrast to a focus on library staff) as a key driver of library UX maturity.27 MacDonald reports that UX work is defined by applied methods, in particular, qualitative research.28 Sharon describes a UX maturity model based on two primary factors: the presence of UX researchers on staff and whether the organization actually listens to and responds to UX research.29 Finally, Sheldon-Hess bases library UX maturity on the extent of applied UX methods and the level of user-centeredness present in an organization, as indicated by degree to which staff consider user perspectives in internal communications and decision - making.30 Taken together, we see common strands that can help illuminate the key factors of UX maturity in libraries: applied methods, leadership support in the form of resources and strategic alignment, organizational collaboration, and decision-making that includes UX research. Strategies for Climbing the Maturity Scale: Toward a More User-Centered Library Our results reveal a few key barriers and boosts to higher maturity, and one key point of stagnation. Across the maturity scale, important factors that positively influence maturity involve leadership support and resource allocation toward UX in the form of personnel and infrastructure such as physical space, materials, strategic direction, and a working budget. Notably, respondents in our survey reported being stuck at Stage 3 due to a lack of leadership support. For instance, when resource-related comments appeared, we primarily heard about a lack of resources, which impaired maturity. Participants reported a mixture of personnel in support of UX work. Some libraries have a staff member dedicated to UX but lack a committee structure to support and advocate for the work. Other libraries do not have dedicated UX staff but had formed committee infrastructure to collaboratively move UX forward. Participants who lacked either a UX group or a UX lead reported lower levels of maturity and were particularly stagnated at Stage 3 (see figure 5 above). Alternatively, libraries are boosted to Stage 5 with the presence of a fully empowered UX lead who has the support of a UX group or committee that can network throughout the organization and drive collaboration and cross-functional implementation of UX methods and research data. We found that respondents from libraries that possessed both a dedicated UX staff and a UX group tended to place themselves higher on the maturity scale. For those who reside at Stage 5, having a UX group or a UX lead are the two main themes present in the survey. To move forward to Stage 5, a library needs to organize a UX group with an appointed lead to coordinate UX practice widely throughout the organization, including in library spaces, web presence, learning services, and digital initiatives. A systematic and cooperative UX approach planned by an official UX group and led by a designated UX lead is the key indicator of Stage 5. The support for the group and its lead needs to come rom not only leadership, but also colleagues throughout the library, which relates to the two major themes of leadership support and organizational collaboration. Stage 7–8 is achievable only with significant investment in UX. Given parent entity pressures, existing hierarchies, and prevailing non-user-centered cultures, libraries face a formidable set of challenges on the road to becoming user-centered organizations.31 This road is somewhat illuminated by the small number of survey respondents that marked themselves at a Stage 7 or 8. Highlights from their responses are instructive. One respondent told us, INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 21 We have multiple teams in the library to help with service design, conducting and gathering user research, and helping library staff think more about the user in their everyday work. We also have a couple Special Interest Groups (SIGs) dedicated to user research, UX, and assessment. We also have multiple departments within the library with UX expertise. From this response, we can see the key characteristics of UX maturity: leadership support up the line along with wide-spread collaboration throughout the organization. Staff infrastructures including multiple UX-oriented committees help drive and coordinate UX work. This respondent also reported the recent hiring of an assessment librarian situated in the library’s administration department who will help coordinate UX work throughout the organization. These elements work together to meaningfully integrate user perspectives into both digital and physical spaces and in multiple units. Moreover, this respondent marked 19 out of 20 UX methods currently in use (all but diary/camera studies), thus reinforcing the symbiotic relationship between UX maturity and UX methods: the variety of methods in use are a signal of maturity, and correspondingly, a greater maturity allows the space and resources for the application of more and different methods. Another survey respondent at Stage 7 remarked the following, My workplace has been very supportive in addressing UX issues both in digital and physical spaces. Since being hired, I have created workflows that incorporate data that we gather from users. If there isn’t data gathered in a certain area, we usually find a way to update workflows so that we can get that data. Almost every project that I have worked on digitally and in the physical spaces at the library has been the result of UX/UI data that has been gathered from our users. The elevated level of maturity at this library is especially reflected through the practice of “almost every project” being driven by user data. A truly user-centered library indeed integrates user data across all projects and advocates for the user at every opportunity. This respondent also marked a high variety of methods currently being practiced: 17 out of a possible 20 (methods not in use include diary/camera studies, user stories, and competitive analysis), further underscoring the two-way connection between methods and maturity. In further considering the upper reaches of maturity, we are inspired by an emerging theory of design-oriented librarianship that signals a professional paradigm shift such that UX could become recognized as a fundamental component of library research, education, and practice. 32 By investing more in UX methods, practices, and principles, libraries can achieve greater value and empowerment for our communities by designing more user-centered services and tools.33 Ultimately, achieving Stage 7–8 will result from deeply integrating user-centeredness across all operational phases, strategic planning, and decision-making of a library organization. LIMITATIONS We note a few limitations of our study. First, the UX stages used in the survey were defined by Jakob Nielson in 2006 for corporate application, so it is perhaps a bit dated. Further, the main goal of our statistical analysis is to develop a model that can accurately predict the UX maturity of a INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 22 library based on the UX methods employed at the institution combined with organizational characteristics. Allison outlines three broad categories of error in regression analysis: Measurement error: Very few variables can be measured with perfect accuracy, especially in the social sciences. Sampling error: In many cases, our data are only a sample from a larger population, and the sample will never be exactly like the population. Uncontrolled variation: [Age and schooling] are surely not the only variables that affect a person’s income, and these uncontrolled variables may “disturb” the relationship.34 In terms of measurement error, survey respondents may have bias when self-reporting maturity stage due to social pressures to produce desirable responses, meaning people tend to respond to self-report items in a manner that makes themselves look good.35 The resulting measurement error takes the form of over-reporting “desirable behavior” or under-reporting “undesirable behavior.” This is evident in some responses for UX maturity stages. For example, one respondent chose Stage 5—“Managed Usability”—but the comment described a slightly different picture: I think we are still floundering between “Dedicated UX Budget” and “Managed Usability.” . . . We are at the stage where people know they should consult with us, but either they don’t OR they do but don’t really hear the results, they are using us to confirm what they want to hear. In terms of sampling error, self-selection bias is a factor: our respondents might not be representative of the full population of UX librarians. We also did not make all of our questions mandatory, and as a result were not able to make use of all possible data within the scope of our survey. In terms of uncontrolled variation, our survey and statistical model does not fully account for all variables that influence UX maturity in libraries; for example, we included a limited list of UX methods, and we did not include questions that inquired specifically into the presence of a UX lead or UX group. FUTURE DIRECTIONS We see at least three paths forward for future research related to UX methods and maturity. First, librarianship would benefit from a UX maturity scale created specifically with and for our field’s theoreticians and practitioners. We propose one such scale above, but our scale has not undergone further testing, research, or validation. We note especially the library UX maturity scales of Sheldon-Hess and MacDonald, which could be further synthesized or built upon.36 Second, a self- assessment tool for diagnosing UX maturity could be developed based on a validated maturity scale. And third, the theory advanced by Clarke that librarianship can be usefully conceived of and practiced as a design discipline warrants further critical attention, especially as it relates to the application of UX methods and the development of UX maturity.37 CONCLUSION We applied a mixed-methods approach that involved content analysis and statistical analysis to a profession-wide survey. Our research data and analysis demonstrate the type and extent of UX methods currently in use by academic libraries. The five most common methods are usability INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 23 testing, user surveys, user interviews, accessibility evaluation, and field studies. The five least common methods are journey maps, benchmark testing, design review, FAQ review, and diary/camera studies. Furthermore, we identify the organizational characteristics that help or hinder the development of UX maturity. UX maturity in libraries is related to four key factors: the number of UX methods currently in use; the level of support from leadership in the form of strategic alignment, budget, and personnel; the extent of collaboration throughout the organization; and the degree to which organizational decisions are influenced by UX research. When one or more of these four connected factors advances, so too does UX maturity. We close by emphasizing three key factors for reaching higher levels of UX maturity. First, we encourage library leadership to see the value of UX and support its practice through strategic alignment and resource allocation. Second, we encourage libraries to commit to integrating UX principles and practices across all units, especially into leadership groups and through organization-wide collaboration and workflows. Third, UX methods should be reinforced and amplified with personnel, such as a standing UX group and a dedicated UX lead that can help direct UX work and enhance UX maturity. Libraries have the promise and potential to more deeply practice UX. Doing so can allow libraries to more deeply connect with users and reach higher levels of UX maturity, with the ultimate result of delivering tools and services that further empower our user communities. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 24 APPENDIX A: GLOSSARY OF STATISTICAL TERMS Term Working Definition adjusted R-squared Adjusted R2 = variance of fitted model values / variance of response values. “The adjusted R-squared compares the descriptive power of regression models—two or more variables—that include a diverse number of independent variables—known as a predictor. Every predictor or independent variable, added to a model increases the R-squared value and never decreases it. So, a model that includes several predictors will return higher R-Squared values and may seem to be a better fit. However, this result is due to it including more terms. The adjusted R-squared compensates for the addition of variables and only increases if the new predictor enhances the model above what would be obtained by probability. Conversely, it will decrease when a predictor improves the model less than what is predicted by chance.” Source: https://www.investopedia.com/ask/answers/012615/whats- difference-between-rsquared-and-adjusted-rsquared.asp confidence level “The confidence level tells you how sure you can be. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval. The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain. Most researchers use the 95% confidence level.” Source: https://researchbasics.education.uconn.edu/confidence- intervals-and-levels/ confidence interval “A confidence interval is an interval which has a known and controlled probability (generally 95% or 99%) to contain the true value.” Source: https://stats.oecd.org/glossary/detail.asp?ID=5055 explained variance “Explained variance (also called explained variation) is used to measure the discrepancy between a model and actual data. In other words, it’s the part of the model’s total variance that is explained by factors that are actually present and isn’t due to error variance.” Source: https://www.statisticshowto.datasciencecentral.com/explained -variance-variation/ explanatory and response variables “The response variable is the focus of a question in a study or experiment. An explanatory variable is one that explains changes in that variable. It can be anything that might affect the https://www.investopedia.com/ask/answers/012615/whats-difference-between-rsquared-and-adjusted-rsquared.asp https://www.investopedia.com/ask/answers/012615/whats-difference-between-rsquared-and-adjusted-rsquared.asp https://researchbasics.education.uconn.edu/confidence-intervals-and-levels/ https://researchbasics.education.uconn.edu/confidence-intervals-and-levels/ https://stats.oecd.org/glossary/detail.asp?ID=5055 https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/variance/ https://www.statisticshowto.datasciencecentral.com/explained-variance-variation/ https://www.statisticshowto.datasciencecentral.com/explained-variance-variation/ INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 25 response variable.” Source: https://www.statisticshowto.datasciencecentral.com/explanato ry-variable/ multiple regression “Multiple regression is a statistical method for studying the relationship between a single dependent [or response] variable and one or more independent [or explanatory] variables. It is unquestionably the most widely used statistical technique in the biological and physical sciences.”38 null hypothesis “In general, this term relates to a particular hypothesis under test, as distinct from the alternative hypotheses which are under consideration. It is therefore the hypothesis which determines the probability of the type I error. In some contexts, however, the term is restricted to an hypothesis under test of ‘no difference’.” Source: https://stats.oecd.org/glossary/detail.asp?ID=3767 probability or p-value “The p value is the probability of getting our observed result, or a more extreme result, if the null hypothesis is true.”39 simple linear regression “Simple linear regression models the relationship between the magnitude of one variable and that of a second - for example, as X increases, Y also increases. Or as X increases, Y decreases.”40 statistical significance “Statistical significance refers to the claim that a result from data generated by testing or experimentation is not likely to occur randomly or by chance but is instead likely to be attributable to a specific cause. Having statistical significance is important for academic disciplines or practitioners that rely heavily on analyzing data and research, such as economics, finance, investing, medicine, physics, and biology. Statistical significance can be considered strong or weak. When analyzing a data set and doing the necessary tests to discern whether one or more variables have an effect on an outcome, strong statistical significance helps support the fact that the results are real and not caused by luck or chance. Simply stated, if a statistic has high significance then it's considered more reliable.” Source: https://www.investopedia.com/terms/s/statistical- significance.asp variance “The variance is the mean square deviation of the variable around the average value. It reflects the dispersion of the empirical values around its mean.” Source: https://stats.oecd.org/glossary/detail.asp?ID=5160 https://www.statisticshowto.datasciencecentral.com/explanatory-variable/ https://www.statisticshowto.datasciencecentral.com/explanatory-variable/ https://stats.oecd.org/glossary/detail.asp?ID=3767 https://www.investopedia.com/terms/s/statistical-significance.asp https://www.investopedia.com/terms/s/statistical-significance.asp https://stats.oecd.org/glossary/detail.asp?ID=5160 INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 26 APPENDIX B: ADDITIONAL DATA ANALYSIS Model: Stage as a Function of Population Rank Model: Stage as a Function of Total Methods INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 27 Model: Variables that Combine to Produce the Most Accurate Stage Predictions INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 28 ENDNOTES 1 David Liddle, “Best Value—The Impact on Libraries: Practical Steps in Demonstrating Best Value,” Library Management 20, no. 4 (June 1, 1999): 206–14, https://doi.org/10.1108/01435129910268982. 2 Daniel Pshock, “The User Experience of Libraries: Serving The Common Good,” User Experience 17, no. 2 (2017), https://web.archive.org/web/20190822051708/http://uxpamagazine.org/the-user- experience-of-libraries/. 3 Bruce Massis, “The User Experience (UX) in Libraries,” Information and Learning Sciences 119, no. 3/4 (March 12, 2018): 241–44, https://doi.org/10.1108/ILS-12-2017-0132. 4 Rachel Fleming-May et al., “Experience Assessment: Designing an Innovative Curriculum for Assessment and UX Professionals,” Performance Measurement and Metrics 19, no. 1 (December 15, 2017): 30–39, https://doi.org/10.1108/PMM-09-2017-0036; Rachel Ivy Clarke, Satyen Amonkar, and Ann Rosenblad, “Design Thinking and Methods in Library Practice and Graduate Library Education,” Journal of Librarianship and Information Science (September 8, 2019), https://doi.org/10.1177/0961000619871989; Aja Bettencourt-McCarthy and Dawn Lowe- Wincentsen, “How Do Undergraduates Research? A User Experience Experience,” OLA Quarterly 22, no. 3 (February 22, 2017): 20–25, https://doi.org/10.7710/1093-7374.1866; Juan Carlos Rodriguez, Kristin Meyer, and Brian Merry, “Understand, Identify, and Respond: The New Focus of Access Services,” portal: Libraries and the Academy 17, no. 2 (April 8, 2017): 321–35, https://doi.org/10.1353/pla.2017.0019; Asha L. Hegde, Patricia M. Boucher, and Allison D. Lavelle, “How Do You Work? Understanding User Needs for Responsive Study Space Design,” College & Research Libraries 79, no. 7 (2018), https://doi.org/10.5860/crl.79.7.895; Amy Deschenes, “Improving the Library Homepage through User Research—Without a Total Redesign,” Weave: Journal of Library User Experience 1, no. 1 (2014), https://doi.org/10.3998/weave.12535642.0001.102. 5 Amanda Kraft, “Parsing the Acronyms of User-Centered Design,” in 2019 ASCUE Proceedings (Association Supporting Computer Users in Education (ASCUE), Myrtle Beach, South Carolina, 2019), 61–69, https://eric.ed.gov/?id=ED597115. 6 IDEO, The Field Guide to Human-Centered Design (San Francisco: IDEO, 2015); Joe Marquez and Annie Downey, Library Service Design: A LITA Guide to Holistic Assessment, Insight, and Improvement (Lanham, MD: Rowman & Littlefield, 2016); Scott W. H. Young and Celina Brownotter, “Toward a More Just Library: Participatory Design with Native American Students,” Weave: Journal of Library User Experience 1, no. 9 (2018), https://doi.org/10.3998/weave.12535642.0001.901. 7 Aaron Schmidt and Amanda Etches, Useful, Usable, Desirable: Applying User Experience Design to Your Library (Chicago: ALA Editions, 2014); Joe J. Marquez and Annie Downey, Getting Started in Service Design: A How-To-Do-It Manual for Librarians (Chicago: American Library Association, 2017). https://doi.org/10.1108/01435129910268982 https://web.archive.org/web/20190822051708/http:/uxpamagazine.org/the-user-experience-of-libraries/ https://web.archive.org/web/20190822051708/http:/uxpamagazine.org/the-user-experience-of-libraries/ https://doi.org/10.1108/ILS-12-2017-0132 https://doi.org/10.1108/PMM-09-2017-0036 https://doi.org/10.1177/0961000619871989 https://doi.org/10.7710/1093-7374.1866 https://doi.org/10.1353/pla.2017.0019 https://doi.org/10.5860/crl.79.7.895 https://doi.org/10.3998/weave.12535642.0001.102 https://eric.ed.gov/?id=ED597115 https://doi.org/10.3998/weave.12535642.0001.901 INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 29 8 Zoe Chao, “Rethinking User Experience Studies in Libraries: The Story of UX Café,” Weave: Journal of Library User Experience 2, no. 2 (2019), https://doi.org/10.3998/weave.12535642.0002.203. 9 Daniel Pshock, “Results from the 2017 Library User Experience Survey,” Designing for Digital, March 6, 2018, https://web.archive.org/web/20190829163234/https://d4d2018.sched.com/event/DM8H/d 16-02-results-from-the-2017-library-user-experience-survey. 10 Andy Priestner, “Approaching Maturity? UX Adoption in Libraries,” in User Experience in Libraries: Yearbook 2017, ed. Andy Priestner (Cambridge, England: UX in Libraries, 2017), 1–8. 11 Craig M. MacDonald, “‘It Takes a Village’: On UX Librarianship and Building UX Capacity in Libraries,” Journal of Library Administration 57, no. 2 (February 17, 2017): 194–214, https://doi.org/10.1080/01930826.2016.1232942. 12 Tomer Sharon, “UX Research Maturity Model,” Prototypr (blog), 2016, https://web.archive.org/web/20190829163113/https://blog.prototypr.io/ux-research- maturity-model-9e9c6c0edb83?gi=c462f7ac4600. 13 Coral Sheldon-Hess, “UX, Consideration, and a CMMI-Based Model,” Coral Sheldon-Hess Blog (blog), 2013, https://web.archive.org/web/20190117144529/http://www.sheldon- hess.org/coral/2013/07/ux-consideration-cmmi/. 14 Jakob Nielsen, “Corporate UX Maturity: Stages 1–4,” Nielsen Norman Group, 2006, https://web.archive.org/web/20190709231540/https://www.nngroup.com/articles/ux- maturity-stages-1-4/; Jakob Nielsen, “Corporate UX Maturity: Stages 5–8,” Nielsen Norman Group, 2006, https://web.archive.org/web/20190709231533/https://www.nngroup.com/articles/ux- maturity-stages-5-8/. 15 Nikki Anderson, “UX Maturity: How to Grow User Research in Your Organization,” Medium, May 1, 2019, https://medium.com/researchops-community/ux-maturity-how-to-grow-user- research-in-your-organization-848715c3543. 16 Including: the User Experience Working Group under the Digital Libraries Federation Assessment Interest Group (DLF AIG UX), Code4Lib, Assessment Listserv of Association of Research Libraries (ARL), Access Conference List, Coalition for Networked Information (CNI), Library and Information Technology Association (LITA), Library User Experience (LibUX) Slack Channel, and ALA User Experience Interest Group. 17 Current index and classification list available from http://www.carnegieclassifications.iu.edu/classification_descriptions/size_setting.php ; Data at time of analysis available from Indiana University Center for Postsecondary Research (2018). Carnegie Classifications 2018 public data file, https://web.archive.org/web/20191006220952/http://carnegieclassifications.iu.edu/downl oads/CCIHE2018-PublicDataFile.xlsx. https://doi.org/10.3998/weave.12535642.0002.203 https://web.archive.org/web/20190829163234/https:/d4d2018.sched.com/event/DM8H/d16-02-results-from-the-2017-library-user-experience-survey https://web.archive.org/web/20190829163234/https:/d4d2018.sched.com/event/DM8H/d16-02-results-from-the-2017-library-user-experience-survey https://doi.org/10.1080/01930826.2016.1232942 https://web.archive.org/web/20190829163113/https:/blog.prototypr.io/ux-research-maturity-model-9e9c6c0edb83?gi=c462f7ac4600 https://web.archive.org/web/20190829163113/https:/blog.prototypr.io/ux-research-maturity-model-9e9c6c0edb83?gi=c462f7ac4600 https://web.archive.org/web/20190117144529/http:/www.sheldon-hess.org/coral/2013/07/ux-consideration-cmmi/ https://web.archive.org/web/20190117144529/http:/www.sheldon-hess.org/coral/2013/07/ux-consideration-cmmi/ https://web.archive.org/web/20190709231540/https:/www.nngroup.com/articles/ux-maturity-stages-1-4/ https://web.archive.org/web/20190709231540/https:/www.nngroup.com/articles/ux-maturity-stages-1-4/ https://web.archive.org/web/20190709231533/https:/www.nngroup.com/articles/ux-maturity-stages-5-8/ https://web.archive.org/web/20190709231533/https:/www.nngroup.com/articles/ux-maturity-stages-5-8/ https://medium.com/researchops-community/ux-maturity-how-to-grow-user-research-in-your-organization-848715c3543 https://medium.com/researchops-community/ux-maturity-how-to-grow-user-research-in-your-organization-848715c3543 http://www.carnegieclassifications.iu.edu/classification_descriptions/size_setting.php https://web.archive.org/web/20191006220952/http:/carnegieclassifications.iu.edu/downloads/CCIHE2018-PublicDataFile.xlsx https://web.archive.org/web/20191006220952/http:/carnegieclassifications.iu.edu/downloads/CCIHE2018-PublicDataFile.xlsx INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 30 18 Five institutions appear in this count twice, that is, on five occasions, two persons from the same institution responded separately to the survey. We invited this type of response to capture diversity of opinion and experience within an organization. 19 One institution appears in this count twice, for the same reason as explained in the previous endnote. 20 Susan Farrell, “UX Research Cheat Sheet,” Nielsen Norman Group, February 12, 2017, https://web.archive.org/web/20190828224735/https://www.nngroup.com/articles/ux- research-cheat-sheet/. 21 Klaus Krippendorff, Content Analysis: An Introduction to Its Methodology, 3rd edition (Los Angeles: SAGE, 2012). 22 Scott W. H. Young, Zoe Chao, and Adam Chandler, “Data From: User Experience Methods and Maturity in Academic Libraries,” Distributed by Dryad, https://doi.org/10.5061/dryad.jwstqjq5d. 23 Paul D. Allison, Multiple Regression: A Primer (Thousand Oaks, CA: Pine Forge, 1999), 6. 24 We are aware that our use of linear regression with this small sample surely “over-fits” the dataset, that is, the model is unlikely to predict as accurately if applied to a different dataset. The model will undergo further refinement in the future. 25 We made a conscious choice to leave in some variables in this model that are statistically insignificant. We did so because it might be too early to fully dismiss these elements as unimportant; it could be that our sample was too small to really be certain. Furthermore, our primary emphasis is in creating a model that does a good job of accurately predicting stage based on an array of different characteristics. Removing all the nonsignificant variables in this model would actually lower the prediction accuracy. Adjusted R-squared accounts for additional variables. 26 For more description of the Multiple Linear Regression Model, please see https://web.archive.org/web/20191006231250/https://newonlinecourses.science.psu.edu/s tat462/node/131/. 27 Priestner, “Approaching Maturity? UX Adoption in Libraries.” 28 Craig M. MacDonald, “User Experience Librarians: User Advocates, User Researchers, Usability Evaluators, or All of the Above?,” Proceedings of the Association for Information Science and Technology 52, no. 1 (2015): 1–10, https://doi.org/10.1002/pra2.2015.145052010055. 29 Sharon, “UX Research Maturity Model.” 30 Sheldon-Hess, “UX, Consideration, and a CMMI-Based Model.” 31 MacDonald, “‘It Takes a Village.’” https://web.archive.org/web/20190828224735/https:/www.nngroup.com/articles/ux-research-cheat-sheet/ https://web.archive.org/web/20190828224735/https:/www.nngroup.com/articles/ux-research-cheat-sheet/ https://doi.org/10.5061/dryad.jwstqjq5d https://web.archive.org/web/20191006231250/https:/newonlinecourses.science.psu.edu/stat462/node/131/ https://web.archive.org/web/20191006231250/https:/newonlinecourses.science.psu.edu/stat462/node/131/ https://doi.org/10.1002/pra2.2015.145052010055 INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2020 USER EXPERIENCE METHODS AND MATURITY IN ACADEMIC LIBRARIES | YOUNG, CHAO, AND CHANDLER 31 32 Rachel Ivy Clarke, “Toward a Design Epistemology for Librarianship,” The Library Quarterly 88, no. 1 (2018): 41–59, https://doi.org/10.1086/694872; Rachel Ivy Clarke, “How We Done It Good: Research through Design as a Legitimate Methodology for Librarianship,” Library & Information Science Research 40, no. 3 (July 1, 2018): 255–61, https://doi.org/10.1016/j.lisr.2018.09.007. 33 Clarke, Amonkar, and Rosenblad, “Design Thinking and Methods in Library Practice and Graduate Library Education.” 34 Allison, Multiple Regression: A Primer, 14. 35 Delroy L. Paulhus, “Socially Desirable Responding: The Evolution of a Construct,” in The Role of Constructs in Psychological and Educational Measurement (Mahwah, NJ: Lawrence Erlbaum, 2002), 49–69. 36 Sheldon-Hess, “UX, Consideration, and a CMMI-Based Model”; MacDonald, “‘It Takes a Village.’” 37 Rachel Ivy Clarke, “Design Thinking for Design Librarians: Rethinking Art and Design Librarianship,” in The Handbook of Art and Design Librarianship, ed. Paul Glassman and Judy Dyki, 2nd edition (Chicago: ALA Neal-Schuman, 2017), 41–49; Clarke, “Toward a Design Epistemology for Librarianship”; Clarke, “How We Done It Good”; Clarke, Amonkar, and Rosenblad, “Design Thinking and Methods in Library Practice and Graduate Library Education”; Shannon Marie Robinson, “Critical Design in Librarianship: Visual and Narrative Exploration for Critical Praxis,” The Library Quarterly 89, no. 4 (October 1, 2019): 348–61, https://doi.org/10.1086/704965. 38 Allison, Multiple Regression: A Primer, 1. 39 Geoff Cumming, Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta- Analysis (New York: Routledge, 2012), 26. 40 Peter Bruce and Andrew Bruce, “Regression and Prediction,” in Practical Statistics for Data Scientists: 50 Essential Concepts (Sebastopol, CA: O’Reilly Media, 2017). https://doi.org/10.1002/pra2.2015.145052010055 https://doi.org/10.1016/j.lisr.2018.09.007 https://doi.org/10.1086/704965 ABSTRACT IntroDUCTION Background and Motivation: UX in Academic Libraries Methods Research Questions Survey Participants Materials and Procedure Demographics and UX Methods UX Maturity Research Data Analysis Content Analysis Statistical Analysis Data Preparation Research Dataset Survey Respondents Results RQ1: How mature is user experience practice within academic libraries? RQ2: What Factors Influence UX Maturity? Overview of Statistical Analysis Results Size of Institution Methods Currently in Use by Academic Libraries Factors that Influence UX Methods: Recency, Formality, Regularity Factors that Influence UX maturity: Leadership Support, Collaboration, UX Lead, UX Group, Growth, and Resources Full UX Maturity Model: UX Maturity as a Function of UX Methods A Statistical Example Case: Estimating UX Maturity Discussion Leadership Support and Strategic Alignment Organizational Collaboration Applied UX Methods UX as an Emerging Practice within Libraries A UX Maturity Scale for Libraries Strategies for Climbing the Maturity Scale: Toward a More User-Centered Library Limitations Future Directions Conclusion Appendix A: Glossary of statistical terms Appendix B: Additional Data Analysis Model: Stage as a Function of Population Rank Model: Stage as a Function of Total Methods Model: Variables that Combine to Produce the Most Accurate Stage Predictions ENDNOTES