570 A Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues Kyle M.L. Jones, Kristin A. Briney, Abigail Goben, Dorothea Salo, Andrew Asher, and Michael R. Perry* Universities are pursuing learning analytics practices to improve returns from their investments, develop behavioral and academic interventions to improve student suc- cess, and address political and financial pressures. Academic libraries are additionally undertaking learning analytics to demonstrate value to stakeholders, assess learning gains from instruction, and analyze student-library usage, et cetera. The adoption of these techniques leads to many professional ethics issues and practical concerns related to privacy. In this narrative literature review, we provide a foundational back- ground in the field of learning analytics, library adoption of these practices, and identify ethical and practical privacy issues. Introduction In the wake of the ubiquitous computing infrastructure that higher education institutions have created, a “rising tide”1 of student data is overflowing as a result of interactions with digital systems. Institutions are motivated to mine this “gold,”2 to extract this “oil”3 as a social, political, and financial resource, and as a means to effect positive educational outcomes. This sociopolitical and sociotechnical movement to collect, describe, and act on student data has created momentum in the field of learning analytics (LA). LA are defined as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.”4 As a practice, LA focuses on student success and student * Kyle M.L. Jones is an Assistant Professor in the Department of Library and Information Sciences at Indiana University-Indianapolis (IUPUI); email: kmlj@iupui.edu. Kristin A. Briney is Biology and Biological Engineering Librarian at California Institute of Technology; email: briney@caltech.edu. Abigail Goben is an Associate Professor and Information Services and Liaison Librarian at the University of Illinois at Chicago; email: agoben@uic.edu. Dorothea Salo is a Distinguished Faculty Associate at the University of Wisconsin-Madison; email: salo@wisc. edu. Andrew Asher is Assessment Librarian and Associate Librarian, Planning and Administration, at Indiana University-Bloomington; email: asherand@indiana.edu. Michael R. Perry is Head of Assessment and Planning at Northwestern University; email: michael.perry@northwestern.edu. This project was made possible in part by the Institute of Museum and Library Services (LG-96-18-0044-18). The views, findings, conclusions, or recom- mendations expressed in this conference proceeding do not necessarily represent those of the Institute of Museum and Library Services. The team would like to thank Aubree Tillett, a research assistant from the University of Wisconsin-Milwaukee, for her support and Maura Smale and Mariana Regaldo for their input. ©2020 Kyle M.L. Jones, Kristin A. Briney, Abigail Goben, Dorothea Salo, Andrew Asher, and Michael R. Perry, Attribution- NonCommercial (https://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC mailto:kmlj@iupui.edu mailto:briney@caltech.edu mailto:agoben@iuc.edu mailto:salo@wisc.edu mailto:salo@wisc.edu mailto:asherand@indiana.edu mailto:michael.perry@northwestern.edu https://creativecommons.org/licenses/by-nc/4.0/ Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 571 learning contexts, and also on institutional efficiency and effectiveness.5 LA materialized in 2010 as a technology to watch for “on the horizon”6 and has gained significant interest among administrators, researchers, technologists, policymakers, and (increasingly) librarians. As documented in the academic library impact (ALI)7 report, learning analytics are of great interest to academic libraries. Library LA projects and research have grown out of adminis- trative pressure to demonstrate returns on investments and a desire to confirm which library practices and resources contribute to institutional priorities and impact student learning. Both of the initiatives of the Association of College & Research Libraries (ACRL), the Value of Aca- demic Libraries (VAL) and Assessment in Action (AiA),8 not only encourage expanded data capture within the library but also the combination of those data with other campus data sets, including demographic and other identifying data. However, this encouragement is not fully aligned with professional ethics codes, such as that of the American Library Association (ALA). Emerging research has begun to examine these LA projects through a critical lens to identify professional and other ethical problems.9 However, for librarians to make informed decisions about LA, it is essential for the profession to develop a foundational understand- ing of LA as a field, its recent impact on academic librarianship, and inherent privacy issues, which this narrative review aims to accomplish. Additionally, the review identifies useful pri- vacy resources and areas where further research is necessary. We refer readers to the existing research on professional ethics to explore these issues through this particular analytical lens. We have divided the review into four substantive parts: 1) background information on LA; 2) the adoption of LA by academic libraries, primarily in the United States; 3) the emerging ethical issues related to library LA; and 4) practical privacy issues arising because of library LA. In each of these sections, appendices point to useful resources. Finally, the article ends with a discussion that promotes ideas and strategies for addressing some of the ethical problems. Background on Learning Analytics The embrace of learning analytics (LA) echoes the 1970s rise of decision support practices10 and, more recently, business intelligence systems.11 However, LA are more accurately described as branching off of the academic analytics movement of the mid-2000s.12 Campbell, DeBlois, and Oblinger write that “[t]hanks to enterprise-wide systems that generate massive amounts of data, data warehouses that aggregate disparate types of data, and processing power that sifts, sorts, and surfaces patterns,” analytics have become a viable set of tools.13 LA researchers focus on digital learning environments because of the bountiful data collected by educational and other technologies in support of running complex academic institutions.14 One example is Unizin and its Common Data Model (UCDM).15 Unizin is an institutional consortium of more than 20 public universities and university systems serving a collective student body totaling more than 900,000.16 The UCDM maps a complex array of student-related data, which Unizin will aggregate in a centralized data warehouse for consortia members. The model contains demographic, economic, and academic information, among other things. It captures students’ academic work (such as assignments and discussion posts) and all their interactions with a learning management system (LMS)—logins, individual clicks, time-on-page—all the “digital footprints” expressed in an LMS’s logs and metadata. Early LA efforts, such as those at Purdue University, used LMS data in combination with academic performance data to predict whether a student was at risk for academic failure in a specific course.17 Purdue’s predictive work continues to serve as a model for many LA 572 College & Research Libraries April 2020 developments, including feature changes to common LMSs like Blackboard and Canvas. Im- portantly, critics later discovered significant issues with Purdue’s predictive model. Jones and McCoy explain that the researchers responsible for the model came to inaccurate conclusions by staking their statistically significant—but wrong—claim on a spurious correlation, seeing patterns in the process that did not exist.18 Other LA systems increasingly “nudge” students toward certain behaviors, such as com- municating with instructors or seeking campus resources, based on these predictions.19 Some analytics systems adapt course materials and assessments based on students’ learning styles, past performance, and system behaviors.20 LA designers have also developed predictive analyt- ics, nudging strategies, and adaptive learning algorithms for advising systems, which provide personalized course and program recommendations and suggest advisor interventions when the system predicts students will do poorly in their academics or they are not engaged with the university community.21 Beyond education technologies, institutions can and often do capture data related to in- teractions with virtual assistants, smartphones, wearable devices, tablets, personal computers, sensors, and ID card readers (example: card swipes). For example, Northeastern University, St. Louis University, Arizona State University, and the University of Texas at Dallas started initiatives using Amazon Echo Dots, whose Alexa personal assistant can retrieve campus information and, in some cases, get personalized information from SISs.22 Both Amazon and these institutions gained access to student data.23 These data were so valuable to UT-Dallas’s IT department that it proposed refusing to allow students to turn the Dots off.24 Purdue and Oral Roberts University pursued “quantified self” technologies to inform students of their learning behaviors and track their nonacademic activities; the former university uses mobile, desktop, and smartwatch applications, while the latter required its students to wear Fitbits.25 At the University of Arizona, a researcher analyzed students swiping their identification cards at more than 700 campus locations to study routines and social networks.26 In addition to campus-based data, some LA proponents have advocated for integrating social media data.27 A significant majority of 18- to 29-year-olds use social media; the enormous amount of these data may explain the desire to mine them for analytics.28 Institutions have already monitored student social media behaviors on a small scale, though not always for educational purposes. Georgia Tech University profiled a student activist,29 and the Univer- sity of Virginia hired a social media monitoring company to surveil students’ social media for threats to campus safety.30 Nevertheless, as Chen, Vorvoreany, and Madhavan demonstrate, it is becoming increasingly easy for institutions to collect their students’ Twitter content and metadata wholesale via APIs using Boolean search terms and geolocation filters.31 Beyond internal usage of student data, institutions have begun to aggregate their student data collaboratively. Reyes notes that new technology enables “institutions to access, col- laborate and contribute their data in an effort to build single platforms with the capacity for sharing multiple data sets.”32 Maturing interoperability standards, like Caliper, are enhancing aggregation initiatives.33 One example of mass aggregation is Carnegie Mellon University’s DataShop, which serves as “a central repository for data on the interactions between students and educational software,”34 primarily for research purposes. While proponents often argue that LA are focused solely on students and their learning,35 the accepted definition of LA and ongoing rhetoric around them reveals a variety of motiva- tions and goals. The ability to surface, analyze, and act on the “data explosion”36 has led some Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 573 to think that LA may serve as panaceas that can resolve seemingly intractable problems like recruitment, retention, and provide data useful for responding to external demands.37 Internal and external stakeholders believe LA can help demonstrate institutional “performance and cost-effectiveness” in ways that highlight solvency and increase public trust in and support of higher education as a costly investment.38 For instance, Mount St. Mary’s University planned to use predictive analytics to proactively dismiss students unlikely to be retained, arguing that such actions would improve its national ranking.39 Others have pushed back against the pressure to quantify student life, claiming that LA are leading institutions to count for the sake of counting without due consideration for ways such analytics and the infrastructures that support analytic practices can have “detrimental consequences.”40 Moreover, present evidence for the effectiveness of LA is mixed at best, raising a concern that the benefits are not worth the financial costs and/or possible harms.41 Academic Library Participation in Learning Analytics Academic libraries, like the institutions they are part of, have identified many reasons to engage in LA, including a need to maintain alignment with greater institutional efforts and interests.42 Library administrators are facing pressure from their peers at the campus level to demonstrate their contributions to university-level goals.43 A survey of chief academic of- ficers (CAOs) found that they desired quantitative evidence as a key requirement to support funding requests, arguing that correlational data “to retention, success, and even evidence of learning information literacy skills” is of the utmost importance to justify resource allocations. Twenty years of research focused on quantifying library value and impact (however defined) have resulted from these pressures to explain ever-increasing library budgets and an institu- tion’s demand for a return on its investments in libraries.44 LA further advances these goals. Information literacy instruction has seen an increased move toward outcomes-based as- sessment in the past two decades, which has normalized the capture of detailed and individual quantitative data for many academic librarians. The Association of College and Research Libraries (ACRL) Information Literacy Competency Standards, adopted in 2000, encouraged a granular approach to data gathering about students and the analysis practices that support LA.45 These standards specifically called for 22 performance indicators, each with multiple outcomes. One of the goals indicated by the implementation of the standards is to identify how information literacy would “enhance the institution’s effectiveness.”46 The ACRL Frame- work for Information Literacy in Higher Education, which was adopted in 2016 and supplanted the Standards, does not explicitly focus on assessment in the same quantitative way but does emphasize measuring impact.47 ACRL’s long-standing Value of Academic Libraries (VAL) initiative has also expanded library assessment and evaluation practices.48 The initial report, Value of Academic Libraries: A Comprehensive Research Review and Report, asserted that academic libraries need to “document and articulate”49 how a library contributes to its institution’s missions and goals, primarily due to ever-increasing accountability measures. The VAL report recognizes in the foreword that measuring library efficacy grates on some who feel such efforts signal a capitulation to neoliberal interests; additionally, the report acknowledged that attempts to quantify library efforts seem like further homogenization of academia. About these critiques, ACRL leadership commented that “this critical perspective [...] seems impractical, given the realities we face today in our institutions.”50 Funding agencies and many academic libraries agreed. The VAL 574 College & Research Libraries April 2020 initiative has received several national grants in support of its work; notable related initiatives have emerged, including the “Assessment in Action: Academic Libraries and Student Suc- cess” program.51 Furthermore, VAL leaders have developed numerous reports demonstrating the success of and continued need for the research agenda.52 To date, hundreds of scholarly artifacts have been published and disseminated as a result of the VAL initiative.53 More recently, library LA literature is explicitly aligning with the methods and goals that higher education LA advocates espouse.54 Megan Oakleaf,55 in particular, has been an outspo- ken advocate of LA, highlighting how “many librarians have embraced the use of assessment and research to explore the links between student library interactions and student learning and success measures.”56 This argument moves beyond the documentation and assessment of library efforts to combining library data with identifiable student data from other sources to seek potential correlative trends. OCLC’s research with higher education administrators signaled that there is an ever-increasing interest in gathering identifiable student data.57 Fur- thermore, while those administrators did not explicitly identify data from the library as of interest, library LA advocates have mentioned it as a particular area of importance. Library participation in institutional LA initiatives to date has been limited in part because library data are siloed from the rest of the campus.58 LA advocates argue that library data should be extracted from various library systems and integrated with centralized systems for cross-institution access and analysis, which some institutions have pursued.59 For example, the universities of Wollongong and Minnesota include sharing individual students’ library usage information with instructors/advisors to encourage them to nudge those students to- ward greater library use.60 Library LA studies are increasingly possible due to a growing amount of accessible data, in part because data exhaust and metadata are inherent to library systems’ main function. Library LA studies demonstrate that analyzing data for secondary purposes (that is to say, not for their designed purposes) provides several analytical opportunities, as does the combina- tion of library data sources with campus-captured student information. Library LA studies harvest data from existing library systems such as integrated library systems (ILS) and EZ- proxy logs, using spreadsheet or statistical software to run analyses (examples: Excel, SPSS, Tableau). Checkout counts are baseline usage statistics in LA studies, including interlibrary loans,61 equipment checkouts,62 and books. Additional facilities data are available, such as card-swipe entries into the library,63 library computer logins,64 and study room reservations.65 Proxy servers provide data about e-resource use.66 Library LA additionally provide new possibilities to assess library interactions with students. Studies quantified transactions between students and library staff at the following service points: face-to-face reference services,67 asynchronous virtual reference services such as through e-mail,68 synchronous virtual reference services such as instant messaging,69 and instruction sessions.70 Others are mining text from chat reference transcripts.71 The scope of data collection varies from project to project; for example, course-based instruction allows for the aggregate analysis of students in an entire class section rather than individual-level analysis.72 Some libraries purchase Springshare’s LibAnalytics,73 OrangeBoy,74 Gale Analytics,75 WhoFi,76 and OCLC’s Wise to support analytic efforts.77 All these systems represent newer commercial library analytics tools, though vendors market these primarily to public libraries. The majority of library LA studies also look to institutional sources for student data. Grade point averages (GPAs), obtained from an institutional research office, are one of the Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 575 most common metrics of student success. Kogut found that more than 40 percent of studies evaluated GPA when examining academic library contributions to undergraduate academic success,78 which is consistent with trends in broader LA practices.79 Occasionally, studies evaluated individual assignments or examined course grades instead of overall GPA.80 Other library LA studies include data gathered from student participation in first-year experience programs81 and uses of tutoring services.82 To this is added data gathered outside classroom environments, such as four-year graduation rates83 and year-to-year retention.84 Additional information often obtained from institutional sources includes demographic information such as age, gender, and ethnicity to create subgroups for comparison.85 A small number of library studies examine sensitive demographic data like socioeconomic status86 and disability status.87 Other studies captured a student’s standardized test scores,88 major,89 and various statuses like on/off campus residence status,90 full/part-time enrollment status,91 and first-generation status.92 The scope of demographic data also fluctuates. Some articles amass a large amount of demographic information,93 while others collect relatively few data or only look at demographics in aggregate.94 The research methods by which libraries conduct LA studies are varied. However, the majority of studies correlate the use of library services with measures of student success or make means-based comparisons. These studies are most frequently quantitative, though some use a mixed-methods approach.95 One form of analysis is propensity score matching, where target students are paired and compared to students with similar characteristics from a control group.96 Most studies run analyses at the individual student level, though a small portion of studies perform analytics on aggregated data such as from IPEDS or library as- sociation statistics.97 Library Learning Analytics, Ethics, and Privacy Issues As academic libraries engage in learning analytics, questions arise surrounding the ethics of capturing, retaining, and using these data as well as concerns about user privacy. Student privacy issues ebb and flow according to new sociotechnical challenges, which are exemplified by privacy issues brought to the fore by LA’s rise. Higher education is struggling to achieve a privacy balance that can both protect students and, simultaneously, lead to new data-based insights. The same holds true for academic libraries. Library ethics may form a foundation from which to examine these new challenges. Professional librarians address privacy and related ethical issues in part by reviewing guidelines in ethics codes, such as those by the American Library Association (ALA),98 the In- ternational Federation of Library Associations (IFLA),99 and the National Information Standards Organization (NISO).100 Each organization, to various degrees, emphasizes the importance and roles of privacy and confidentiality in intellectual processes associated with library use. Where the ALA Code of Ethics is concerned, Principles 1 through 4 and 6 are particularly relevant to library LA projects, as they focus on equitable access, intellectual freedom, privacy, confidentiality, rights of information users and rights holders, and not advancing “private interests at the expense of library users, colleagues, or our employing institutions.”101 Simi- larly, the Library Bill of Rights’ seventh point explicitly calls out a user’s “right to privacy and confidentiality in their library use.”102 Similarly, IFLA’s Statement on Privacy in the Library Environment states that “library and information services should reject electronic surveillance and any type of illegitimate monitoring or collection of users’ personal data or information 576 College & Research Libraries April 2020 behavior that would compromise their privacy and affect their rights to seek, receive, and impart information.”103 Notably, NISO’s Consensus Principles on Users’ Digital Privacy in Library, Publisher, and Software-Provider Systems is intended as “a starting point” for pragmatic discussions on user privacy in library systems and electronic resources.104 In the past, ALA has historically put the onus on libraries to protect user privacy; but, more recently, it has turned to NISO’s principles, which takes a more balanced approach. The principles express the joint privacy obligations that libraries and third parties share. Its principles state that “[l]ibraries, publishers, and software providers have a shared obligation to foster a digital environment that respects library users’ privacy as they search, discover, and use those resources and services.”105 What is lacking from ALA, IFLA, and NISO’s documents is an explicit recognition that library data can be and are often used for research and organizational improvement purposes. Efforts to fill this gap have arisen from ALA itself,106 the Digital Library Federation,107 LITA,108 and the Library Freedom Project.109 Furthermore, the Institute of Museum and Library Services has distributed a significant amount of grant funding to support research into privacy and ethics issues associated with LA; see appendix B for a list of grants. Outside of libraries, researchers and institutions have developed guiding principles for practice, policy, and technological design. The Asilomar conferences brought together researchers, technologists, and ethicists to develop “a framework to inform decisions about appropriate use of data and technology in learning research for higher education.”110 Other researchers, like Slade and Prinsloo, Sclater, and Pardo and Siemens, put forth their recom- mendations for principled practice.111 Some universities have developed institutional guidelines. The Open University was among the first to develop principles explicitly addressing privacy and LA.112 More recently, respective faculty at the University of California and the University of Hawaii passed resolu- tions around student data.113 Both of these institutions’ stated principles address data access, control, and ownership; freedom of expression; ethical uses; and transparency, among other things. Policies from these and other universities are available in appendix C. Despite these efforts, there is very little evidence that libraries have begun to specifically address LA in local policies. The only data suggesting a policy reaction to LA by libraries comes from Perry et al., whose survey of institutional members of the Association of Research Libraries found that “[m]ost respondents indicated that LA has not caused changes in their privacy policies. Only one respondent indicated that they created a ‘Responsible Use of Library Data’ statement in alignment with their learning analytics project.”114 Since there is a notable gap in library policy and ethics vis-à-vis LA, several library-specific projects have sought to work through LA privacy ethics using a critical lens. Asher addresses the potential and demonstrated dangers of increasing data access, storage, and analysis of library users in higher-education environments.115 Building on Asher’s work, Jones and Salo analyzed how these issues clash with ALA’s Code of Ethics, arguing ways in which LA projects may inhibit intellectual freedom and privacy.116 In a recent piece, Jones, Rubel, and LeClere argue institutions are beholden to their students as “information fiduciaries,” that they have a moral obligation to use LA to benefit students and respect the trust students place in their university.117 Given these known ethical problems and quandaries related to LA, Asher et al. argue that librarians must update their sense of threats to user privacy by taking into account algorith- Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 577 mic bias, reidentification, widespread data trading and hoarding, and surveillance creep.118 Briney homes in on research ethics and LA projects, noting that, even though institutional review boards (IRB) are responsible for identifying ethical issues, many miss violations of library privacy ethics. This situation is due to differences in disciplinary ethics and the fact that studies may be marked exempt from review. For instance, for those projects that do not include obvious demographic data or are internal assessment and evaluation practices, they fail to trigger reviews.119 It is unclear from the literature if any academic libraries have established clear internal review processes for addressing data privacy concerns. Practical Library Ethical Privacy Issues Practical ethical privacy issues appear in the library LA literature because of the intersection of data-handling issues and the ethical implications of data practices. These may include in- formed consent, data collection, data retention, and ethical review, among others. At present, no professional library association has provided comprehensive guidance or best practices for ethical data handling, though the Technologies of Surveillance Working Group of the Digital Library Federation is drafting guidelines and has created a glossary explaining research ethics intersecting with the use of library user data.120 A major ethical issue of data collection is the use of informed consent.121 In some instances, students may have the option to consent to or explicitly opt into LA practices and research projects. Alternately, LA initiatives may automatically include student data, putting the onus on students to opt out of data collection and analysis procedures to maintain their privacy. Consent is sometimes uneven across a project; it does not follow that, simply because students opt in to a focus group, the data representing those students should be included in other LA projects.122 For instance, Stone and Ramsden performed quantitative analytics without a re- ported consent process but then conducted a qualitative analysis via a focus group for which they obtained explicit consent.123 Very little information is available on opt-out procedures in academic libraries. One study was nearly unique in its description of how the studied library provided an, admittedly buried, opt-out mechanism.124 A review of data practices in library LA found that around 10 percent of published studies in this area explicitly mention consent, opt- in, or opt-out.125 This finding is similar to those from Perry et al., which found that only about 40 percent of respondent libraries inform students of analytics, and of those only 60 percent (approximately 20% overall) gave students the option to opt in or out of analytic projects.126 From this report, it seems that libraries are making most decisions about LA in the complete absence of student consent. Data selection and retention remains a thorny practical problem for library LA practi- tioners. Arguments exist suggesting that libraries—and their institutions—should collect as much data as they can at the most granular level. Doing so opens up analytical avenues that would be prematurely shut down by collecting fewer data. Naturally, these decisions intersect with wider student privacy concerns. For instance, tracking that students checked out books,127 versus the number of books checked out,128 versus the type of books checked out129 leads to different student privacy implications. Similarly, electronic database usage can be measured as used/not used,130 the number of sessions,131 or even as timed increments within a specific database.132 There is no consistency in data granularity across the library LA literature and, as yet, no best practices or standards. 578 College & Research Libraries April 2020 Where student data are not properly secured or are retained indefinitely instead of de- leted, the potential for leakage, misuse, or a breach of privacy increases. Unfortunately, the library LA literature indicates that libraries keep data for long durations with little known about how data are secured. Briney found that only about 10 percent of published library LA studies mention data security practices.133 These findings are echoed in Perry et al., where just over half of the survey respondents claim that they have security protections (physical and/ or digital) on LA data, and only a third secure data during transit. Additionally, fewer than half of respondents had a retention schedule for LA data, and a full 20 percent planned to hold onto these data “indefinitely.”134 Yoose provides a singular positive example of working through the tradeoffs between collecting adequate data for analysis and maintaining data securely using practices such as data minimization, de-identification, and limiting access to sensitive data.135 Overall, little is known about data security in library LA, with sufficient evidence of questionable data practices indicating that libraries must attend more carefully to these practical concerns. Best practices for dealing with sensitive data, prescribed limited data collection, robust security, and finite data retention and deletion would help in this area.136 There is notable variance in how librarians participating in LA projects approach internal or external ethical reviews of their work, whether from the IRB or another party. Academic libraries frequently reported in their research that they only sought IRB review if planning formal publication and—as previously discussed—that IRBs may report “de-identified” data as exempt or as nonhuman subject data, requiring no formal review and no informed consent procedures.137 Additionally, for the United States, quality improvement projects may not be considered research requiring human subject protection,138 and the Family Educational Rights and Privacy Act (FERPA) allows institutional staff to access data when they have a “legitimate educational interest.”139 FERPA also includes a broad exemption for institutionally affiliated third-party actors, which may include library vendors such as Springshare or the pervasive use of Google Analytics on library websites.140 Because of these exceptions, LA studies may not ever undergo IRB review or receive any scrutiny around the standards of “beneficence” (respecting research subjects and protecting them from harm) and “justice” (ensuring that benefits and harms of research are equitably distributed). Reidentification remains a significant issue as data sets increase in size and are combined with other data. This commonly cited issue outside of the context of higher education is just as applicable to colleges and universities and their libraries. The literature often describes student data used for LA purposes as anonymous (and thus no longer the private information of a single individual); yet, so-called anonymization procedures are not sufficient.141 Anony- mization procedures are often described only superficially, with the argument made that the removal of direct identifiers (such as names and identification numbers) is sufficient while ignoring the fact that other identifiers that include educational information can easily allow for reidentification.142 Several publications also failed to fuzz, combine, or redact populations too small to protect individual subjects (“small n’s”), heightening those subjects’ vulnerability to reidentification. At least one study reported an easily reidentifiable outlier—an 83-year-old person—making that individual’s use of the library no longer private.143 Recommendations Ethical transgressions do not arise solely due to a lack of concern for the harms LA may create. The aforementioned issues are likely due to accessibility to larger swaths of data and librarians Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 579 needing to reskill and reconsider data practices in light of new methodologies. As van Dijk144 and Kitchin145 argue, the datafication of human life has created a new scientific research paradigm with accompanying changes in the study of value, knowledge, and being. The current state of the literature reveals skill gaps in the profession that require addressing to alleviate the ethical issues around LA. Once the profession addresses these gaps, some of the privacy concerns may decrease. Institutional dynamics complicate the ethics of LA. Institutional administrators may push to override the library profession’s ethical principles in an effort to pursue data-mining practices and demonstrate oft-cited outcomes such as increases in retention and graduation. Librarians may feel an obligation to collect as much data as possible, especially as a political tool to curry more favor when budget requests are due; Jones has documented such pressures in interviews with librarians.146 How to navigate these tricky political conditions will depend on local conditions, but there exist a few guiding principles for moving forward. First, ethical positions should be clear to librarians and made clear to those pressuring them to pursue LA. Librarians should reflect on their ethics and discuss their positions with their peers to enter into campus LA discussions with a clear sense of direction; see the “Action Handbook” by Young, Clark, Mannheimer, and Hinchliffe for more ideas in this area.147 Second, while some interpret ethical principles as black/white directives or bright lines that they should not cross, they should instead be pragmatically used as guideposts that point to optimal outcomes and need interpretation at the local level; doing so will lead to a more fruitful discussion with administrators and oth- ers pursuing an LA agenda. Within academic libraries, practitioners and researchers have provided examples of how to balance ethical concerns with practical data needs.148 Librarians are not rigorously prepared to conduct ethical qualitative or quantitative re- search.149 Previous research has found that there is no relationship between taking a research methods course in a library and information science graduate program and feeling prepared to conduct research as a professional librarian.150 Librarians are likely to employ self- and continuing education to fill the research methods gap in their formal education.151 For LA specifically, more than 15 percent of survey respondents in Perry et al. said that staff receive no training at all for analytics projects. Even where training is occurring, it is not uniform or thorough and most often consisted of IRB and FERPA training.152 LA require practitioners to increase their methodological capacities if they are to pursue analytic-based practices and conduct LA research in rigorous and valid ways. It does no good to the profession or the students it serves by taking up LA to appease administrators—and using precious resources and time—only to do it poorly. Moreover, as Robertshaw and Asher highlight, applying analytic practices incorrectly and drawing conclusions without appropri- ate statistical support is an unethical practice.153 A complicating factor for librarians is that institutions expect professional librarians to conduct research yet do not provide sufficient support. Hoffman, Berg, and Koufogiannakis found that “Institutional Structures and Supports” were one of three broad categories that contribute to research productivity.154 Intra-library mentoring also contributes to research success, but not all libraries offer formal mentoring;155 libraries that grant tenure are more likely to provide this type of support.156 Lacking resources like time, institutional support, or professional mentorship may cause librarians to 1) not be prepared methodologically, 2) not fully consider the ethical implications of their study design and implementation, and/or 3) not have time to conduct their research in an ethically rigorous way. 580 College & Research Libraries April 2020 Librarians, and particularly library administrators, should push for institutional support from their administrative peers and colleagues in other offices. Ethically pursuing LA requires time and effort to establish goals, reflect on practices, improve infrastructures, and develop strategies. Training specifically for librarians in this area is beginning to be available from IMLS-funded projects and professional organizations.157 Moreover, librarians should seek the support of those whose expertise is complementary to their own, such as seeking input from statistical experts, getting guidance from institutional researchers, and conferring with philosophy professors to help them navigate thorny ethical issues. These collaborative ap- proaches have the potential to lead to more fully informed and ethically sound LA practices. There exists an unquestionable power imbalance between students and institutional actors, including librarians. Institutions can analytically dig into student lives by accessing, analyz- ing, and acting on granular student data. Jones and McCoy argue that the power disparity enables institutions to render students as data subjects whose lives can be molded according to the interests of educational-technology designers and institutions.158 Rubel and Jones caution that institutional interests should not be conflated with student interests, noting that, while powerful administrators may want to increase retention and graduation rates, the question remains whether the analytical means employed to achieve those ends are justifiable.159 Whose interests are served by LA is a paramount question. Where researchers asked students about their perceptions of learning analytics, their positive views partially depended on an ability to manage their privacy.160 Students may not feel that they can freely opt out of analytics without damaging their academic success, even when they are informed about data practices. Inside the library, this is likely to add to students’ existing library anxiety,161 and it may abruptly and negatively affect the way students view the library as a safe intellectual space. If a power balance is to occur when librarians pursue LA, they are obligated to work hand-in-hand with students; engaging a student advisory board may resolve this concern. Conclusion As LA initiatives continue to evolve, librarians will need to understand the changes in the field and the potential impact on local and national practice. Privacy considerations will continue to emerge as potential and actual threats to student privacy arise.162 Library ethics do not directly inform the Common Rule and ethical interpretations by IRBs, so such things cannot guide library LA projects as well as librarians might wish. While many librarians are undertaking and publishing about LA initiatives, current research has identified several gaps in practice, including the need for adoption of sensitive data-management best practices and the need for more training and interdisciplinary collaboration to improve methodological and ethical practices. Additionally, libraries must integrate student participation to identify their prefer- ences and needs. Further research is also warranted to understand LA projects and practices at nonresearch libraries and academic libraries outside the United States. Academic libraries’ goal should be to balance the potential rewards from LA practices with the risks these practices pose to users. To do this successfully, librarians first need to know which practices endanger user privacy so clearly and dangerously that they are never allowable under any circumstances. Next, librarians need guidelines for balancing concerns with benefits appropriately. Finally, librarians need to ensure that their data-collection and data-management practices accord responsibly with best practices. By identifying and understanding these risks, librarians may begin to approach LA initiatives from an informed and critical perspective to best serve their students. Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 581 APPENDIX A. Abbreviations ACRL = Association of College and Research Libraries ALI = Academic Library Impact ALA = American Library Association ARL = Association of Research Libraries CAO = Chief academic officer GPA = Grade point average IFLA = International Federation of Library Associations ILS = Integrated library system IRB = Institutional review board LA = Learning analytics LMS = Learning management system NISO = National Information Standards Organization OCLC = Online Computer Library Center SIS = Student information system UCDM = Unizin Common Data Model VAL = Value of Academic Libraries 582 College & Research Libraries April 2020 APPENDIX B. Institute of Museum and Library Services Grants on Learning Analytics Title Amount URL Prioritizing Privacy: Training to Improve Practice in Library Analytics Projects $249,198 https://imls.gov/grants/awarded/re-18-19-0014-19 Student Privacy in the Datafied Classroom: Identifying Instructional Student Privacy Practices to Facilitate Librarian/ Faculty Conversations $306,682 https://imls.gov/grants/awarded/lg-18-19-0032-19 Getting to Know Their Data Doubles $514,484 https://www.imls.gov/grants/awarded/lg-96-18- 0044-18 Library Values & Privacy in Our National Digital Strategies: Field Guides, Convenings, and Conversations $90,150 https://www.imls.gov/grants/awarded/lg-73-17- 0062-17 A National Forum on Web Privacy and Web Analytics $100,000 https://www.imls.gov/grants/awarded/lg-73-18- 0100-18 Library Integration in Institutional Learning Analytics (LIILA) $99,876 https://www.imls.gov/grants/awarded/lg-98-17- 0019-17 Connecting Libraries and Learning Analytics for Student Success (CLLASS) $50,000 https://www.imls.gov/grants/awarded/lg-97-18- 0209-18 Scaling Digital Privacy and Data Literacy in Libraries $240,729 https://imls.cg-devel.com/grants/awarded/re-06- 15-0050-15 Privacy in Libraries $249,504 https://www.imls.gov/grants/awarded/re-95-17- 0076-17 https://imls.gov/grants/awarded/re-18-19-0014-19 https://imls.gov/grants/awarded/lg-18-19-0032-19 https://www.imls.gov/grants/awarded/lg-96-18-0044-18 https://www.imls.gov/grants/awarded/lg-96-18-0044-18 https://www.imls.gov/grants/awarded/lg-73-17-0062-17 https://www.imls.gov/grants/awarded/lg-73-17-0062-17 https://www.imls.gov/grants/awarded/lg-73-18-0100-18 https://www.imls.gov/grants/awarded/lg-73-18-0100-18 https://www.imls.gov/grants/awarded/lg-98-17-0019-17 https://www.imls.gov/grants/awarded/lg-98-17-0019-17 https://www.imls.gov/grants/awarded/lg-97-18-0209-18 https://www.imls.gov/grants/awarded/lg-97-18-0209-18 https://imls.cg-devel.com/grants/awarded/re-06-15-0050-15 https://imls.cg-devel.com/grants/awarded/re-06-15-0050-15 https://www.imls.gov/grants/awarded/re-95-17-0076-17 https://www.imls.gov/grants/awarded/re-95-17-0076-17 Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 583 APPENDIX C. Learning Analytics Codes of Ethics and Institutional Policies Name URL The Asilomar Convention for Learning Research in Higher Education http://asilomar-highered.info Asilomar II: Student Data and Records in the Digital Era https://asilomar.sites.stanford.edu Jisc: Code of Practice for Learning Analytics https://www.jisc.ac.uk/guides/code-of-practice-for- learning-analytics Charles Sturt University: Learning Analytics Code of Practice http://www.csu.edu.au/__data/assets/ pdf_file/0007/2160484/2016_CSU_ LearningAnalyticsCodePractice.pdf The Open University: Ethical Use of Student Data for Learning Analytics https://help.open.ac.uk/documents/policies/ethical- use-of-student-data University of California: Learning Data Privacy Principles https://www.ets.berkeley.edu/sites/default/files/ general/university_of_california_learning_data_ privacy_principles_handout.pdf University of Hawaii: Resolution Supporting Learner Data Privacy Principles and Practices https://docs.google.com/document/d/1bqJTZ4tCK3S FdsS4rXVNK5xlS87g0ajfE3EYk3u0Zh8/edit Notes 1. Leah P. Macfadyen et al., “Embracing Big Data in Complex Educational Systems: The Learning Analytics Imperative and the Policy Challenge,” Research and Practice in Assessment 9, no. 2014 (n.d.), available online at www.rpajournal.com/embracing-big-data-in-complex-educational-systems-the-learning-analytics-imperative- and-the-policy-challenge/ [accessed 3 March 2020]. 2. Matt Asay, “Who’s Getting Rich In The Big Data Gold Rush?” ReadWrite (blog, Nov.11, 2013), available online at https://readwrite.com/2013/11/11/whos-getting-rich-in-the-big-data-gold-rush/ [accessed 3 March 2020]; David Steinberg, “Unlocking the Big Data Goldmine for SMBs,” WIRED (Apr. 25, 2013), available online at https://www.wired.com/insights/2013/04/unlocking-the-big-data-goldmine-for-smbs/ [accessed 3 March 2020]; Brad Peters, “The Big Data Gold Rush,” Forbes (June 21, 2012), available online at https://www.forbes.com/sites/ bradpeters/2012/06/21/the-big-data-gold-rush/ [accessed 3 March 2020]. 3. Edwin Lane, “Businesses Are Using Big Data to Outsmart Rivals,” CNN, available online at https://www. cnn.com/2014/10/13/business/moneyball-businesses-outsmarting-rivals/index.html [accessed 8 July 2019]; Viktor Mayer-Schönberger and Kenneth Cukier, Big Data: A Revolution That Will Transform How We Live, Work, and Think, Reprint edition (Boston, MA: Eamon Dolan/Mariner Books, 2014); Perry Rotella, “Is Data the New Oil?” Forbes (Apr. 2, 2012), available online at https://www.forbes.com/sites/perryrotella/2012/04/02/is-data-the-new-oil/ [ac- cessed 3 March 2020]; Simon Moss and Pneuron Corporation, “Big Data: New Oil or Snake Oil?” WIRED (Oct. 23, 2014), available online at https://www.wired.com/insights/2014/10/big-data-new-oil-or-snake-oil/ [accessed 3 March 2020]. 4. George Siemens, “Learning Analytics: Envisioning a Research Discipline and a Domain of Practice,” in Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (New York, NY: ACM, 2012), 4–8, https://doi.org/10.1145/2330601.2330605. 5. Angela van Barneveld, Kimberly E. Arnold, and John P. Campbell, “Analytics in Higher Education: Es- tablishing a Common Language,” Educause 1, no. 1 (2012): 1–11. 6. L. Johnson et al., The 2010 Horizon Report (Austin, TX: New Media Consortium, 2010), available online at https://eric.ed.gov/?id=ED510220 [accessed 3 March 2020. 7. Association of College and Research Libraries (ACRL), Academic Library Impact: Improving Practice and Es- sential Areas to Research, prepared by Lynn Silipigni Connaway, William Harvey, Vanessa Kitzie, and Stephanie Mikitish of OCLC Research (Chicago, IL: ACRL, 2017), available online at www.ala.org/acrl/sites/ala.org.acrl/ http://asilomar-highered.info https://asilomar.sites.stanford.edu https://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics https://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics http://www.csu.edu.au/__data/assets/pdf_file/0007/2160484/2016_CSU_LearningAnalyticsCodePractice.pdf http://www.csu.edu.au/__data/assets/pdf_file/0007/2160484/2016_CSU_LearningAnalyticsCodePractice.pdf http://www.csu.edu.au/__data/assets/pdf_file/0007/2160484/2016_CSU_LearningAnalyticsCodePractice.pdf https://help.open.ac.uk/documents/policies/ethical-use-of-student-data https://help.open.ac.uk/documents/policies/ethical-use-of-student-data https://www.ets.berkeley.edu/sites/default/files/general/university_of_california_learning_data_privacy_principles_handout.pdf https://www.ets.berkeley.edu/sites/default/files/general/university_of_california_learning_data_privacy_principles_handout.pdf https://www.ets.berkeley.edu/sites/default/files/general/university_of_california_learning_data_privacy_principles_handout.pdf https://docs.google.com/document/d/1bqJTZ4tCK3SFdsS4rXVNK5xlS87g0ajfE3EYk3u0Zh8/edit https://docs.google.com/document/d/1bqJTZ4tCK3SFdsS4rXVNK5xlS87g0ajfE3EYk3u0Zh8/edit http://www.rpajournal.com/embracing-big-data-in-complex-educational-systems-the-learning-analytics-imperative-and-the-policy-challenge/ http://www.rpajournal.com/embracing-big-data-in-complex-educational-systems-the-learning-analytics-imperative-and-the-policy-challenge/ https://readwrite.com/2013/11/11/whos-getting-rich-in-the-big-data-gold-rush/ https://www.wired.com/insights/2013/04/unlocking-the-big-data-goldmine-for-smbs/ https://www.forbes.com/sites/bradpeters/2012/06/21/the-big-data-gold-rush/ https://www.forbes.com/sites/bradpeters/2012/06/21/the-big-data-gold-rush/ https://www.cnn.com/2014/10/13/business/moneyball-businesses-outsmarting-rivals/index.html https://www.cnn.com/2014/10/13/business/moneyball-businesses-outsmarting-rivals/index.html https://www.forbes.com/sites/perryrotella/2012/04/02/is-data-the-new-oil/ https://www.wired.com/insights/2014/10/big-data-new-oil-or-snake-oil/ https://doi.org/10.1145/2330601.2330605 https://eric.ed.gov/?id=ED510220 http://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/whitepapers/academiclib.pdf 584 College & Research Libraries April 2020 files/content/publications/whitepapers/academiclib.pdf [accessed 3 March 2020]. 8. Kara Malenfant, “Assessment in Action: Academic Libraries and Student Success” (text, Association of College & Research Libraries, Oct. 9, 2012), available online at www.ala.org/acrl/AiA [accessed 3 March 2020]; Association of College and Research Libraries, “Assessment in Action Final Narrative” (Sept. 30, 2015), available online at www.ala.org/acrl/sites/ala.org.acrl/files/content/AiA%20final%20report%20to%20IMLS%20FINAL%20 no%20cover.pdf [accessed 3 March 2020]; Association of College and Research Libraries (ACRL), The Value of Academic Libraries: A Comprehensive Research Review and Report, prepared by Megan Oakleaf (Chicago, IL: ACRL, 2010), available online at www.acrl.ala.org/value [accessed 3 March 2020]. 9. Kyle M.L. Jones and Dorothea Salo, “Learning Analytics and the Academic Library: Professional Ethics Commitments at a Crossroads,” College & Research Libraries 17 (2017): 304–23; Karen P. Nicholson, Nicole Pagowsky, and Maura Seale, “Just-in-Time or Just-in-Case? Time, Learning Analytics, and the Academic Library,” Library Trends 68 (2019): 54–75; Tami Oliphant and Michael R. Brundin, “Conflicting Values: An Exploration of the Ten- sions between Learning Analytics and Academic Librarianship,” Library Trends 68 (2019): 5–23. 10. Thomas H. Davenport, Big Data at Work: Dispelling the Myths, Uncovering the Opportunities (Boston, MA: Harvard Business Review Press, 2014). 11. Jacqueleen A. Reyes, “The Skinny on Big Data in Education: Learning Analytics Simplified,” TechTrends 59, no. 2 (2015): 75–80. 12. van Barneveld, Arnold, and Campbell, “Analytics in Higher Education”; Kyle M.L. Jones, “All the Data We Can Get: A Contextual Study of Learning Analytics and Student Privacy” (PhD diss., The Univer- sity of Wisconsin-Madison, 2015), available online at https://search.proquest.com/docview/1751011265/abstract/ F69EA27D0E3D4A43PQ/1.van [accessed 3 March 2020]. 13. J.P. Campbell and D.G. Oblinger, “Academic Analytics” (Oct. 2007), available online at www.educause. edu/library/resources/academic-analytics [accessed 3 March 2020]. 14. Abelardo Pardo and Carlos Delgado Kloos, “Stepping Out of the Box: Towards Analytics Outside the Learning Management System,” in Proceedings of the 1st International Conference on Learning Analytics and Knowl- edge—LAK ’11 (the 1st International Conference, Banff, Alberta, Canada: ACM Press, 2011), 163, https://doi. org/10.1145/2090116.2090142.pardo. 15. Unizin Consortium, “Unizin Common Data Model,” available online at https://docs.udp.unizin.org/ [ac- cessed 3 March 2020]. 16. Unizin Consortium, “About,” Unizin (blog), available online at https://unizin.org/about/ [accessed 8 July 2019]. 17. Kimberly E. Arnold, “Signals: Applying Academic Analytics,” EDUCAUSE Quarterly 33, no. 1 (2010), available online at https://er.educause.edu/articles/2010/3/signals-applying-academic-analytics [accessed 3 March 2020]. 18. Kyle M.L. Jones and Chase McCoy, “Reconsidering Data in Learning Analytics: Opportunities for Criti- cal Research Using a Documentation Studies Framework,” Learning, Media and Technology 44, no. 1 (January 2, 2019): 52–63, https://doi.org/10.1080/17439884.2018.1556216. 19. C. Carmean and P. Mizzi, “The Case for Nudge Analytics,” EDUCAUSE Quarterly 33, no. 4 (2010), avail- able online at www.educause.edu/ero/article/case-nudge-analytics [accessed 3 March 2020]; Jill Frankfort et al., “Analytics, Nudges, and Learner Persistence,” EDUCAUSE Review 19 (July 18, 2012), available online at https:// digitalcommons.tacoma.uw.edu/academic_affairs_pubs/19/ [accessed 3 March 2020]. 20. Marc Parry, “Big Data on Campus,” The New York Times, July 18, 2012, sec. Education Life, available online at https://www.nytimes.com/2012/07/22/education/edlife/colleges-awakening-to-the-opportunities-of-data-mining. html [accessed 3 March 2020]. 21. Stephen Aguilar, Steven Lonn, and Stephanie D. Teasley, “Perceptions and Use of an Early Warning System During a Higher Education Transition Program,” in Proceedings of the Fourth International Conference on Learning Analytics and Knowledge—LAK ’14 (New York, NY: ACM, 2014), 113–17, https://doi.org/10.1145/2567574.2567625; Stephanie Kraft-Terry and Cheri Kau, “Manageable Steps to Implementing Data-Informed Advising” (NACADA, 2016), available online at https://www.nacada.ksu.edu/Resources/Clearinghouse/View-Articles/Manageable- Steps-to-Implementing-Data-Informed-Advising.aspx [accessed 3 March 2020]. 22. Lindsay McKenzie, “In the News: Alexa, What’s the Deal with You, Anyway?” Education Next (Aug. 23, 2018), available online at https://www.educationnext.org/news-alexa-whats-deal-anyway/ [accessed 3 March 2020]. 23. Diamond Naga Siu, “Amazon Echo Dots Are Coming to Every St. Louis University Dorm Room,” Mash- able (Aug. 16, 2018), available online at https://mashable.com/article/amazon-alexa-dot-saint-louis-university/ [accessed 3 March 2020]. 24. Thomas Hobohm, “‘Alexa, Why Are You Here?” The Mercury (Apr. 29, 2019), available online at https:// http://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/whitepapers/academiclib.pdf http://www.ala.org/acrl/AiA http://www.ala.org/acrl/sites/ala.org.acrl/files/content/AiA%20final%20report%20to%20IMLS%20FINAL%20no%20cover.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/AiA%20final%20report%20to%20IMLS%20FINAL%20no%20cover.pdf http://www.acrl.ala.org/value https://search.proquest.com/docview/1751011265/abstract/F69EA27D0E3D4A43PQ/1.van https://search.proquest.com/docview/1751011265/abstract/F69EA27D0E3D4A43PQ/1.van http://www.educause.edu/library/resources/academic-analytics http://www.educause.edu/library/resources/academic-analytics https://doi.org/10.1145/2090116.2090142.pardo https://doi.org/10.1145/2090116.2090142.pardo https://docs.udp.unizin.org/ https://unizin.org/about/ https://er.educause.edu/articles/2010/3/signals-applying-academic-analytics https://doi.org/10.1080/17439884.2018.1556216 http://www.educause.edu/ero/article/case-nudge-analytics https://digitalcommons.tacoma.uw.edu/academic_affairs_pubs/19/ https://digitalcommons.tacoma.uw.edu/academic_affairs_pubs/19/ https://www.nytimes.com/2012/07/22/education/edlife/colleges-awakening-to-the-opportunities-of-data-mining.html https://www.nytimes.com/2012/07/22/education/edlife/colleges-awakening-to-the-opportunities-of-data-mining.html https://doi.org/10.1145/2567574.2567625 https://www.nacada.ksu.edu/Resources/Clearinghouse/View-Articles/Manageable-Steps-to-Implementing-Data-Informed-Advising.aspx https://www.nacada.ksu.edu/Resources/Clearinghouse/View-Articles/Manageable-Steps-to-Implementing-Data-Informed-Advising.aspx https://www.educationnext.org/news-alexa-whats-deal-anyway/ https://mashable.com/article/amazon-alexa-dot-saint-louis-university/ https://utdmercury.com/alexa-where-are-you-here/ Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 585 utdmercury.com/alexa-where-are-you-here/ [accessed 3 March 2020]. 25. Jessica Chasmar, “Oklahoma University Requires Freshmen to Wear Fitbit, Track 10K Steps per Day,” The Washington Times (Jan. 11, 2016), available online at https://www.washingtontimes.com/news/2016/jan/11/ oklahoma-university-requires-freshmen-to-wear-fitb/ [accessed 3 March 2020]; Steve Tally, “Students Can Study Their Study Habits on Apple Watch,” Purdue University News (Apr. 8, 2015), available online at https://www. purdue.edu/newsroom/releases/2015/Q2/students-can-study-their-study-habits-on-apple-watch.html [accessed 3 March 2020]. 26. Alexis Blue and University Communications, “Researcher Looks at ‘Digital Traces’ to Help Students,” UANews, available online at https://uanews.arizona.edu/story/researcher-looks-digital-traces-help-students [ac- cessed 12 August 2018]. 27. S. Buckingham Shum and R. Ferguson, “Social Learning Analytics,” Journal of Educational Technology & Society 15, no. 3 (2012): 3–26. 28. Aaron Smith and Monica Anderson, “Social Media Use 2018: Demographics and Statistics | Pew Research Center” (Mar. 1, 2018), available online at https://www.pewinternet.org/2018/03/01/social-media-use-in-2018/ [ac- cessed 3 March 2020]. 29. Jeremy Bauer-Wolf, “Georgia Tech’s Monitoring of Student’s Social Media Causes Concern,” Inside Higher Education (Dec. 21, 2017), available online at https://www.insidehighered.com/news/2017/12/21/georgia-techs- monitoring-students-social-media-causes-concern [accessed 3 March 2020]. 30. Ruth Serven, “With Safety in Mind, UVa Police Monitoring Social Media,” The Daily Progress (Feb. 11, 2018), available online at https://www.dailyprogress.com/news/local/with-safety-in-mind-uva-police-monitoring- social-media/article_94dde34a-0f8e-11e8-b8c0-cba42c10d40f.html [accessed 3 March 2020]. 31. X. Chen, M. Vorvoreanu, and K. Madhavan, “Mining Social Media Data for Understanding Students’ Learning Experiences,” IEEE Transactions on Learning Technologies 7, no. 3 (July 2014): 246–59, https://doi.org/10.1109/ TLT.2013.2296520. 32. Reyes, “The Skinny on Big Data in Education,” 76. 33. IMS Global Learning Consortium, “IMS Leadership Activities” (2018), available online at www.imsglobal. org/leadershipactivities [accessed 3 March 2020]. 34. PSLC DataShop, “About / Frequently Asked Questions (FAQ),” DataShop@CMU, available online at https:// pslcdatashop.web.cmu.edu/about/faq.html [accessed 25 February 2020]. 35. Malcolm Brown, “Learning Analytics: The Coming Third Wave” (EDUCAUSE Learning Initiative, Apr. 2011), available online at https://library.educause.edu/resources/2011/4/learning-analytics-the-coming-third-wave [accessed 3 March 2020]. 36. P. Long and G. Siemens, “Penetrating the Fog: Analytics in Learning and Education,” EDUCAUSE Review 2011(Sept./Oct. 2011): 30–40. 37. Colin Beer, Rolley Tickner, and David Jones, “Three Paths for Learning Analytics and Beyond: Moving from Rhetoric to Reality,” in Proceedings of the 31st Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2014), eds. Bronwyn Hegarty, Jenny McDonald, and Swee-Kin Loke (31st Australasian Society for Computers in Learning in Tertiary Education Conference [ASCILITE 2014]: Rhetoric and Reality: Critical Perspectives on Educational Technology, Sydney, Australia: Macquarie University, 2014), 242–50, available online at http://ascilite.org/conferences/dunedin2014/files/fullpapers/185-Beer.pdf [accessed 3 March 2020]. 38. Thaddieus W. Conner and Thomas M. Rabovsky, “Accountability, Affordability, Access: A Review of the Recent Trends in Higher Education Policy Research,” Policy Studies Journal 39, no. s1 (2011): 93–112, https://doi. org/10.1111/j.1541-0072.2010.00389_7.x.tha. 39. Paula Young Lee, “‘Drown the Bunnies’: Mount St. Mary’s President Fires Faculty for Backlash against His ‘Put a Glock to Their Heads’ Freshman Retention Plan,” Salon (Feb. 10, 2016), available online at https://www. salon.com/2016/02/10/drown_the_bunnies_mount_st_marys_president_fires_faculty_for_backlash_against_ his_put_a_glock_to_their_heads_freshman_retention_plan/ [accessed 3 March 2020]. 40. Dragan Gašević, Shane Dawson, and George Siemens, “Let’s Not Forget: Learning Analytics Are about Learning,” TechTrends 59, no. 1 (Dec. 2014): 64–71, https://doi.org/10.1007/s11528-014-0822-x. 41. Doug Lederman, “Advocates for Student Learning Assessment Say It’s Time for a Different Approach,” available online at https://www.insidehighered.com/news/2019/04/17/advocates-student-learning-assessment- say-its-time-different-approach [accessed 7 May 2019]; Olga Viberg et al., “The Current Landscape of Learning Analytics in Higher Education,” Computers in Human Behavior 89 (Dec. 1, 2018): 98–110, https://doi.org/10.1016/j. chb.2018.07.027. 42. Kyle M.L. Jones, “‘Just Because You Can Doesn’t Mean You Should’: Practitioner Perceptions of Learning Analytics Ethics,” portal: Libraries and the Academy 19, no. 3 (2019): 407–28, https://doi.org/10.1353/pla.2019.0025. https://utdmercury.com/alexa-where-are-you-here/ https://www.washingtontimes.com/news/2016/jan/11/oklahoma-university-requires-freshmen-to-wear-fitb/ https://www.washingtontimes.com/news/2016/jan/11/oklahoma-university-requires-freshmen-to-wear-fitb/ https://www.purdue.edu/newsroom/releases/2015/Q2/students-can-study-their-study-habits-on-apple-watch.html https://www.purdue.edu/newsroom/releases/2015/Q2/students-can-study-their-study-habits-on-apple-watch.html https://uanews.arizona.edu/story/researcher-looks-digital-traces-help-students https://www.pewinternet.org/2018/03/01/social-media-use-in-2018/ https://www.insidehighered.com/news/2017/12/21/georgia-techs-monitoring-students-social-media-causes-concern https://www.insidehighered.com/news/2017/12/21/georgia-techs-monitoring-students-social-media-causes-concern https://www.dailyprogress.com/news/local/with-safety-in-mind-uva-police-monitoring-social-media/article_94dde34a-0f8e-11e8-b8c0-cba42c10d40f.html https://www.dailyprogress.com/news/local/with-safety-in-mind-uva-police-monitoring-social-media/article_94dde34a-0f8e-11e8-b8c0-cba42c10d40f.html https://doi.org/10.1109/TLT.2013.2296520 https://doi.org/10.1109/TLT.2013.2296520 http://www.imsglobal.org/leadershipactivities http://www.imsglobal.org/leadershipactivities https://pslcdatashop.web.cmu.edu/about/faq.html https://pslcdatashop.web.cmu.edu/about/faq.html https://library.educause.edu/resources/2011/4/learning-analytics-the-coming-third-wave http://ascilite.org/conferences/dunedin2014/files/fullpapers/185-Beer.pdf https://doi.org/10.1111/j.1541-0072.2010.00389_7.x.tha https://doi.org/10.1111/j.1541-0072.2010.00389_7.x.tha https://www.salon.com/2016/02/10/drown_the_bunnies_mount_st_marys_president_fires_faculty_for_backlash_against_his_put_a_glock_to_their_heads_freshman_retention_plan/ https://www.salon.com/2016/02/10/drown_the_bunnies_mount_st_marys_president_fires_faculty_for_backlash_against_his_put_a_glock_to_their_heads_freshman_retention_plan/ https://www.salon.com/2016/02/10/drown_the_bunnies_mount_st_marys_president_fires_faculty_for_backlash_against_his_put_a_glock_to_their_heads_freshman_retention_plan/ https://doi.org/10.1007/s11528-014-0822-x https://www.insidehighered.com/news/2019/04/17/advocates-student-learning-assessment-say-its-time-different-approach https://www.insidehighered.com/news/2019/04/17/advocates-student-learning-assessment-say-its-time-different-approach https://doi.org/10.1016/j.chb.2018.07.027 https://doi.org/10.1016/j.chb.2018.07.027 https://doi.org/10.1353/pla.2019.0025 586 College & Research Libraries April 2020 43. Beverly P. Lynch et al., “Attitudes of Presidents and Provosts on the University Library,” College & Research Libraries 68, no. 3 (May 1, 2007), https://doi.org/10.5860/crl.68.3.213. 44. Adam Murray and Ashley Ireland, “Provosts’ Perceptions of Academic Library Value & Preferences for Communication: A National Study,” College & Research Libraries 79, no. 3 (2017), available online at http://crl.acrl. org/index.php/crl/article/view/16693 [accessed 3 March 2020]. 45. Association of College and Research Libraries, “Information Literacy Competency Standards for Higher Education” (Jan. 2000), available online at https://alair.ala.org/handle/11213/7668 [accessed 3 March 2020]. 46. Association of College and Research Libraries, “Information Literacy Competency Standards for Higher Education”, 7. 47. Association of College and Research Libraries (ACRL), Framework for Information Literacy for Higher Educa- tion (Chicago, IL: ACRL, 2015), available online at www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/infolit/ Framework_ILHE.pdf [accessed 3 March 2020]. 48. Oakleaf, The Value of Academic Libraries; Association of College and Research Libraries (ACRL), Connect, Collaborate, and Communicate: A Report from the Value of Academic Libraries Summits (Chicago, IL: ACRL, 2012). 49. Oakleaf, The Value of Academic Libraries, 6. 50. Oakleaf, The Value of Academic Libraries, 7. 51. Malenfant, “Assessment in Action.” 52. Karen Brown and Kara Malenfant, Academic Library Contributions to Student Success: Documented Practices from the Field (Chicago, IL: ACRL, 2015), available online at www.acrl.ala.org/value [accessed 3 March 2020]; Karen Brown and Kara J. Malenfant, Documented Library Contributions to Student Learning and Success: Building Evidence with Team-Based Assessment in Action Campus Projects (Chicago, IL: ACRL, 2016), available online at www.ala. org/acrl/sites/ala.org.acrl/files/content/issues/value/contributions_y2.pdf [accessed 3 March 2020]; Karen Brown and Kara J. Malenfant, Academic Library Impact on Student Learning and Success: Findings from Assessment in Action Team Projects (Chicago, IL: ACRL, 2017), available online at www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/ value/findings_y3.pdf [accessed 3 March 2020]. 53. “Assessment in Action Bibliography | ACRL Value of Academic Libraries,” available online at https:// www.acrl.ala.org/value/?page_id=980 [accessed 9 July 2019]; “Visualizing Academic Library Impact: The ACRL/ OCLC Literature Analysis Dashboard,” available online at http://experimental.worldcat.org/valresearch [accessed 9 July 2019]. 54. Megan Oakleaf and Martha Kyrillidou, “Revisiting the Academic Library Value Research Agenda: An Opportunity to Shape the Future,” Journal of Academic Librarianship 42, no. 6 (Nov. 1, 2016): 757–64, https://doi. org/10.1016/j.acalib.2016.10.005; Megan Oakleaf et al., “Academic Libraries & Institutional Learning Analytics: One Path to Integration,” Journal of Academic Librarianship 43, no. 5 (Sept. 1, 2017): 454–61, https://doi.org/10.1016/j. acalib.2017.08.008. 55. Megan Oakleaf, “The Library’s Contribution to Student Learning: Inspirations and Aspirations | Oakleaf | College & Research Libraries,” College & Research Libraries 76, no. 3 (2015), https://doi.org/10.5860/crl.76.3.353; M. Oakleaf, “Getting Ready & Getting Started: Academic Librarian Involvement in Institutional Learning Analyt- ics Initiatives,” Journal of Academic Librarianship 42, no. 4 (2016): 472–75, https://doi.org/10.1016/j.acalib.2016.05.013; Megan Oakleaf, “The Problems and Promise of Learning Analytics for Increasing and Demonstrating Library Value and Impact,” Information and Learning Science 119, no. 1/2 (Jan. 2018): 16–24, https://doi.org/10.1108/ILS-08- 2017-0080. 56. Oakleaf, “The Problems and Promise of Learning Analytics for Increasing and Demonstrating Library Value and Impact,” 17. 57. Connaway et al., Academic Library Impact. 58. Megan Oakleaf, “Library Integration in Institutional Learning Analytics,” EDUCAUSE (Nov. 15, 2018), available online at https://library.educause.edu/-/media/files/library/2018/11/liila.pdf [accessed 3 March 2020]. 59. Oakleaf et al., “Academic Libraries & Institutional Learning Analytics.” 60. Jan Fransen and Kate Peterson, “Graduate in Four Years? Yes, the Library Can Help with That!” in Li- brary Assessment Conference (2016), available online at http://old.libraryassessment.org/bm~doc/77-fransen-2016. pdf [accessed 3 March 2020]; Shane Nackerud et al., “Retention, Student Success and Academic Engagement at Minnesota (University of Minnesota),” in Library Analytics and Metrics: Using Data to Drive Decisions and Services, ed. Ben Showers (Facet Publishing, 2015), 58–66; Margie Jantti, “Libraries and Big Data: A New View on Impact and Affect,” in Quality and the Academic Library: Reviewing, Assessing and Enhancing Service Provision, ed. Jeremy Atkinson (Chandos Publishing, 2016), 267–74. 61. Mitchell Scott, “Interlibrary Loan Article Use and User GPA: Findings and Implications for Library Ser- vices,” Journal of Access Services 11, no. 4 (Oct. 2014): 229–38, https://doi.org/10.1080/15367967.2014.945116. 62. Lisa Massengale, Pattie Piotrowski, and Devin Savage, “Identifying and Articulating Library Connections https://doi.org/10.5860/crl.68.3.213 http://crl.acrl.org/index.php/crl/article/view/16693 http://crl.acrl.org/index.php/crl/article/view/16693 https://alair.ala.org/handle/11213/7668 http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/infolit/Framework_ILHE.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/infolit/Framework_ILHE.pdf http://www.acrl.ala.org/value http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/contributions_y2.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/contributions_y2.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/findings_y3.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/findings_y3.pdf https://www.acrl.ala.org/value/?page_id=980 https://www.acrl.ala.org/value/?page_id=980 http://experimental.worldcat.org/valresearch https://doi.org/10.1016/j.acalib.2016.10.005 https://doi.org/10.1016/j.acalib.2016.10.005 https://doi.org/10.1016/j.acalib.2017.08.008 https://doi.org/10.1016/j.acalib.2017.08.008 https://doi.org/10.5860/crl.76.3.353 https://doi.org/10.1016/j.acalib.2016.05.013 https://doi.org/10.1108/ILS-08-2017-0080 https://doi.org/10.1108/ILS-08-2017-0080 https://library.educause.edu/-/media/files/library/2018/11/liila.pdf http://old.libraryassessment.org/bm~doc/77-fransen-2016.pdf http://old.libraryassessment.org/bm~doc/77-fransen-2016.pdf https://doi.org/10.1080/15367967.2014.945116 Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 587 to Student Success,” College & Research Libraries 77, no. 2 (Mar. 2016): 227–35, https://doi.org/10.5860/crl.77.2.227. 63. Jennifer Link Jones, “Using Library Swipe-Card Data to Inform Decision Making,” University Library Faculty Presentations (2010), available online at http://scholarworks.gsu.edu/univ_lib_facpres/21 [accessed 3 March 2020]; Edith A. Scarletto, Kenneth J. Burhanna, and Elizabeth Richardson, “Wide Awake at 4AM: A Study of Late Night User Behavior, Perceptions and Performance at an Academic Library,” Journal of Academic Librarianship 39, no. 5 (Sept. 2013): 371–77, https://doi.org/10.1016/j.acalib.2013.02.006. 64. Shane Nackerud et al., “Analyzing Demographics: Assessing Library Use Across the Institution,” portal: Libraries and the Academy 13, no. 2 (Apr. 14, 2013): 131–45, https://doi.org/10.1353/pla.2013.0017. 65. Massengale, Piotrowski, and Savage, “Identifying and Articulating Library Connections to Student Suc- cess.” 66. Tiffany LeMaistre, “Cost Per User: Analyzing EZProxy Logs for Collection Development,” in Charleston Conference (Charleston, SC, 2015), available online at https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=177 3&context=charleston [accessed 3 March 2020]. 67. Massengale, Piotrowski, and Savage, “Identifying and Articulating Library Connections to Student Suc- cess.” 68. Angie Thorpe et al., “The Impact of the Academic Library on Student Success: Connecting the Dots,” portal: Libraries and the Academy 16, no. 2 (2016): 373–92. 69. Krista M. Soria et al., “Library Use and Undergraduate Student Outcomes: New Evidence for Students’ Retention and Academic Success,” portal: Libraries and the Academy 13, no. 2 (2013): 147–64, https://doi.org/10.1353/ pla.2013.0010. 70. Shun Han Rebekah Wong and Dianne Cmor, “Measuring Association between Library Instruction and Graduation GPA,” College & Research Libraries 72, no. 5 (Sept. 2011): 464–73, https://doi.org/10.5860/crl-151. 71. Pamela J. Howard, “Can Academic Library Instant Message Transcripts Provide Documentation of Un- dergraduate Student Success?” Journal of Web Librarianship 13, no. 1 (Feb. 4, 2019): 1–26, https://doi.org/10.1080/19 322909.2018.1555504. 72. Mary O’Kelly, “Academic Libraries and Student Retention: The Implications for Higher Education,” in Library Assessment Conference (2016), available online at https://scholarworks.gvsu.edu/library_proceedings/7/ [accessed 3 March 2020]. 73. “Springshare: The SaaS Platform for Libraries and Educational Institutions,” available online at https:// springshare.com/ [accessed 15 July 2019]. 74. OrangeBoy, Inc. (2017), available online at https://www.orangeboyinc.com/ [accessed 3 March 2020]. 75. Gale: A Cengage Company, “Gale Analytics,” available online at www.gale.com/databases/gale-analytics [accessed 7 May 2019]. 76. WhoFi, “Presence Analytics for Business and Government,” available online at https://whofi.com/presence- analytics/ [accessed 8 October 2019]. 77. “OCLC Wise: Community Engagement System for Public Libraries,” available online at https://www.oclc. org/en/wise.html [accessed 7 September 2018]. 78. Ashlynn Kogut, “Academic Library Services and Undergraduate Academic Success: Trends in Research Literature,” in Library Assessment Conference (2016), available online at http://old.libraryassessment.org/bm~doc/59- kogut-2016.pdf [accessed 3 March 2020]. 79. Travis T. York, Charles Gibson, and Susan Rankin, “Defining and Measuring Academic Success,” Practical Assessment, Research & Evaluation 20, no. 5 (Mar. 2015), available online at https://scholarworks.umass.edu/pare/ vol20/iss1/5/ [accessed 3 March 2020]. 80. Sue Samson, “Information Literacy Learning Outcomes and Student Success,” Journal of Academic Librari- anship 36, no. 3 (May 2010): 202–10, https://doi.org/10.1016/j.acalib.2010.03.002. 81. Felly Chiteng Kot and Jennifer L. Jones, “The Impact of Library Resource Utilization on Undergraduate Students’ Academic Performance: A Propensity Score Matching Design,” College & Research Libraries 76, no. 5 (July 2015): 566–86, https://doi.org/10.5860/crl.76.5.566. 82. Soria et al., “Library Use and Undergraduate Student Outcomes.” 83. John K. Stemmer and David M. Mahan, “Assessing the Library’s Influence on Freshman and Senior Level Outcomes with User Surveys,” Evidence Based Library and Information Practice 10, no. 2 (June 2015): 8, https://doi. org/10.18438/B8PG62. 84. Elizabeth L. Black and Sarah Anne Murphy, “The Out Loud Assignment: Articulating Library Contribu- tions to First-Year Student Success,” Journal of Academic Librarianship 43, no. 5 (Sept. 1, 2017): 409–16, https://doi. org/10.1016/j.acalib.2017.06.008. 85. Adam Murray, Ashley Ireland, and Jana Hackathorn, “The Value of Academic Libraries: Library Ser- vices as a Predictor of Student Retention,” College & Research Libraries 77, no. 5 (Sept. 2016): 631–42, https://doi. https://doi.org/10.5860/crl.77.2.227 http://scholarworks.gsu.edu/univ_lib_facpres/21 https://doi.org/10.1016/j.acalib.2013.02.006 https://doi.org/10.1353/pla.2013.0017 https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1773&context=charleston https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1773&context=charleston https://doi.org/10.1353/pla.2013.0010 https://doi.org/10.1353/pla.2013.0010 https://doi.org/10.5860/crl-151 https://doi.org/10.1080/19322909.2018.1555504 https://doi.org/10.1080/19322909.2018.1555504 https://scholarworks.gvsu.edu/library_proceedings/7/ https://springshare.com/ https://springshare.com/ https://www.orangeboyinc.com/ http://www.gale.com/databases/gale-analytics https://whofi.com/presence-analytics/ https://whofi.com/presence-analytics/ https://www.oclc.org/en/wise.html https://www.oclc.org/en/wise.html http://old.libraryassessment.org/bm~doc/59-kogut-2016.pdf http://old.libraryassessment.org/bm~doc/59-kogut-2016.pdf https://scholarworks.umass.edu/pare/vol20/iss1/5/ https://scholarworks.umass.edu/pare/vol20/iss1/5/ https://doi.org/10.1016/j.acalib.2010.03.002 https://doi.org/10.5860/crl.76.5.566 https://doi.org/10.18438/B8PG62 https://doi.org/10.18438/B8PG62 https://doi.org/10.1016/j.acalib.2017.06.008 https://doi.org/10.1016/j.acalib.2017.06.008 https://doi.org/10.5860/crl.77.5.631 588 College & Research Libraries April 2020 org/10.5860/crl.77.5.631; Krista M. Soria, Jan Fransen, and Shane Nackerud, “Stacks, Serials, Search Engines, and Students’ Success: First-Year Undergraduate Students’ Library Use, Academic Achievement, and Retention,” Journal of Academic Librarianship 40, no. 1 (Jan. 2014): 84–91, https://doi.org/10.1016/j.acalib.2013.12.002; Krista M. Soria, Jan Fransen, and Shane Nackerud, “The Impact of Academic Library Resources on Undergraduates’ De- gree Completion,” College & Research Libraries 78, no. 6 (Sept. 2017): 812–23, https://doi.org/10.5860/crl.0.0.16626. 86. Gaby Haddow, “Academic Library Use and Student Retention: A Quantitative Analysis,” Library & Information Science Research 35, no. 2 (Apr. 2013): 127–36, https://doi.org/10.1016/j.lisr.2012.12.002; Gaby Haddow and Jayanthi Joseph, “Loans, Logins, and Lasting the Course: Academic Library Use and Student Retention,” Australian Academic & Research Libraries 41, no. 4 (Dec. 2010): 233–44, https://doi.org/10.1080/00048623.2010.10721 478. 87. Sue Samson, “Usage of E-Resources: Virtual Value of Demographics,” Journal of Academic Librarianship 40, no. 6 (2014): 620–25. 88. Kot and Jones, “The Impact of Library Resource Utilization on Undergraduate Students’ Academic Per- formance.” 89. Graham Stone, Dave Pattern, and Bryony Ramsden, “Does Library Use Affect Student Attainment? A Preliminary Report on the Library Impact Data Project,” LIBER Quarterly 21, no. 1 (Oct. 2011): 5, https://doi. org/10.18352/lq.8005; Shun Han Rebekah Wong and T.D. Webb, “Uncovering Meaningful Correlation between Student Academic Performance and Library Material Usage,” College & Research Libraries 72, no. 4 (Jan. 2011): 361–370, available online at http://crl.acrl.org/index.php/crl/article/view/16168 [accessed 3 March 2020]. 90. Sandra Calemme McCarthy, “At Issue: Exploring Library Usage by Online Learners with Student Suc- cess,” Community College Enterprise 23, no. 2 (Jan. 2017): 27–31. 91. Tiffany LeMaistre, Qingmin Shi, and Sandip Thanki, “Connecting Library Use to Student Success,” portal: Libraries & the Academy 18, no. 1 (2018): 117–40, https://doi.org/10.1353/pla.2018.0006. 92. Soria, Fransen, and Nackerud, “The Impact of Academic Library Resources on Undergraduates’ Degree Completion.” 93. Kot and Jones, “The Impact of Library Resource Utilization on Undergraduate Students’ Academic Per- formance.” 94. Jean Marie Cook, “A Library Credit Course and Student Success Rates: A Longitudinal Study,” College & Research Libraries 75, no. 3 (May 2014): 272–83, https://doi.org/10.5860/crl12-424. 95. Kogut, “Academic Library Services and Undergraduate Academic Success”; M. Brooke Robertshaw and Andrew Asher, “Unethical Numbers? A Meta-Analysis of Library Learning Analytics Studies,” Library Trends 68, no. 1 (Oct. 24, 2019): 76–101, https://doi.org/10.1353/lib.2019.0031. 96. Soria, Fransen, and Nackerud, “The Impact of Academic Library Resources on Undergraduates’ Degree Completion.” 97. Kogut, “Academic Library Services and Undergraduate Academic Success.” 98. ALA, “Code of Ethics of the American Library Association” (Chicago, IL: American Library Association, 2008), available online at http://www.ala.org/advocacy/sites/ala.org.advocacy/files/content/proethics/codeofethics/ Code%20of%20Ethics%20of%20the%20American%20Library%20Association.pdf], Code of Ethics of the American Library Association.pdf [accessed 3 March 2020]; American Library Association (ALA), “Library Bill of Rights,” Text, Advocacy, Legislation & Issues (June 30, 2006), available online at www.ala.org/advocacy/intfreedom/libra- rybill [accessed 3 March 2020]. 99. International Federation of Library Associations (IFLA), “IFLA Statement on Privacy in the Library En- vironment” (Aug. 14, 2015): 2, available online at https://www.ifla.org/publications/node/10056 [accessed date 3 March 2020]. 100. National Information Standards Organization (NISO), “NISO Consensus Principles on Users’ Digital Privacy in Library, Publisher, and Software-Provider Systems” (Dec. 10, 2015), available online at https://www. niso.org/publications/privacy-principles [accessed 3 March 2020]. 101. ALA, “Code of Ethics of the American Library Association.” 102. ALA, “Library Bill of Rights.” 103. IFLA, “IFLA Statement on Privacy in the Library Environment.” 104. NISO, “NISO Consensus Principles on Users’ Digital Privacy in Library, Publisher, and Software-Provider Systems.” 105. NISO, “NISO Consensus Principles on Users’ Digital Privacy in Library, Publisher, and Software-Provider Systems,” 3. 106. Connaway et al., Academic Library Impact. 107. Andrew Asher et al., “Ethics in Research Use of Library Patron Data: Glossary and Explainer” (Oct. 2, 2018), https://doi.org/10.17605/OSF.IO/XFKZ6. https://doi.org/10.5860/crl.77.5.631 https://doi.org/10.1016/j.acalib.2013.12.002 https://doi.org/10.5860/crl.0.0.16626 https://doi.org/10.1016/j.lisr.2012.12.002 https://doi.org/10.1080/00048623.2010.10721478 https://doi.org/10.1080/00048623.2010.10721478 https://doi.org/10.18352/lq.8005 https://doi.org/10.18352/lq.8005 http://crl.acrl.org/index.php/crl/article/view/16168 https://doi.org/10.1353/pla.2018.0006 https://doi.org/10.5860/crl12-424 https://doi.org/10.1353/lib.2019.0031 http://www.ala.org/advocacy/sites/ala.org.advocacy/files/content/proethics/codeofethics/Code%20of%20Ethics%20of%20the%20American%20Library%20Association.pdf http://www.ala.org/advocacy/sites/ala.org.advocacy/files/content/proethics/codeofethics/Code%20of%20Ethics%20of%20the%20American%20Library%20Association.pdf http://www.ala.org/advocacy/sites/ala.org.advocacy/files/content/proethics/codeofethics/Code http://www.ala.org/advocacy/intfreedom/librarybill http://www.ala.org/advocacy/intfreedom/librarybill https://www.ifla.org/publications/node/10056 https://www.niso.org/publications/privacy-principles https://www.niso.org/publications/privacy-principles https://doi.org/10.17605/OSF.IO/XFKZ6 Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 589 108. Library Information Technology Association (LITA), “Patron Privacy Technologies Interest Group” (Nov. 25, 2014), available online at www.ala.org/lita/about/igs/public/lit-Pp [accessed 3 March 2020]. 109. Alison Macrina, “Library Freedom Project: Making Real the Promise of Intellectual Freedom in Librar- ies” (2017), available online at https://libraryfreedomproject.org/ [accessed 3 March 2020]. 110. “The Asilomar Convention for Learning Research in Higher Education,” available online at http:// asilomar-highered.info/ [accessed 15 July 2019]. 111. Sharon Slade and Paul Prinsloo, “Learning Analytics: Ethical Issues and Dilemmas,” American Behavioral Scientist 57, no. 10 (Mar. 2013): 1510–29, https://doi.org/10.1177/0002764213479366; N. Sclater, “Code of Practice for Learning Analytics: A Literature Review of the Ethical and Legal Issues” (2014), available online at http:// repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf [accessed 3 March 2020]; Abelardo Pardo and George Siemens, “Ethical and Privacy Principles for Learning Analytics,” British Journal of Educational Technology 45, no. 3 (2014): 438–50, https://doi.org/10.1111/bjet.12152. 112. The Open University, “Ethical Use of Student Data for Learning Analytics,” Student Policies and Regula- tions, available online at https://help.open.ac.uk/documents/policies/ethical-use-of-student-data [accessed 15 July 2019]. 113. University of California, “University of California: Learning Data Privacy Principles” (Mar. 5, 2018), available online at https://www.ets.berkeley.edu/sites/default/files/general/university_of_california_learn- ing_data_privacy_principles_handout.pdf [accessed 3 March 2020]; University of Hawaii Mānoa Committee on Academic Policy and Planning, “Resolution Supporting Learning Data Privacy Principles and Practices,” Google Docs (Nov.14, 2018), available online at https://docs.google.com/document/d/1bqJTZ4tCK3SFdsS4rXVN K5xlS87g0ajfE3EYk3u0Zh8/edit?usp=embed_facebook [accessed 3 March 2020]. 114. Michael R. Perry et al., “Learning Analytics, SPEC Kit 360” (Sept. 4, 2018), available online at https:// publications.arl.org/Learning-Analytics-SPEC-Kit-360/ [accessed 3 March 2020]. 115. Andrew Asher, “Risk, Benefits, and User Privacy: Evaluating the Ethics of Library Data,” in Protecting Patron Privacy, eds. Bobbi Newman and Bonnie Tijerina (Lanham, MA: Rowman & Littlefield, 2017), 43–56. 116. Jones and Salo, “Learning Analytics and the Academic Library.” 117. Kyle M.L. Jones, Alan Rubel, and Ellen LeClere, “A Matter of Trust: Higher Education Institutions as Information Fiduciaries in an Age of Educational Data Mining and Learning Analytics.” Journal of the Association for Information Science and Technology, https://doi.org/10.1002/asi.24327. 118. Asher et al., “Ethics in Research Use of Library Patron Data.” 119. Kristin A. Briney, “Data Management Practices in Academic Library Learning Analytics: A Critical Re- view,” Journal of Librarianship and Scholarly Communication 7, no. 1 (Feb. 22, 2019): 1–39, https://doi.org/10.7710/2162- 3309.2268. 120. Kristin Briney et al., “A Practical Guide to Performing a Data Risk Assessment in a Library (Draft),” Technologies of Surveillance Working Group (Digital Library Federation, Oct. 2019), available online at https:// docs.google.com/document/d/1FF75widvFPW2Ean68FIu6ddN7bgjfk2LzntYpZVc-KA/edit [accessed 3 March 2020]; Asher et al., “Ethics in Research Use of Library Patron Data.” 121. Asher, “Risk, Benefits, and User Privacy.” 122. Melissa Bowles-Terry, “Library Instruction and Academic Success: A Mixed-Methods Assessment of a Library Instruction Program,” Evidence Based Library and Information Practice 7, no. 1 (Mar. 2012), https://doi. org/10.18438/B8PS4D; Graham Stone and Bryony Ramsden, “Library Impact Data Project: Looking for the Link between Library Usage and Student Attainment,” College & Research Libraries 74, no. 6 (2013): 546–59, https://doi. org/10.5860/crl12-406. 123. Stone and Ramsden, “Library Impact Data Project.” 124. LeMaistre, Shi, and Thanki, “Connecting Library Use to Student Success.” 125. Briney, “Data Management Practices in Academic Library Learning Analytics.” 126. Perry et al., “Learning Analytics, SPEC Kit 360.” 127. Murray, Ireland, and Hackathorn, “The Value of Academic Libraries.” 128. Haddow and Joseph, “Loans, Logins, and Lasting the Course.” 129. Becky Yoose, “Patrons, Privacy, and Technology in the Library” (2017), available online at https://docs. google.com/presentation/d/14gaycdLUCiD71FlY2BAsNggsG3D6fmVQFDmkSSdD1XA/edit#slide=id.p [accessed 3 March 2020]. 130. Fransen and Peterson, “Graduate in Four Years? Yes, the Library Can Help with That!” 131. Ellen Collins and Graham Stone, “Understanding Patterns of Library Use Among Undergraduate Stu- dents from Different Disciplines,” Evidence Based Library and Information Practice 9, no. 3 (Sept. 2014): 51, https:// doi.org/10.18438/B8930K; LeMaistre, “Cost Per User.” 132. Alison Pepper and Margie Jantti, “The Tipping Point: How Granular Statistics Can Make a Big Differ- http://www.ala.org/lita/about/igs/public/lit-Pp https://libraryfreedomproject.org/ http://asilomar-highered.info/ http://asilomar-highered.info/ https://doi.org/10.1177/0002764213479366 http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf https://doi.org/10.1111/bjet.12152 https://help.open.ac.uk/documents/policies/ethical-use-of-student-data https://www.ets.berkeley.edu/sites/default/files/general/university_of_california_learning_data_privacy_principles_handout.pdf https://www.ets.berkeley.edu/sites/default/files/general/university_of_california_learning_data_privacy_principles_handout.pdf https://docs.google.com/document/d/1bqJTZ4tCK3SFdsS4rXVNK5xlS87g0ajfE3EYk3u0Zh8/edit?usp=embed_facebook https://docs.google.com/document/d/1bqJTZ4tCK3SFdsS4rXVNK5xlS87g0ajfE3EYk3u0Zh8/edit?usp=embed_facebook https://publications.arl.org/Learning-Analytics-SPEC-Kit-360/ https://publications.arl.org/Learning-Analytics-SPEC-Kit-360/ https://doi.org/10.1002/asi.24327 https://doi.org/10.7710/2162-3309.2268 https://doi.org/10.7710/2162-3309.2268 https://docs.google.com/document/d/1FF75widvFPW2Ean68FIu6ddN7bgjfk2LzntYpZVc-KA/edit https://docs.google.com/document/d/1FF75widvFPW2Ean68FIu6ddN7bgjfk2LzntYpZVc-KA/edit https://doi.org/10.18438/B8PS4D https://doi.org/10.18438/B8PS4D https://doi.org/10.5860/crl12-406 https://doi.org/10.5860/crl12-406 https://docs.google.com/presentation/d/14gaycdLUCiD71FlY2BAsNggsG3D6fmVQFDmkSSdD1XA/edit#slide=id.p https://docs.google.com/presentation/d/14gaycdLUCiD71FlY2BAsNggsG3D6fmVQFDmkSSdD1XA/edit#slide=id.p https://doi.org/10.18438/B8930K https://doi.org/10.18438/B8930K 590 College & Research Libraries April 2020 ence in Understanding and Demonstrating Value,” in Australian Library and Information Association Information Online (Sydney, Australia, 2015), available online at https://works.bepress.com/mjantti/34/ [accessed 3 March 2020]; Maximiliano Montenegro et al., “Library Resources and Students’ Learning Outcomes: Do All the Resources Have the Same Impact on Learning?” 42, no. 5 (2016): 551–56. 133. Briney, “Data Management Practices in Academic Library Learning Analytics.” 134. Perry et al., “Learning Analytics, SPEC Kit 360.” 135. Becky Yoose, “Balancing Privacy and Strategic Planning Needs: A Case Study in de-Identification of Patron Data,” Journal of Intellectual Freedom and Privacy 2, no. 1 (July 2017): 15–22, https://doi.org/10.5860/jifp. v2i1.6250. 136. Yoose, “Balancing Privacy and Strategic Planning Needs”; Simson L. Garfinkel, “NISTIR 8053: De-Iden- tification of Personal Information” (2015), https://doi.org/10.6028/NIST.IR.8053; Mohammad Khalil and Martin Ebner, “De-Identification in Learning Analytics,” Journal of Learning Analytics 3, no. 1 (Apr. 2016): 129–38, https:// doi.org/10.18608/jla.2016.31.8. 137. Briney, “Data Management Practices in Academic Library Learning Analytics.” 138. “49 CFR Part 11 Federal Policy for the Protection of Human Subjects,” Federal Register 82, no. 12 (Jan. 19, 2017): 7149–7274. 139. US Department of Education, “Family Educational Rights and Privacy Act (FERPA),” Guides (Mar. 1, 2018), available online at https://www2.ed.gov/policy/gen/guid/fpco/ferpa/index.html [accessed 3 March 2020]. 140. Scott W.H. Young, “Achieving Privacy in the Age of Analytics” (2018), available online at https:// scottwhyoung.com/talks/web-privacy-web-analytics-niso-2018/ [accessed 3 March 2020]; M Ryan Hess, “Google Analytics and Privacy,” Fail!Lab (blog, Apr. 27, 2016), available online at https://faillab.wordpress.com/2016/04/27/ google-analytics-and-privacy/ [accessed 3 March 2020]. 141. Briney, “Data Management Practices in Academic Library Learning Analytics.” 142. Perry et al., “Learning Analytics, SPEC Kit 360.” 143. Briney, “Data Management Practices in Academic Library Learning Analytics.” 144. Jose van Dijck, “Datafication, Dataism and Dataveillance: Big Data between Scientific Paradigm and Ideology,” Surveillance & Society 12, no. 2 (May 9, 2014): 197–208, https://doi.org/10.24908/ss.v12i2.4776. 145. Rob Kitchin, “Big Data, New Epistemologies and Paradigm Shifts,” Big Data & Society 1, no. 1 (Jan. 1, 2014): 2053951714528481, https://doi.org/10.1177/2053951714528481. 146. Jones, “Just Because You Can Doesn’t Mean You Should.” 147. Scott W.H. Young et al., “A National Forum on Web Privacy and Web Analytics: Action Handbook,” (May 2019), https://doi.org/10.15788/20190416.15446. 148. Yoose, “Balancing Privacy and Strategic Planning Needs”; LeMaistre, Shi, and Thanki, “Connecting Library Use to Student Success.” 149. Soyeon Park, “The Study of Research Methods in LIS Education: Issues in Korean and U.S. Universities,” Library & Information Science Research 26, no. 4 (Sept. 1, 2004): 501–10, https://doi.org/10.1016/j.lisr.2004.04.009; Juris Dilevko, “Inferential Statistics and Librarianship,” Library & Information Science Research 29, no. 2 (June 1, 2007): 209–29, https://doi.org/10.1016/j.lisr.2007.04.003. 150. R.R. Powell, L.M. Baker, and J.J. Mika, “Library and Information Science Practitioners and Research,” Library & Information Science Research 24, no. 1 (2002): 49–72, https://doi.org/10.1016/S0740-8188(01)00104-9; Marie R. Kennedy and Kristine R. Brancolini, “Academic Librarian Research: A Survey of Attitudes, Involvement, and Perceived Capabilities,” College & Research Libraries 73, no. 5 (Sept. 2012): 431–48, https://doi.org/10.5860/crl-276. 151. Marie Kennedy and Kristine Brancolini, “Academic Librarian Research: An Update to a Survey of At- titudes, Involvement, and Perceived Capabilities,” College & Research Libraries 79, no. 6 (2018): 822–51, https://doi. org/10.5860/crl.79.6.822. 152. Perry et al., “Learning Analytics, SPEC Kit 360.” 153. Robertshaw and Asher, “Unethical Numbers?” 154. Kristin Hoffmann, Selinda Adelle Berg, and Denise Koufogiannakis, “Examining Success: Identify- ing Factors That Contribute to Research Productivity across Librarianship and Other Disciplines,” Library and Information Research 38, no. 119 (2014): 13–28, https://doi.org/10.29173/lirg639. 155. Catherine Sassen and Diane Wahl, “Fostering Research and Publication in Academic Libraries,” College & Research Libraries 75, no. 4 (July 2014): 458–91, https://doi.org/10.5860/crl.75.4.458. 156. Elizabeth M. Smigielski, Melissa A. Laning, and Caroline M. Daniels, “Funding, Time, and Mentoring: A Study of Research and Publication Support Practices of ARL Member Libraries,” Journal of Library Administra- tion 54, no. 4 (May 19, 2014): 261–76, https://doi.org/10.1080/01930826.2014.924309. 157. Sarah Hartman-Caverly and Alex Chisholm, “Digital Shred: Privacy Literacy Toolkit,” available online at https://sites.psu.edu/digitalshred/ [accessed 8 October 2019]; Macrina, “Library Freedom Project”; Scott W.H. https://works.bepress.com/mjantti/34/ https://doi.org/10.5860/jifp.v2i1.6250 https://doi.org/10.5860/jifp.v2i1.6250 https://doi.org/10.6028/NIST.IR.8053 https://doi.org/10.18608/jla.2016.31.8 https://doi.org/10.18608/jla.2016.31.8 https://www2.ed.gov/policy/gen/guid/fpco/ferpa/index.html https://scottwhyoung.com/talks/web-privacy-web-analytics-niso-2018/ https://scottwhyoung.com/talks/web-privacy-web-analytics-niso-2018/ https://faillab.wordpress.com/2016/04/27/google-analytics-and-privacy/ https://faillab.wordpress.com/2016/04/27/google-analytics-and-privacy/ https://doi.org/10.24908/ss.v12i2.4776 https://doi.org/10.1177/2053951714528481 https://doi.org/10.15788/20190416.15446 https://doi.org/10.1016/j.lisr.2004.04.009 https://doi.org/10.1016/j.lisr.2007.04.003 https://doi.org/10.1016/S0740-8188(01)00104-9 https://doi.org/10.5860/crl-276 https://doi.org/10.5860/crl.79.6.822 https://doi.org/10.5860/crl.79.6.822 https://doi.org/10.29173/lirg639 https://doi.org/10.5860/crl.75.4.458 https://doi.org/10.1080/01930826.2014.924309 https://sites.psu.edu/digitalshred/ Comprehensive Primer to Library Learning Analytics Practices, Initiatives, and Privacy Issues 591 Young et al., “A Roadmap for Achieving Privacy in the Age of Analytics: A White Paper from A National Forum on Web Privacy and Web Analytics” (May 2019), https://doi.org/10.15788/20190416.15445; Scott W.H. Young et al., “A National Forum on Web Privacy and Web Analytics: Action Handbook” (Montana State University, Apr. 30, 2019), https://doi.org/10.15788/20190416.15446; Alex Chisholm and Sarah Hartman-Caverly, “If Not Us, Who? Privacy Literacy Instruction in Academic Libraries, Connect & Communicate” (College and Research Division of the Pennsylvania Library Association), available online at https://www.youtube.com/watch?v=49mDqex6K1 o&feature=youtu.be [accessed 8 October 2019]. 158. Jones and McCoy, “Reconsidering Data in Learning Analytics.” 159. Alan Rubel and Kyle M.L. Jones, “Student Privacy in Learning Analytics: An Information Ethics Per- spective,” The Information Society 32, no. 2 (Mar. 2016): 143–59, https://doi.org/10.1080/01972243.2016.1130502. 160. Clara Schumacher and Dirk Ifenthaler, “Features Students Really Expect from Learning Analytics,” Computers in Human Behavior 78 (Jan. 1, 2018): 397–407, https://doi.org/10.1016/j.chb.2017.06.030; Lynne D. Roberts et al., “Student Attitudes toward Learning Analytics in Higher Education: The Fitbit Version of the Learning World,” Frontiers in Psychology 7 (2016), https://doi.org/10.3389/fpsyg.2016.01959. 161. Constance A. Mellon, “Library Anxiety: A Grounded Theory and Its Development,” College & Research Libraries 76, no. 3 (Mar. 2015): 276–82, https://doi.org/10.5860/crl.76.3.276. 162. Steve Witt, “The Evolution of Privacy within the American Library Association, 1906–2002,” Library Trends 65, no. 4 (Sept. 8, 2017): 639–57, https://doi.org/10.1353/lib.2017.0022. https://doi.org/10.15788/20190416.15445 https://doi.org/10.15788/20190416.15446 https://www.youtube.com/watch?v=49mDqex6K1o&feature=youtu.be https://www.youtube.com/watch?v=49mDqex6K1o&feature=youtu.be https://doi.org/10.1080/01972243.2016.1130502 https://doi.org/10.1016/j.chb.2017.06.030 https://doi.org/10.3389/fpsyg.2016.01959 https://doi.org/10.5860/crl.76.3.276 https://doi.org/10.1353/lib.2017.0022