87 Employer Perceptions of Critical Information Literacy Skills and Digital Badges Victoria Raish and Emily Rimland Victoria Raish is a PhD Student in the College of Education and Emily Rimland is Sally W. Kalin Librarian for Learning Innovations at The Pennsylvania State University; e-mail: victoria@psu.edu, erimland@psu. edu. © 2016 Victoria Raish and Emily Rimland, Attribution-NonCommercial (http://creativecommons. org/licenses/by-nc/3.0/) CC BY-NC. Digital badges are an educational innovation used to measure learning of specific skills, such as information literacy. However, few studies have quantitatively surveyed employers for their perceptions about information literacy skills or digital badges. An online survey was developed and sent to employers to gauge perceptions of information literacy skills of college graduates and the use of the innovation of digital badges to represent competencies that students have accumulated. Here are the results of the survey: information literacy and metaliteracy skills are valued in the workplace; employers would like more detailed representations of stu- dent skills; and digital badges are a possible way to showcase student achievement. n the workplace, employees are expected to have several skills and compe- tencies in the area of information literacy. The information literacy standards have recently been revised to incorporate a model of metaliteracy, which espouses a comprehensive array of dynamic dispositions and collabora- tive abilities in addition to the traditional skills and practices of information literacy. Recent changes in the workplace environment emphasize the use of digital resources, the ability to create and share artifacts using digital resources, and the expectation that employees can collaboratively work in teams. These new workplace demands parallel a shift in librarianship from the traditionally skill-focused information literacy toward the overarching framework of metaliteracy. Although college students may possess these critical skills, competencies, and literacies, there is a lack of transparency and proof to show employers that new graduates possess them. A sustainable solution to the challenges of the “one-shot” nature of information literacy instruction has not been realized and there remains a lack of systemic information literacy instruction in higher education. A potential way to alleviate issues associated with traditional forms of instruction and the desire to certify granular skills of college graduates is to use digital badges. Digital badges are a recent technological innovation that can scaffold the learning environment for students and visually represent the abilities gained to a broader community. doi:10.5860/crl.77.1.87 crl15-712 88 College & Research Libraries January 2016 Both students in academia and employees in the workplace are frequently expected to possess critical thinking, problem solving, innovative thinking, and lifelong learn- ing abilities. These abilities serve as the foundation for the acquisition of information literacy and metaliteracy. At the core of information literacy and metaliteracy is higher order thinking. However, the term “information literacy” has traditionally been lim- ited to specific contexts and does not extend into workplace discussions. This lack of transfer connotes a need for translation of these skills from the academic setting to the workplace.1 Previous research on information literacy in the workplace shows that the skills valued by the American Library Association (ALA) are not exactly the same as the skills valued in the workplace.2 Being information literate is a critical factor to success in the information age, and it is necessary to include information literacy skills valued by employers into library instruction curricula for students. Thomas Mackey and Trudi Jacobson’s article “Reframing Information Literacy as Metaliteracy” discusses the transition from information literacy to metaliteracy.3 As part of this article, they advo- cate that the ALA information literacy standards be revised to include a more holistic picture of learning. This new model should include new learning technologies and tools for interacting with information. These technologies should provide abilities for sharing and creating products in online environments to prepare students for today’s complex and increasingly digital environment. The new focus on metaliteracy allows for the representation of relevant skills by a digitally cognitive technology like digital badges. The coinciding of timing between the increased use of digital badges and the ALA revision process presents an excellent opportunity to use digital badges as a way for students to represent metaliteracy mastery both within and outside academia. At the time of our research, the ALA revision process was ongoing and the new Informa- tion Literacy Framework was not available. However, due to our use of a variety of learning theories including metaliteracy to develop and shape our badges, we were able to incorporate many of the tenets of the new Information Literacy Framework and metaliteracy learning objectives into our research and a set of digital badges. At the most basic level, it is helpful to think of a digital badge as a microcredential, similar to a boy or girl scout badge, that certifies skills at a more granular level than grades or transcripts. They serve as a way to represent learning across a broad range of settings and are used to certify that a learner has gained competence or expert knowl- edge in a certain skill, concept, or phenomena. From a learning sciences perspective, a digital badge can be thought of as a cognitively oriented technology or mindtool that can be used to deepen and extend the learning process for students.4 More formally, digital badges are a “representation of an accomplishment, interest, or affiliation that is visual and available online, containing metadata including links that help explain the context, meaning, process, and result of an activity.”5 Digital badges have recently garnered attention as a way to certify learning in a variety of contexts. Badges are being considered in numerous environments including medical schools and teacher professional development, from elementary to graduate schools.6 Recently, an initiative has been announced between several universities, in- cluding Georgetown University, and various employers and experts to initiate the “21st Century Skills Badging Challenge” to design badges that are recognized as valuable by employers.7 According to Everett Rogers, one of the founding fathers of the study of diffusion of innovation, stakeholders currently adopting digital badges can be classi- fied as “early adopters.” 8 Digital badges are gaining a lot of attention from a variety of groups because their basic premise is similar to well-established learning tools (such as traditional badges, motivation, educational gaming). 9 There is no literature to date on the digital badge adoption rate among human resource professionals or hiring manag- ers; however, Mozilla, a staunch supporter of the digital badge movement, maintains Employer Perceptions 89 a growing list of the organizations adopting digital badges.10 In a blog posting on the Institute for Credentialing Excellence Web site, Frank Catalano notes that digital badges are clearly gaining ground and increasingly being adopted by certain sectors such as manufacturing and software industries.11 In addition, Jonathan Finkelstein, Erin Knight, and Susan Maning’s work focuses on how digital badges can be used and leveraged by certain demographics such as adult learners.12 New digital badge programs within libraries have only recently been initiated. For example, the University of Maryland’s Terrapin Learning Commons Workshop series “badgified” out-of-class learning to help students gain media production and citation skills.13 The State University of New York at Albany received grant funding to launch a Metaliteracy Learning Collaborative in which SUNY students can earn a variety of metaliteracy badges.14 These programs will provide a wealth of information but are currently in their research and development stages. Mozilla’s open badging infrastructure and documentation initiative have established a badge creation standard that anyone can use to ensure that their badges align with those of the larger digital badging community.15 Dozens of institutions are currently adhering to this standard, and, according to Mozilla, companies such as TopCoder have begun accepting these badges as a means for identifying top job candidates. An Association of College and Research Libraries’ blog touched on this issue by featuring a post about how digital badges can help information literacy become in- tegrated in existing curricula and show the value of information literacy to others.16 Employers value the higher-order abilities of metaliteracy such as communication, problem solving, and synthesizing information; and a credentialing system to certify student competency of these areas could be useful for recognizing and representing them to new audiences.17 Digital badges “bring students together in new types of learn- ing communities, communities that support all students in more effective learning and deeper understanding of the material.”18 To explore these issues further, the authors created and administered a national sur- vey of employers to determine which competencies should be included in our digital badges and to evaluate the potential worth of digital badges to a community outside academia. The intent of this survey was to gain the perspective of human resource personnel about which metaliteracy- and information literacy–related competencies recent college graduates are expected to have when hired and which activities support the development of those competencies. Rogers notes that gaining collective approval of an innovation is difficult and adoption of innovations by organizations is typically a slower process than individual adoption decisions.19 By including employers in the conversation around digital badges, the authors felt the process of adoption could be increased. A secondary intent of administering this survey was to gauge potential interest in digital badges and improve the odds of acceptance by visually representing the advantages of them.20 Literature Review Our literature review examines the research already completed on information literacy in the workplace as well as the framework around which we designed our survey questions and interpreted the results. The content of our survey instrument was drawn from available research related to the following: information literacy, the transition to metaliteracy, learning sciences, connectivism, and digital badges. How Information Literacy and Information Literacy Instruction Are Changing Much of the existing research related to information literacy skills desired by employ- ers is qualitative and involves small samples such as interviews and focus groups.21 90 College & Research Libraries January 2016 In addition, this research has not focused on the foundational skills or metaliteracies underpinning the information literacy skills that are valued in the workplace. Prior sur- veys and qualitative research of employers have found that employers are consistently surprised by the lack of information literacy skills that students have when entering the workforce.22 This partially stems from the inadequacy of systemic information literacy instruction and a difference in expectations between information literacy in higher education and in the workplace.23 Information literacy skills are taught in a variety of ways, contexts, and settings, with the most common form being the “one-shot” or single-instance class taught by a librarian. The result of the one-shot class is an uneven outreach to students about information literacy, with some students never having such a session and some having multiple sessions.24 A potential solution to assist with this problem is the use of digital badges to specify, certify, and represent student competen- cies of information literacies. The American Library Association emphasizes that information literacy, and now metaliteracies, are broad-spectrum abilities, dispositions, and skills that inform many of the core competencies that students should learn in a higher education environment.25 Maybee conducted an empirical study on student perceptions of information sense and found that information literacy skills should build from simple to more complex and become relational in their instruction.26 Relational, in this sense, means that the information literacy skills are hierarchical in nature and relate to one another. However, current information literacy instruction is typically linear with the concepts not being connected to one another and without building them in a way for students to be able to use them in relation to one another.27 The ACRL (Association of College and Research Libraries) information literacy standards have been revised to consider not only the context in which students are typically expected to find information, but rapid changes in the shift of how people use information. The traditional meaning of information literacy is being challenged to consider changes in technology, format, delivery mechanism, the collaborative nature of information sharing, affective learning, and more. 28 The original definition for information literacy is still relevant; however, the expanded, more flexible defini- tion is even more apropos. 29 A major new feature of the revised Information Literacy Framework is the idea of metaliteracy. Metaliteracy is associated with promoting “critical thinking and collaboration in a digital age,” and it completes a more holistic view of information literacy that considers modern web technology, the participatory environment of the web, and the rapidly changing technologies in use today.30 Our view of metaliteracy aligns with the perspective taken by Mackey and Jacobsen. Many of our survey questions and ultimately our digital badges reflect the shift toward developing metaliteracies. The need to revise information literacy instruction due to the information age neces- sitates theories of learning that espouse learning not solely as memorizing but also as being able to find information, evaluate it, and use it adequately for one’s purpose. This type of learning features connections and knowledge building in an information-rich environment. Connectivism is a theory that helps to explain the perspective of this type of learning.31 Connectivism recognizes that we exist in a complex and rapidly changing environment and connections in networks allow us to learn. 32 Connectiv- ism, much like metaliteracy, embraces the new competencies students need to succeed in a changing networked environment. Tjeerd Plomp recognizes that the transition from an industrial age to an information age has led to a change in how we think about processing information and what it means to learn and know.33 In addition, a new information processing environment leads to a dynamic system in which change is expected and connections to various resources are essential to learning through Employer Perceptions 91 many forms of media.34 The importance of technology and using resources wisely in the information age emphasizes even more the need to have effective information literacy instruction. An emergent outcome of the information age is the idea of “21st century skills.” Ronald Anderson extensively reviewed 21st century skill frameworks and found that they have their basis in knowledge societies and are “predicated on knowledge and information society concepts and concerns.” 35 While there is not a universal definition of 21st century skills, our research references the skills from the Partnership for 21st Century Skills, which includes information and media literacy among many other criti- cal components.36 Information literacy concepts hold the potential to instill in students many of the skill sets desired in this new society while furthering their education and enhancing their careers. However, information literacy is also changing to include more than just the skills to effectively find, filter, evaluate, analyze, and store informa- tion.37 Carmel O’Sullivan acknowledges the criticisms of information literacy in that it does not typically extend beyond libraries and is not seen as an important concept in the business world. 38 However, the need for people to prepare to contribute in a knowledge society where innovation is emphasized exists in all modern fields.39 It is important for employees to be able to realize the potential value of an innovation and apply it appropriately to their own setting.40 Information literacy and metaliteracy are ways to provide students with the types of skills that will allow them to be creative and innovative.. The question then becomes how to help students develop the 21st century skills to be innovative. A person who is able to do this efficiently can improve his or her creativity and ways of using in- formation. An information-literate person is able to create knowledge, and creating knowledge is a required activity for one to be considered innovative. Innovation and creativity are clearly valued within metaliteracy and the new Information Literacy Framework, with the ability to “exhibit mental flexibility and creativity” being clearly stated or assumed in the dispositions.41 Thus, innovation was a key concept included in our survey of employers. How Digital Badges Can Help Technological innovations have been used throughout history in educational contexts. Prior to digital badges, there have been numerous efforts to use innovations to deliver information literacy instruction. Digital badges are increasingly being used by a number of collegiate libraries. The University of Maryland has started a digital badge program for students to gain technology competencies, and the University of Central Florida has an “infolit” digital badge platform for their information literacy modules.42 We believe that digital badges will be an appropriate innovation for current conceptions of information literacy and metaliteracy moving forward for four major reasons. First, the information literacy standards revision and transition to metaliteracy have the potential to be well represented by digital badges. Second, digital badges are a bridge between connectivism and information literacy. This is important because connectivism is cur- rently a proposed learning theory. Connectivism is a developmental theory that “may attempt to take strides toward becoming an established formal theory over time” and can “lead to empirical research that can then validate—or disprove—formal hypothesis posited within the framework of the scientific method.” 43 Although connectivism is a recently proposed learning theory, the basic tenets of connectivism fit well with the tenets of information literacy and metaliteracy. Digital badges can serve both as a node in a learner’s network to better connect them to social networks for possible connec- tions and as a way to structure activities that introduce students to metaliteracy. Third, digital badges serve dual roles—as scripts for students in their learning environment 92 College & Research Libraries January 2016 and as visual representations to a broader audience for the skills which those students possess. Finally, information literacy can be thought of as a set of more granular skills that make up a whole. Digital badges can rep- resent those granular skills and show a potential employer what the student actually can do.44 In addition to being a valuable learning tool for students, digital badges are an outward- facing visual representation of what the student has learned. Digital badges can serve as a starting point in conversation and the visual image can aid in the inter- pretation of the badge to some- one outside the system.45 Penn State University has developed a platform for digital badges with many communities and colleges from around the university participating. Figure 1 is an example of a digital badge that is being used as an information literacy digital badge at Penn State University. The information literacy digital badges created at this Penn State University have not been adopted by a broad audience but have been used in pilot sessions and are going to be piloted again with more students. Connectivism and Other Learning Theories Connectivism is an underlying theory that draws comparisons to information literacy and helps to contextualize the concept of metaliteracy. Connectivism has also helped to give meaning to our survey questions and research.46 George Siemens was one of the first originators of connectivism, and his extensive experience in online learning helped him realize that we now live in a networked information landscape focused on the interconnected world.47 The shifting environment of modern web technologies, connected learning, and changes in the way people are expected to adapt to chang- ing environments brings information literacy and technology together in a way that requires learners to critically evaluate, organize information, and effectively search for information and resources on the Internet.48 The U.S. Department of Labor indicates that students can expect to have 10–14 jobs by the age of 38; lifelong learning is important for learning the requirements of those new jobs.49 The effective design of digital badges for information literacy can provide an opportunity to gain these skills for all students, regardless of field or concentration. In addition to connectivism and metaliteracy, inspiration for our badge development was drawn from the learning sciences. One area of emphasis in the learning sciences is the idea of adaptive expertise, which may be useful in informing effective design of digital badges. Adaptive experts are much more likely to “evolve their core competen- cies and continually expand the breadth and depth of their expertise as the need arises or as their interests demand.”50 One who becomes an adaptive expert may be better equipped to handle the skills and competencies that are expected for lifelong learn- ing and change. Adaptive experts frequently have experience in solving ill-structured FIGURE 1 Example of a Digital Badge for Information Literacy Employer Perceptions 93 problems and are exposed to a variety of situations that require different responses.51 Thus, many of our survey questions and the beta design of our digital badges were intended to evaluate the importance of certain skills that were thought to contribute to the development of adaptive expertise. Studies surveying employer reactions to digital badges as a way to represent student competencies are scarce. If digital badges become widely adopted, their worth beyond the existing learning community needs to be assessed.52 Thus, our survey provides initial empirical insights into perceptions across a wide range of industries. Peck and Carr specify an equation for adopting innovations, which states that, for an innova- tion to be adopted, the “dissatisfaction x vision x first steps is greater than resistance,” which must be achieved for people to see the relative advantage of the innovation.53 When this equation is met, it will increase the odds of the innovation being adopted. When looking at the adoption of a technological innovation, such as digital badges, Rogers emphasizes that many of the possible adopters of the system do not necessar- ily see the advantage over the existing technology or system.54 The findings from our survey can help increase the adoption of digital badges by examining their worth in the workplace and increasing the social networks where digital badges are known. Rogers notes that these two factors are important characteristics in the diffusion of an innovation. 55 The goal of our survey was two-fold. First, we wanted to ascertain which abilities were both: 1) viewed as most essential by employers; and 2) addressable with an infor- mation literacy activity that could be designed from a learning sciences or connectivist perspective. Our survey and research aim to begin a conversation about digital badges with employers who would theoretically be reviewing them as a way to highlight skills of potential employees. Methodology Previous studies have investigated the skills students already possess or the skills em- ployers think students should have upon entering the workplace.56 Prior literature has discussed communities of practice to see how information literacy is used in context in various professions.57 Lloyd has considered the contextual component of information literacy extensively and has described what being “information literate” means in a variety of environments, especially highlighting the lack of research on workplace information literacy.58 The purpose of this study is to transform the findings from previous, mainly qualitative studies into quantifiable questions that ascertain factors related to employer perceptions of the skills identified as important in both previous studies and information literacy and now metaliteracy.59 The possible relationship be- tween employer category or industry and which skills are viewed as important in that field and employer willingness to evaluate digital badges as representations of student competencies are investigated. The research questions of this study are: 1. What skills or underlying information literacy competencies are valued the most across all human resource personnel of various employer types? Are there certain information literacy concepts that are valued as important and are also frequently not found in the skill set of employees who are recent col- lege graduates? 2. What collegiate experiences are most important to employers when evaluating potential candidates? Is one or more of these experiences hard to represent in traditional transcripts and grades? 3. Is there a significant percentage of human resource personnel who are interested in using digital badges as a way to certify student skills? Does this vary across employment sectors? 94 College & Research Libraries January 2016 A 56-item in-house online survey was designed to gauge employer perceptions of information literacy skills and the potential use of digital badges to represent these skills. The major themes and results found in the qualitative research articles and one survey with employers about the general preparedness of college graduates were synthesized and combined into a logical resource for the design of our questions.60 Our survey was developed through three iterations and in consultation with two information literacy content knowledge experts and the existing information literacy standards at the time of the study. Data analysis involved both descriptive and comparative statistics depending on the research question and the existing literature in the field. Our survey involved two sections of ranking questions, four sections of Likert-scale questions with the inclusion of slightly agree/slightly disagree options instead of neutral, descriptive, and demographic questions. We conducted pilot usability testing with people familiar and unfamiliar with information literacy concepts but did not descriptively analyze the results of the pilot. In addition to this, we sent the survey through multiple rounds of revisions and checks with experts in survey research and information literacy. We also sent the survey to people not involved in either of these fields to check for accuracy, simplicity, and clarity of our questions. The survey-research sampling company that administered our survey also conducted four rounds of pilot testing to ensure that the logistics of our survey made sense. The development of our survey took place over the course of three months through extensive literature review, feedback from experts in information literacy and quantitative research, and refinements through initial pilot testing. We employed a survey-research sampling company in which 188 human resource personnel of various employers nationwide were contacted electronically for possible participation in the survey. Out of the 188 possible participants, 114 were identified as eligible and completed the survey. A total of 35 people out of the 188 dropped out of the survey, and 20 had timeouts (started survey but didn’t finish it), for a 60 percent response rate. To be eligible for our survey, the employer had to hire recent college graduates for positions in their company. Not everyone who was eligible finished the survey, which eliminated them from inclusion in the data. This was not a random sample, as survey participants were all part of a panel sourced by the survey sampling company in which par- ticipants expected to receive surveys relating to their work and received compensation for completed surveys. Thus, one limitation of this form of survey research is that it is not a ran- dom sample. These companies and employees expect to receive surveys and are therefore not representative of the workforce as a whole. Thus, the results that follow have limited generalization. Survey responses were distributed and collected through an online survey tool called Redcap (the complete survey can be viewed in appendix C). As shown in the final demographic report (see tables 1 and 2), there were a variety of employer types represented. TABLE 1 Industry Field (n=114) Industry Percentage of Respondents Professional Services (n=31) 27 IT (n=19) 17 Engineering (n=9) 9 Finance (n=8) 7 Medicine (n=7) 6 Logistics (n=6) 5 Education (n=6) 5 Manufacturing (n=6) 5 Other (n=22) 20 Note. Industries with fewer than 6 participants were put into “Other” Employer Perceptions 95 Results The results of our survey are presented in order by the validity and reliability of our survey instrument tests, the demographic information, and the analysis of the survey results by research question. To estimate the reliability of our survey, a Cron- bach’s alpha was determined for the “index of reli- ability associated with the variation accounted for by the true score of the underlying construct” for the Likert scale questions.61 According to Mohsen Tavakol and Reg Dennick, a Cronbach’s alpha determines the internal consistency to see what items in the survey “measure the same concepts or construct.”62 The grouping for the Cronbach’s alpha was determined using a factor analysis. The factor analysis determined that the five factors in table 1 accounted for 66.234 percent of the variance in the 20 skills listed in the survey.63 This variance means that these five factors account for 66.234 percent of the differences from the mean (see appendix A for the full factor analysis). This table illustrates the Cronbach’s alpha for all of the groupings. Additionally, the participants in the survey had the opportunity to leave comments that are considered in the discussion section to illustrate the perspective of people taking the survey. Participants responded that they hire employees completing different levels of education. Cumulative responses (n=114) show that 33 percent of hires hold bachelor’s degrees, 18 percent hold associate’s degrees, 17 percent require no advanced degree, 14 percent require a certificate, 12 percent require a master’s degree, and 6 percent require a professional/doctorate degree. Participants receive a variety of applications for open positions, with the minimum number of applications received being 1 and the maximum 100, with a median of 25. When asked about hiring more or fewer recent college graduates in the next few years (n=112), 44 percent said they plan to hire more, 50 percent said they plan to maintain the same rate, and 6 percent plan to hire fewer entry-level college graduates. The analysis of the importance of information literacy and digital badges in the workplace are presented in relation to the research questions outlined above. Related to our first question, the information literacy competencies most valued by employers are identified. In addition, certain subskills are grouped in relation to their broader category to determine if the skills that students tend to lack are found to be valuable to employers. Using factor analysis, our 20 skill subsets were grouped into five broader categories. These five broad categories were then statistically analyzed using a two- tailed t-test to determine which skills were most valued. The purpose of a two-tailed TABLE 2 Employment Sector (n=114) Employment Sector Percentage (%) Private (n=74) 65 Public (n=33) 29 Nonprofit (n=5) 4 TABLE 3 Cronbach’s Alpha Category Cronbach’s Alpha Critical Thinking/Using Quality Information (n=114) .813 Collaboration (n=110) .607 Technology/Ethics/Self-Awareness (n=111) .879 Finding and Gathering Information (n=109) .815 Innovation (n=109) N/A 96 College & Research Libraries January 2016 t-test is to see if the means of each of the five categories are statistically different from one another and, in this case, to see which skills were statistically different from other skills. For a factor or category to be linked together, the p-values needed to be similar to one another (as will be seen below). The results are presented in figure 2. The verti- cal line represents the statistically significant difference between categories where the first three categories all had p-values that were high (p=.999, .868, and .866) and thus grouped together into one category. The bottom two categories had a p-value that was similar (p=.3106) to other categories. The skill categories valued most by employers are: innovation (which had only one skill group included), critical thinking and using quality information, and collaboration. The technology, ethics, and self-awareness category and the finding and gathering information category were less valued by employers. Further information on the categories and their characteristics can be found in the appendix. Using the ACRL information literacy standards performance indicators as guidance, a ranking-list question was administered that asked employers which skills were im- portant for the workforce that recent college graduates were unlikely to possess. These skills were then placed under one of the information literacy skill categories based on similarities between the category and a particular information literacy skill, such as critical thinking. Table 4 shows the results of this. The average importance number was found by taking all of the employers’ responses to a specific category and averaging it. For example, to find the average importance for innovation, all of the questions revolv- ing around innovation were averaged. From these data, we note that employers value critical thinking and using quality information, but the subskills of applying knowledge to real-world context and finding patterns and making connections are consistently missing in recent college graduates. By focusing on areas where there are mismatches between graduates’ skills and employers’ expectations, digital badge content can fill needs that are not currently being met by traditional instruction and courses. The second research question sought to descriptively answer which collegiate ex- periences were most important to employers when they evaluate potential candidates. The use of a two-tailed, independent, two sample t-test grouped these experiences into three categories based on participant responses. Table 5 shows the result of this test, and FIGURE 2 Average Importance Rating for Information Literacy Categories Employer  Perceptions  of  Info  Lit  and  Digital  Badges       34         Figure 2 4.4   4.6   4.8   5.0   5.2   5.4   Informa(on  Literacy  Categories   Average  Importance  Ra(ng  for     Informa(on  Literacy    Categories   InnovaDon  (n=114)   CriDcal  Thinking  and  Using   Quality  InformaDon   (n=110)   CollaboraDon  (n=111)   Technology,  Ethics,  and   Self-­‐awareness  (n=109)   Finding  and  Gathering   InformaDon  (n=109)   Employer Perceptions 97 the implications for the utility of digital badges based on the participants’ answers of important activities in college are considered. Based on the p-values, it was determined which skills were very important, moderately important, and less important. Table 6 shows the p-values that determined the grouping of skills in table 5. Any p-value above .09 led to a grouping of that skill along with the skill it crossed with in the table. For the design of digital badges, the most significant result from this test is that the ability to demonstrate extra learning beyond the classroom was grouped with TABLE 4 Sub Skills of Information Literacy Skill Categories, (n=114) IL Skill Category Average Importance Likelihood of Possession Innovation 5.2 Ability to solve problems on their own when there is an unclear outcome (50%) Critical Thinking and Using Quality Information 5.2 Ability to apply knowledge to real-world contexts (29%) Ability to find patterns and make connections (18%) Collaboration 5.2 Ability to use collaborative skills to help solve problems (50%) Ability to learn how to work with people from diverse backgrounds (29%) Technology, Ethics, and Self- Awareness 4.9 A desire to learn new skills on the job (49%) Ability to identify and understand ethical information issues (38%) Finding and Gathering Information 4.7 Job-specific knowledge or training (61%) TABLE 5 Collegiate Experiences Ranked in Relation to Hiring Importance (Approximate important score, scale 1–8) Very Important Importance Score College Courses Related to Candidate’s Major 4.76 Ability to demonstrate extra learning beyond the classroom 4.60 Moderately Important Leadership roles in organizations 4.06 Excellent grades 3.72 Internships 3.70 Evidence of experience with technology tools 3.37 Less Important E-portfolio standardized test score 2.50 Test score 2.02 98 College & Research Libraries January 2016 college courses as the most important collegiate experiences. This indicates there is value to students for earning digital badges beyond their college courses. The final research question seeks to answer whether there is a significant percent- age of human resource personnel who are interested in using digital badges to certify skills. Figure 2 shows the results of a question asking human resource personnel if a more specific representation of student skills is desired over grades and transcripts. From the descriptive statistics in figure 3, it can be seen that the last two statements FIGURE 3 Employers’ Evaluation of Current Applicants Grades and GPA have a high correlation with preparedness for the workplace Collaborating in a variety of contexts is extremely important to the workplace A more speci�c representation of a student’s speci�c skills and competencies would be helpful when evaluating The ability to �nd and evaluate information is important for adapting to change in the workplace TABLE 6 Two-tailed T-test Results College Courses Leadership Roles Extra Learning Excellent Grades E-portfolio Experience with Technology Standardized Test Scores Internships .0006 .2493 .0033 .9481 .0002 .3128 <.0001 College Courses — .0094 .5343 <.0001 <.0001 <.0001 <.0001 Leadership Roles — — .0439 .2110 <.0001 .0189 <.0001 Extra Learning — — — .0009 <.0001 <.0001 <.0001 Excellent Grades — — — — <.0001 .2241 <.0001 E-portfolio — — — — — .0033 .0976 Experience with Technology — — — — — — <.0001 Employer Perceptions 99 had an agreement rate of 96 percent and 95 percent respectively. In contrast, 24 percent of respondents disagreed that there is a high correlation between grades and be- ing prepared for the workplace. The third skill especially points strongly to the value of badges in providing ways that specific skills and competencies can be evaluated by employers. When asked specifically if employers would be interested in using a digital badge to evaluate skills of recent college graduates, the response was extremely promis- ing. This question was posed with a visual representation of a digital badge and a brief description of what content was included in the presentation of this badge. The results are shown in table 7. The only industries that had negative responses to this question came from logistics, accounting, and medicine, but there was no significant difference between employment fields and interest in digital badges. The high percentage of respondents who said “maybe” indicates that future education about digital badges would be necessary to improve adoption. Discussion Previous research has shown that the contexts for information literacy in academia and the workplace are different and that the skills students are learning in higher education are not necessarily the same skills they need in the workplace.64 Through the results of our survey, three broad themes can be identified to inform educators about the information literacy skills desired in the workplace. The first theme validates what previous information literacy research has shown, which is that information literacy skills are important to employers. The second theme is that there are fundamental job skills that employers expect from recent college graduates that are seemingly lacking and have a basis in information literacy. The final theme is that employers would like a more specific representation of skills or abilities to better evaluate college graduates for potential jobs. It is clear that information literacy skills are considered important to employers from figure 1, as even the lowest-ranked skill—finding and using information—still has an average importance score of 4.7. When analyzing the results further, it can be seen that there are clear and specific skills that are lacking in recent college graduates that employers want. From table 4, the three areas where employers least expect students to have competency are the ability to find patterns and make connections (18%), the ability to apply knowledge to real-world contexts (29%), and the ability to work with people from diverse backgrounds (29%). All three of these more granular abilities fall under the two broader categories of critical thinking and using quality information and collaboration. These two broad categories were grouped as the most valued information literacy areas by employers in our analysis. The combination of findings—that two of the most-valued skill categories and three of the subskills are lacking for many recent college graduates—makes them prime candidates around which to design digital badge activities. This information combined with the data about the potential worth of digital badges to employers (only 5 percent of employers said they would not be interested in digital badges) holds much promise for using badges to fill a void in information literacy and metaliteracy instruction. Figure 2 provides broader information about employers’ viewpoints about grades being valuable for indicating student skills in the workplace and if more specific repre- TABLE 7 Level of Interest in Digital Badges (n=114) Interest in Using Digital Badges Percentage Yes 33% Maybe, have to learn more 62% No 5% 100 College & Research Libraries January 2016 sentations of student skills would be helpful. Only 38 percent of employers agree that grades and GPA have a high correlation with preparedness in the workplace, while 79 percent of employers desire a more specific representation of students’ skills when evaluating them for a potential job. To add even more evidence to the worth of digital badges, the two-tailed t-test results presented in table 6 grouped college courses in a candidate’s major (4.76) as being important for hiring decisions along with the ability to demonstrate extra learning beyond the classroom (4.60). In other words, if students earn digital badges beyond their prescribed coursework, this is a clear and visual way that they can show employers they are learning beyond the classroom. Participants had the opportunity to insert open-ended comments after various sec- tions and at the end of the survey. A few of these comments were additional evidence of what a human resource department may consider when hiring a recent college graduate. Such evidence is illustrated by a comment where a participant noted that he or she would “consider a person’s personality.” However, the vast majority of the comments received were positive in relation to the survey. For example, one partici- pant responded that the survey “was very telling, gave me a lot more to think about when hiring.” Additionally, numerous participants commented that they found the survey interesting or informative. These additional comments give user-validation to the quality of our survey instrument and the types of questions asked in the survey. Conclusions Digital badges are a burgeoning way of documenting and certifying student skills and offer the potential to alleviate some of the long-standing concerns with informa- tion literacy such as the lack of consistent instruction. However, if employers do not recognize digital badges as a valid way for certifying student competencies—as they do today with degrees and transcripts— then even the most valued skills earned by way of a badge may not receive the credit they deserve. From table 7, we see that only 5 percent of employers were not interested in using digital badges as a way to certify student skills, which demonstrates that badges clearly hold potential for representing these skills. However, the very large percentage of respondents who said “maybe, have to learn more” (62%) emphasizes the need for widespread enhanced awareness or education to increase the adoption rate. Rogers notes that there is often a critical mass, especially with communication technologies, before the system can be self-sustaining in adoption of the innovation.65 Knowledge of an innovation is an important precursor to the actual implementation of that innovation, so information will need to be shared with employers about digital badges.66 Programs like the “21st Century Skills Badging Challenge,” which involves a col- lective of universities and industries exploring how to best implement badges in a manner that is useful to employers, is a promising step in this direction toward broader adoption of badges.67 There are three areas we can see that could benefit from further research. Additional research with larger sample sizes of employers in specific industries would allow re- sults to be broken down into information literacy skills desired by various job fields as well as significant differences in awareness and acceptance of digital badges among different fields. A second area for further research is to increase sample breadth to be inclusive of all of the company types that hire recent college graduates. The final area for further research is related to the fact that only broad and general meanings can be constructed from a survey. In response to these limitations, we urge other researchers to build on the quantitative findings from our survey by conducting more in-depth, possibly qualitative research with employers on their perspectives of digital badges and about information literacy skills needed within specific fields or careers. Employer Perceptions 101 The findings of this preliminary research prove promising for information literacy, metaliteracy, and digital badges. This research validated previous findings of other researchers that employers continue to find information literacy skills valuable. More detailed research into identifying the gaps between current information literacy skills being taught and what skills employers value will continue to be critical to the profession as well as the adoption of digital badges or other forms of competency-based learning. As digital badges gain greater adoption, continued research into the buy-in and use of digital badges by employers will be needed since student demand for badges will depend on employers’ adoption. Additionally, research about students’ perceptions of digital badges and their utility will be an important aspect to consider. All in all, this area of research is ready for further inquiry. COMPONENT MATRIX* Component 1 2 3 4 5 Ability to innovate (come up with novel ways to solve problems or generate solutions) 0.544 0.564 Ability to work in a multi-person team (greater than 2 people) 0.498 0.554 Ability to think critically and use problem- solving skills 0.400 0.323 0.527 Ability to effectively use online free and open sources of information (nonsubscription sources) for gaining information and knowledge 0.523 0.387 –0.492 0.353 Ability to use online subscription–based sources paid for by your organization for gaining information and knowledge 0.601 –0.370 0.406 Ability to demonstrate personal responsibility for continuous learning on the job 0.492 0.432 –0.344 Ability to synthesize information from multiple resources 0.641 0.305 Ability to devise a plan for how to find quality and reliable information 0.664 0.390 –0.334 Ability to execute a plan to find quality and reliable information 0.574 0.330 –0.367 –0.428 Acquisition of social or “soft skills,” such as willingness to approach others for help 0.419 0.327 0.421 Ability to broaden perspective and accept other viewpoints 0.390 0.461 Appendix A. Factor Analysis 102 College & Research Libraries January 2016 COMPONENT MATRIX* Component 1 2 3 4 5 Ability to gather information from multiple sources (such as social media, personal contacts, e-mail, online sources, and so on) and provide this information to coworkers 0.601 0.355 0.415 Ability to find information that is not available online (such as using print and other sources) 0.644 –0.354 The ability to conduct work via mobile technologies is increasingly important in the workplace (such as using a tablet/phone to find information immediately) 0.545 –0.418 The ability to find information from social resources, such as other coworkers or colleagues, is important to finding the answers to problems with unclear outcomes 0.584 –0.482 0.428 The ability to look for answers to questions using web resources is a critical skill to success in the workplace 0.673 –0.452 The ability to be an aware learner is important for building personal development 0.512 –0.653 Awareness of one’s strengths and weaknesses as it relates to skills needed for the job 0.487 –0.667 Ethical awareness about information use including copyright, fair use, and citation 0.532 –0.628 The ability to solve problems rapidly is more important than conducting a thorough evaluation and analysis of options 0.398 –0.491 Extraction Method: Principal Component Analysis. *5 components extracted. Appendix A. Factor Analysis Reliability Statistics Cronbach's Alpha N of Items .813 5 Appendix B: Cronbach Alpha Employer Perceptions 103 Survey for Employer Perceptions Survey for Employer Perceptions of Skills Needed for New Hires Thank you for taking the time to complete our survey. This survey is meant to gauge your perceptions of the skills and competencies needed by new college graduates when they start a new job at your company. This survey should take about 10–15 minutes to complete, and your answers will help us make decisions on designing tasks for students to complete to prepare them for the workforce. In the survey you will see the term “recent college graduate.” For the purpose of this survey, recent college graduate is defined as someone who has graduated college within the past three years. Thank you. College Graduate Hiring Patterns Do you hire college graduates for positions that are open in your company (associate’s degree or higher OR specialized education such as LPN, RN, dental hygenist, etc.)? q Yes q No The following question asks you to weigh the importance of certain skills needed by ALL college graduates (not only recent) hired at your company. Least Important Of Little Importance Moderately Important Important Very Important Not Applicable Ability to innovate (come up with novel ways to solve problems or generate solutions) q q q q q q Ability to work in a multiperson team (greater than 2 people) q q q q q q Ability to think critically and use problem-solving skills q q q q q q Ability to effectively use online free and open sources of information (nonsubscription sources) for gaining information and knowledge q q q q q q Ability to effectively evaluate these free and open sources for credibility and quality of information q q q q q q 104 College & Research Libraries January 2016 Least Important Of Little Importance Moderately Important Important Very Important Not Applicable Ability to use online subscription-based sources paid for by your organization for gaining information and knowledge q q q q q q Ability to demonstrate personal responsibility for continuous learning on the job q q q q q q The following section asks you to rank collegiate experiences as they relate to jobs at your company. Please rank the following collegiate experiences in terms of how important they are in hiring a candidate for a position at your company using the scale of 1–8. In this scale, 1 is the most important (bold or italics) experience and 8 is the least important (bold/ italics) experience. If you rank two experiences equally, they can have the same number (e.g. two #1’s), but then you would not have a #8 ranking. _____ Internships _____ College courses related to candidate’s major or degree _____ Leadership roles in organizations (such as student clubs, fraternities) _____ Ability to demonstrate extra learning beyond the classroom (such as civic engagement, volunteering) _____ Excellent Grades (above a 3.5) _____ E-portfolio (online display of candidate’s work) _____ Evidence of experience with technology tools (such as digital media, online collaboration tools) _____ Score on standardized tests (academic or work-related tests) Please list any other experiences not listed in the above ranking that you feel are important: Employer Perceptions 105 Please rank the following skills from least important to very important for recent college graduate (within the past three years) hires to possess to be effective on the job at your company. Least Important Of Little Importance Moderately Important Important Very Important Not Applicable Ability to synthesize information from multiple resources q q q q q q Ability to devise a plan for how to find quality and reliable information q q q q q q Ability to execute a plan to find quality and reliable information q q q q q q Acquisition of social or “soft skills,” such as willingness to approach others for help q q q q q q Ability to broaden perspective and accept other viewpoints q q q q q q Ability to gather information from multiple sources (such as social media, personal contacts, e-mail, online source.) and provide this information to coworkers q q q q q q Ability to find information that is not available online (such as using print and other sources) q q q q q q 106 College & Research Libraries January 2016 The following questions ask you to consider skills required on the job and how important they are to the success of an employee. Based on the following scale (strongly agree to strongly disagree), select your answer for the following statements related to skills required in the workplace: Strongly Agree Agree Slightly Agree Slightly Disagree Disagree Strongly Disagree Not Applicable The ability to conduct work via mobile technologies is increasingly important in the workplace (such as using a tablet/phone to find information immediately) q q q q q q q The ability to find information from social resources, (such as other coworkers or colleagues) is important to finding the answers to problems with unclear outcomes q q q q q q q The ability to look for answers to questions using web resources is a critical skill to success in the workplace q q q q q q q The ability to be an aware learner is important for building personal development q q q q q q q Awareness of one’s strengths and weaknesses as it relates to skills needed for the job q q q q q q q Ethical awareness about information use including copyright, fair use, and citation q q q q q q q The ability to solve problems rapidly is more important than conducting a thorough evaluation and analysis of options q q q q q q q Employer Perceptions 107 Based on the scale (strongly agree to strongly disagree) select your answer for the following statements related to skills required in the workplace: Strongly Agree Agree Slightly Agree Slightly Disagree Disagree Strongly Disagree Not Applicable Transcripts are too vague a representation of the skills that students may have when applying for jobs q q q q q q q Grades and GPA have a high correlation with preparedness for the workplace q q q q q q q Collaborating in a variety of contexts is extremely important to the workplace q q q q q q q A more specific representation of a student’s specific skills and competencies would be helpful when evaluating potential candidates q q q q q q q The ability to find and evaluate information is important for adapting to change in the workplace q q q q q q q Yes No Not Sure My company uses proprietary databases to find information q q q Entry-level workers are granted access to these proprietary databases q q 108 College & Research Libraries January 2016 The following question asks you to rank recent college graduates’ skills when they start the job. In this section, consider what skills new hires have when they enter the workforce in order of most likely to least likely to have. Please rank these skills using a scale of 1 to 8, 1 is the most likely to have this skill and 8 is the least likely to have this skill. If you rank two experiences equally, they can have the same number (such as two #1’s), but then you would not have a #8 ranking. _____ Job-specific knowledge/training _____ Ability to identify and understand ethical information issues including privacy, copyright, how to use social media, and so on. _____ Ability to use collaborative skills to help solve problems _____ Ability to solve problems on their own when there is an unclear outcome _____ Ability to learn how to work with people from diverse backgrounds or cultures _____ A desire to learn new skills on the job _____ Finding patterns and making connections between activities, concepts, and ideas _____ Ability to apply knowledge to real-world contexts If there is another skill you notice is present/not present in a new hire, please list it here: Microcredentialing Microcredentialing is a way to recognize skills learned in a variety of fields and formats. In this section, we are interested in your perceptions of the microcredential- ing technique called digital badges. Digital badges are a “clickable credential” that show that someone has a specific skill and also provide detailed information about how that person earned that specific skill. Digital badges make this information available to others to re- view, such as to a potential employer. An image of what a digital badge might look like appears to the right. q Yes q Maybe, have to learn more q No If a potential new hire approached you with visual representation of his or her skills such as the above image (a digital badge) that represented a skill the candidate had achieved, would you be interested in using the badge to determine the qualifications of the candidate? Employer Perceptions 109 Would you be more likely to click on this visual representation if you could see more details such as the issuer of the visual representation, the work the student did to earn it, the date issued, the ex- piration of the skill? How many of these visual representations would you be willing to evaluate for hiring a candidate? If there were a tool that allowed you to filter an ap- plicant’s badges by keyword or tag to quickly iden- tify specific badges or skills, would you be more likely to adopt badges into your hiring practices? q Yes q Maybe, have to learn more q No q Yes q Maybe, have to learn more q No q 0 q 1–5 q 6–10 q 11–15 q 16–20 q greater than 21 The following section details your organization’s demographic information and hiring patterns. What employment sector is your organization classified as? Please fill in the employment sector for your orga- nization. If you are not sure, please describe your organization briefly. __________________________________________ __________________________________________ __________________________________________ a. Private b. Public c. Not for profit d. Social Enterprise (FairTrade, and the like.) For profit com- panies concerned with social issues e. Other What field or category best describes your organization? (such as engineering, IT, government, education) ___________________________________________________ If there is a subfield for your organization, please enter it here. (such as higher educa- tion or K–12 education for education) _______________________________________ For your most recent opening, how many resumes/applications did you receive from potential hires? ___________________________________________________________ In the next two years, do you plan to: a. Hire more entry-level college graduate candidates b. Maintain the same amount of entry- level college graduate candidates c. Hire fewer entry-level college gradu- ates d. We do not hire recent college gradu- ates for entry-level positions 110 College & Research Libraries January 2016 The following questions ask you to consider the percentage of new hires who hold various degrees. Since you cannot enter the character % in the form, please make sure your total equals 100. _____ Certificate (such as paralegal, dental hygienist, LPN) _____ Associate Degree _____ Bachelor Degree _____ Master Degree _____ Professional Degree (such as MD, PhD, DO) _____ No advanced degree required Please click the link below to have your response recorded. Select this link to have your response recorded. Please click the link below to have your response recorded. (If you did not click this on the previous page). Select this link to have your response recorded. Please use the following section to add any comments you may have about the survey (ease of use, difficulty, confusion) or other factors you consider when hiring a new employee. Comments: This confirms the end of the survey. Thank you very much for taking the time to com- plete our survey. The following link takes you to another brief survey to fill out your contact information for future potential contact. It is not tied to this survey in any way, is confidential, and is completely voluntary. Thank you again for taking our survey. We value your time and input. Employer Perceptions 111 Notes 1. Jennifer Kirton and Lyn Barham, “Information Literacy in the Workplace,”Australian Library Journal 54, no. 4 (2005): 365–76; Anne Lloyd and Kirsty Williamson, “Towards an Understanding of Information Literacy in Context: Implications for Research,” Journal of Librarianship and Infor- mation Science 40, no. 1 (2008): 3–12; Carmel O’Sullivan, “Is Information Literacy Relevant in the Real World?” Reference Services Review 30, no. 1 (2002): 7–14. 2. Christine Susan Bruce, “Workplace Experiences of Information Literacy,” International Jour- nal of Information Management 19, no. 1 (1999): 33–47; Douglass K. Hawes, “Information Literacy and the Business Schools,” Journal of Education for Business 70, no. 1 (1994): 54–61; Alison J. Head, Michele Van Hoeck, Jordan Eschler, and Sean Fullerton, “What Information Competencies Matter in Today’s Workplace?” Library and Information Research 37, no. 114 (2013): 74–104. 3. Thomas P. Mackey and Trudi E. Jacobson, “Reframing Information Literacy as a Metalit- eracy,” College & Research Libraries (2011): 62–78. 4. Barry Fishman, Ronald W. Marx, Phyllis Blumenfeld, Joseph Krajcik, and Elliot Soloway, “Creating a Framework for Research on Systemic Technology Innovations,” Journal of the Learning Sciences 13, no. 1 (2004): 43–76; David H. Jonassen, Chad Carr, and Hsiu-Ping Yueh, “Computers as Mindtools for Engaging Learners in Critical Thinking,” Tech Trends 43, no. 2 (1998): 24–32. 5. David Gibson, Nathaniel Ostashewski, Kim Flintoff, Sheryl Grant, and Erin Knight. “Digital Badges in Education,” Education and Information Technologies (2013): 2. 6. Neil B. Mehta, Alan L. Hull, James B. Young, and James K. Stoller, “Just Imagine: New Paradigms for Medical Education,” Academic Medicine 88, no. 10 (2013): 1418–23; Ferial Khaddage, Rowland Baker, and Gerald Knezek, “If Not Now! When? A Mobile Badge Reward System for K–12 Teachers,” Proceedings of Society for Information Technology and Teacher Education International Conference (2012): 2900–05; Christopher Gamrat, Heather Toomey Zimmerman, Jaclyn Dudek, and Kyle Peck, “Personalized Workplace Learning: An Exploratory Study on Digital Badging within a Teacher Professional Development Program,” British Journal of Educational Technology 45, no. 6 (2014): 1136–48. 7. Goldie Blumenstyk, “If B.A.’s Can’t Lead Graduates to Jobs, Can Badges Do the Trick?” Chronicle of Higher Education (Mar. 2, 2015), available online at http://chronicle.com/article/If-BA- s-Can-t- Lead/228073/?cid=wc&utm_source=wc&utm_medium=en [accessed 19 March 2015]. 8. Everett M. Rogers, “Diffusion of Innovations,” New York (2003): 1–472; Mozilla Open Badges, “Open Badges,” available online at http://openbadges.org/ [accessed 6 December 2014]. 9. Jonathan Finklestein, Erin Knight, and Susan Manning, “Using Digital Badges for Adult Learners” (Jul 6, 2013), available online at https://lincs.ed.gov/publications/pdf/AIR_Digital_Badge_ Report_508.pdf [accessed 15 February, 2015]. 10. Mozilla, “Mozilla Wiki, Badges” (Oct. 30, 2014), available online at https://wiki.mozilla. org/Badges [accessed 9 December 2014]. 11. Frank Catalano, “Digital ‘Badges’ Emerge as Part of Credentialing’s Future,” Institute for Credentialing Excellence, available online at www.credentialingexcellence.org/p/cm/ld/fid=203 [accessed 9 December 2014]. 12. Finklestein et al., “Using Digital Badges.” 13. University of Maryland Libraries, “The TLC Digital Badge Program,” available online at www.lib.umd.edu/tlc/tlc-learn [accessed 13 March 2015]. 14. Metaliteracy Learning Collaborative, “Metaliteracy,” available online at http://metaliteracy. learningtimes.net [accessed 13 March 2015]. 15. Mozilla, “Mozilla Wiki, Badges.” 16. Association for College and Research Libraries, “Keeping Up with…Digital Badges for Instruction,” available online at www.ala.org/acrl/publications/keeping_up_with/digital_badges [accessed 6 December 2014]. 17. Head et al., “What Information Competencies Matter,” 74–104. 18. Amy Bruckman, “Learning in Online Communities,” in The Cambridge Handbook of the Learning Sciences, ed. Robert Keith Sawyer (New York: Cambridge University Press, 2006): 461. 19. Rogers, “Diffusion of Innovations,” 1–472. 20. Ibid. 21. Head et al., “What Information Competencies Matter”; Jason Sokoloff, “Information Lit- eracy in the Workplace: Employer Expectations,” Journal of Business & Finance Librarianship 17, no. 1 (2012): 1–17; Louise Klusek and Jerry Bornstein, “Information Literacy Skills for Business Careers: Matching Skills to the Workplace,” Journal of Business & Finance Librarianship 11, no. 4 (2006): 3–21; Theresa M. Conley and Esther L. Gil, “Information Literacy for Undergraduate Busi- ness Students: Examining Value, Relevancy, and Implications for the New Century,” Journal of Business & Finance Librarianship 16, no. 3 (2011): 213–28; Hart Research Associates (HRA) (2010), 112 College & Research Libraries January 2016 “Raising the Bar: Employers’ Views on College Learning in the Wake of the Economic Downturn (survey completed for The Association of American Colleges and Universities), available online at https://www.aacu.org/leap/documents/2009_EmployerSurvey.pdf [accessed 17 January 2014]. 22. HRA, “Raising the bar”; Hawes, “Information Literacy and the Business Schools”; Head et al, “What Information Competencies Matter”; Anne Lloyd and Kirsty Williamson, “Towards an Understanding of Information Literacy in Context: Implications for Research,” Journal of Li- brarianship and Information Science 40, no. 1 (2008): 3–12. 23. Head et al., “What Information Competencies Matter,” 74–104. 24. Yvonne Mery, Jill Newby, and Ke Peng, “Why One-Shot Information Literacy Sessions Are Not the Future of Instruction: A Case for Online Credit Courses,” College and Research Libraries (2011): crl-271. 25. American Library Association, “Information Literacy Competency Standards for Higher Education” (2000), available online at. www.ala.org/acrl/standards/informationliteracycompetency [accessed 13 December 2013] 26. Clarence Maybee, “Undergraduate Perceptions of Information Use: The Basis for Creating User-Centered Student Information Literacy Instruction,” Journal of Academic Librarianship 32, no. 1 (2006): 79–85. 27. Ibid. 28. Mackey and Jacobson, “Reframing Information Literacy,” 62–78. 29. Note that the original definition for information literacy is: Determine the extent of informa- tion needed, access the needed information effectively and efficiently, evaluate information and its sources critically, incorporate selected information into one’s knowledge base, use information effectively to accomplish a specific purpose, and understand the economic, legal, and social issues surrounding the use of information, and access and use information ethically and legally (ALA, 2000, para. 2).The newly expanded definition is: Information literacy is the set of integrated abili- ties encompassing the reflective discovery of information, the understanding of how information is produced and valued, and the use of information in creating new knowledge and participating ethically in communities of learning. American Library Association, “Framework for Information Literacy in Higher Education” (2015), available online atwww.ala.org/acrl/standards/ilframework [accessed 13 March 2015]. 30. Mackey and Jacobson, “Reframing Information Literacy.” 31. George Siemens, “Connectivism: A Learning Theory for the Digital Age,” International Journal of Instructional Technology and Distance Learning 2, no. 1 (2005): 3–10. 32. Ibid. 33. Tjeerd Plomp, “Preparing Education for the Information Society: The Need for New Knowledge and Skills,” International Journal of Social Media and Interactive Learning Environments 1, no. 1 (2013): 3–18. 34. Douglas Thomas and John Seely Brown, A New Culture of Learning: Cultivating the Imagina- tion for a World of Constant Change, vol. 219 (Lexington, Ky.: CreateSpace, 2011), 1–126. 35. Ronald E. Anderson, “Implications of the Information and Knowledge Society for Educa- tion,” in International Handbook of Information Technology in Primary and Secondary Education, eds. Joke Voogt and Gerald Knezek (New York: Springer Science + Business Media LLC, 2008), 10. 36. The Partnership for 21st Century Skills, available online at www.p21.org/ [accessed 14 August 2014]. 37. R. Keith Sawyer, “Educating for Innovation,” Thinking Skills and Creativity 1, no. 1 (2006): 41–48. 38. Carmel O’Sullivan, “Is Information Literacy Relevant in the Real World?” Reference Services Review 30, no. 1 (2002): 7–14. 39. Ibid. 40. Sawyer, “Educating for Innovation.” 41. American Library Association, “Framework for Information Literacy in Higher Education.” 42. University of Maryland Libraries and Metaliteracy Learning Collaborative. 43. Rita Kop and Adrian Hill, “Connectivism: Learning Theory of the Future or Vestige of the Past?” International Review of Research in Open & Distance Learning 9, no. 3 (2008). 44. Sheryl Grant, “Badges: Show What You Know,” Young Adult Library Services 12, no. 2 (2014): 28–32. 45. Razvan Rughinis, “Talkative Objects in Need of Interpretation: Rethinking Digital Badges in Education,” in CHI’13 Extended Abstracts on Human Factors in Computing Systems (New York: Association for Computer Machinery, 2013), 2099–2108. 46. Michelle Kathleen Dunaway, “Connectivism: Learning Theory and Pedagogical Practice for Networked Information Landscapes,” Reference Services Review 39, no. 4 (2011): 675–85. 47. Siemens, “Connectivism: A Learning Theory for the Digital Age.” 48. Annemarie Palinscar and Barbara Ladewski, “Literacy and the Learning Sciences,” in Employer Perceptions 113 The Cambridge Handbook of the Learning Sciences, ed. Robert Keith Sawyer (New York: Cambridge University Press, 2006), 299–316. 49. Plomp, “Preparing Education for the Information Society,” 5. 50. Susan M. Barnett and Barbara Koslowski, “Adaptive Expertise: Effects of Type of Experience and the Level of Theoretical Understanding It Generates,” Thinking & Reasoning 8, no. 4 (2002): 237–67. 51. Ibid. 52. Chris Dede, “Scaling Up: Evolving Innovations beyond Ideal Settings to Challenging Contexts of Practice,” in The Cambridge Handbook of the Learning Sciences, ed. Robert Keith Sawyer (New York: Cambridge University Press, 2006), 535–50. 53. Kyle Peck and Alison A. Carr, “Restoring Public Confidence in Schools through Systems Thinking,” International Journal of Educational Reform 6, no. 3 (1997): 316–23. 54. Rogers, “Diffusion of Innovations,” 1–472. 55. Ibid. 56. Alison J. Head, Michele Van Hoeck, Jordan Eschler, and Sean Fullerton, “What Information Competencies Matter in Today’s Workplace?” Library and Information Research 37, no. 114 (2013): 74–104; Sokoloff, “Information Literacy in the Workplace”; HRA, “Raising the Bar.” 57. C. Hoadley, “What Is a Community of Practice and How Can We Support It?” in Theoreti- cal Foundations of Learning Environments, eds. David Jonassen and Susan Land (Routledge, 2012), 287–300; J.G. Greeno, “Learning in Activity,” in The Cambridge Handbook of the Learning Sciences, ed. Robert Keith Sawyer (New York: Cambridge University Press, 2006), 54–61, 79–96; Etienne Wenger, Communities of Practice: Learning, Meaning, and Identity (Cambridge University Press, 1998), 1–309. 58. Annemaree Lloyd, “Trapped between a Rock and a Hard Place: What Counts as Information Literacy in the Workplace and How Is It Conceptualized?” Library Trends 60, no. 2 (2011): 277–96. 59. Hawes, “Information Literacy and the Business Schools”; Head et al., “What Information Competencies Matter”; Sokoloff, “Information Literacy in the Workplace”; Bruce, “Workplace Experiences”; HRA, “Raising the Bar.” 60. HRA, “Raising the Bar.” 61. J. Reynaldo A. Santos. “Cronbach’s Alpha: A Tool for Assessing the Reliability of Scales,” Journal of Extension 37, no. 2 (1999): 1–5. 62. Mohsen Tavakol and Reg Dennick, “Making Sense of Cronbach’s Alpha,” International Journal of Medical Education 2 (2011): 53–55. 63. Jamie DeCoster, “Overview of Factor Analysis” (Aug. 1, 1998), available online at www. stat-help.com/factor.pdf [accessed 9 December 2014]. 64. Bruce, “Workplace Experiences”; Hawes, “Information Literacy and the Business Schools”; Head et al., “What Information Competencies Matter”; Sokoloff, “Information Literacy in the Workplace.” 65. Rogers, “Diffusion of Innovations,” 1–472. 66. Ibid. 67. Blumenstyk, “If B.A.’s Can’t Lead Graduates to Jobs, Can Badges Do the Trick?”