Introduction

Information literacy and other generic academic skills are generally considered essential for successful student transition at university (Fergy, Heatley, Morgan, & Hodgson, 2008; Goldfinch & Hughes, 2007; Hafford-Letchfield, 2007; Moore, Brewster, Dorroh, & Moreau, 2002; Ooms, Fergy, Marks-Maran, Burke, & Sheehy, 2013). Information literacy is defined as “an understanding and set of abilities enabling individuals to recognize when information is needed and have the capacity to locate, evaluate, and use effectively the needed information” (Bundy, 2004, p. 3). Academic skills are a broader domain, focused on generic educational abilities such as reading, writing, citation skills (Hitch et al., 2012), critical thinking, problem solving and information literacy (Gunn, Hearne, & Sibthorpe, 2011), technology skills, time management, communication, and working with others (Goldfinch & Hughes, 2007). On many levels, information literacy and academic skills are not discrete domains (for example, effective use of information and citation skills) and development of such skills are frequently targeted together (Bailey et al., 2007; Cassar, Funk, Hutchings, Henderson, & Pancini, 2012; Pryjmachuk, Gill, Wood, Olleveant, & Keeley, 2012).

Data has shown first year heath science students are underprepared for university study in the areas of information literacy (Birmingham et al., 2008; Dubicki, 2013; Moore et al., 2002) and academic skill (Palmer, Levett-Jones, Smith, & McMillan, 2014; Sacre & Nash, 2010; Stevens & Miretzky, 2012). Recent papers suggest students have the greatest chance of acquiring such skills when strategies are embedded into course curricula (Boruff & Thomas, 2011; Chanock, Horton, Reedman, & Stephenson, 2012; Gunn et al., 2011; McWilliams & Allan, 2014; Shorten, Wallace, & Crookes, 2001), as embedding allows for the delivery in a meaningful context, thereby providing opportunity for students to develop the skills in the course of their discipline specific study program (Chanock, 2013; Chanock et al., 2012). Increasingly, embedded strategies are developed collaboratively with academic language and learning and information literacy specialists working together with discipline academics (Ambery, Manners, & Smith, 2005; Einfalt & Turley, 2009; Rae & Hunn, 2015; Wilkes, Godwin, & Gurney, 2015).

Previous systematic reviews on the teaching of information literacy and academic skills to undergraduate students, particularly in health science disciplines, are limited. Koufogiannakis and Wiebe (2006) investigated the effectiveness of teaching methods for delivering information literacy to undergraduate students. It was concluded that computer assisted instruction is as effective as traditional methods, and self-directed independent learning and traditional instruction are better than no instruction. While not limited to undergraduate students, similarly, Zhang, Watson and Banfield (2007) showed that face-to-face delivery was equally as effective as computer assisted instruction for developing the information literacy skills of academic library patrons. More specifically, for health and medical students and clinicians there is limited evidence that information skills training improves skills, and insufficient evidence to determine the most effective training methods (Brettle, 2003). For academic skills, Oermann et al. (2015) found that despite a wide range of educational strategies for developing nursing students’ and practitioners’ writing, the majority of individual studies reporting on such strategies do not provide estimates of effectiveness, with the authors concluding that the best strategies to improve writing ability could not be identified.

The view that the best approach to develop information literacy and academic skills is to embed these within curricula is often based on expert opinion or descriptive accounts of strategies rather than empirical research, and where available, empirical evidence is often subject to methodological bias (Pryjmachuk et al., 2012). Furthermore, despite the notion that embedded strategies are preferred practice, strategies adjunct to the curricula continue to be frequently offered (for example, see Bailey et al., 2007; Edwards & O'Connor, 2011; Fenton-Smith & Frohman, 2013). Additionally, it has been specifically identified that there is a knowledge gap in relation to academic language and learning support for the health science disciplines (Fenton-Smith & Frohman, 2013). In this context, the following paper is a review of the current literature evaluating information literacy and academic skills support strategies for first year health science students.

Objective

The specific objective of this review is to identify evidence for best practice to support the development of first year university health science students’ information literacy and academic skills using a systematic review framework. Here, health science students include those studying health professions, excluding medicine. While systematic reviews are not common place in higher education (Bearman et al., 2012), this approach is beneficial as it provides evidence via a logical, comprehensive synthesis and appraisal of literature on the topic (Bearman et al., 2012; Hammick, Dornan, & Steinert, 2010).

Methods

The current systematic review was planned and conducted using a protocol developed by the two authors based on guidelines and previous work involving systematic review methodology in health and higher education (Bearman et al., 2012; Hammick et al., 2010; Steinert et al., 2006). The planned protocol was undertaken without significant variation and is represented by the methods reported here.

Information Sources

The following electronic databases were searched: Academic Search Premier, CINAHL Plus with Full Text, ERIC, Education Research Complete, PsycINFO, SPORTDiscus with Full Text, the Psychology and Behavioral Sciences Collection and ProQuest. In addition, hand searching of the Journal of Academic Language and Learning, International Journal of the First Year in Higher Education, the Journal of English for Academic Purposes, and papers from the First Year in Higher Education Conferences was performed, as these journals were not indexed in any of the institution’s subscribed databases. Manual checking of the reference list of included articles for relevant papers was also performed.

Search Strategy

The reviewers had identified a selection of relevant articles which were used as a search strategy validity tool. The initial search strategy failed to locate two papers identified in the pilot phase of the search. The search was therefore modified and repeated in the databases listed above. The modified search was also structured to exclude articles related to children and filtering strategies applied to limit results to a date range from 2000 to 2014, and to English language articles (see Appendix A). Results were exported to an EndNote Library.

Study Selection

Exclusion Criteria

Articles not written in English and not within the date range specified had already been excluded in the databases by the search criteria. Newspaper articles, theses, and book reviews were also excluded. Reviews, conference papers not written in full, short opinion, discussion, or descriptive pieces were also excluded. Remaining articles were initially screened independently by the authors by reading the article abstract. Where there was not initial consensus between reviewers, the full text was retrieved and further discussion ensued until consensus was reached.

Inclusion Criteria

Following exclusion based on the above criteria, the full text of the remaining articles was screened independently by each author. To be included, articles had to describe original research, evaluating the effect of a clearly identified support strategy to improve academic or information literacy skills development of first year undergraduate students studying health science professions (excluding medicine) at university. In addition, learning outcomes for first year health student cohorts had to be clearly identified. Again, where there were discrepancies between authors regarding an article’s inclusion, discussion ensued until consensus was reached.

Data Collection and Synthesis

Relevant information and data were extracted from included papers into tables on each study’s aims, design, cohort, support strategy characteristics (including mode of delivery), outcome measures, analysis, and findings. Authors independently evaluated individual papers to identify and record potential methodological limitations, predominantly associated with selection bias. These limitations were then considered collaboratively and comments recorded about the possible risk of such biases on interpreting the reported outcome of individual studies. A narrative synthesis of potential risks and their possible influence on findings is reported.

Our approach to critically synthesize the literature in this review was based around two frameworks. First, for classifying the support strategy’s mode of delivery and second, for classifying learning outcomes.

Categories for mode of delivery were adapted from the criteria outlined by Bonanno (2002) as follows:

Embedded model: this model moves the development of information literacy and academic skills into the mainstream unit curriculum.

Integrated resources, modules or workshops: these resources, modules or workshops are offered within a unit timetable or unit learning site. They are discipline and/or assessment task specific.

Adjunct resources, modules or workshops: these resources, modules or workshops are offered outside of timetabled unit sessions, either lectures or tutorials. They are generic, and may be delivered online, in an electronic format or face-to-face.

There is no consistency in the literature regarding such terminology, and terms are used inconsistently by academics, information literacy specialists, and language and learning specialists. This can lead to ambiguity when attempting to make sense of the literature. As such, our categorization may not align with terminology used in the individual papers themselves (Betts, Bostoek, Elder, & Trueman, 2012; Hendricks, Andrew, & Fowler, 2014; Wray, Aspland, Taghzouit, & Pace, 2013), nor with other frameworks used (Bundy, 2004, p. 6; Harris & Ashton, 2011, p. A-81).

Within the categories classifying mode of delivery, an outcome classification was assigned based on the study’s outcome measure. This classification was adapted from Kirkpatrick’s model of four levels of evaluation (Kirkpatrick, 1996) for training programs. The evaluation levels used, from lowest to highest level of the hierarchy, are:

Reaction: a measure of how students thought or felt about the intervention. For example, agreement or helpfulness Likert questionnaires.

Learning: a measure of the skills or knowledge students have gained as a result of the intervention. For example, a change in score on skill quiz.

Behaviour: a measure of how the student’s behaviour has changed as a result of the intervention. For example, applied learning where assessment grades, or retention rates have changed.

Organizational change [listed as Results by Kirkpatrick (1996)]: a measure of change that has occurred at the institutional level as a result of the intervention; generally involving a curriculum change. For example, following the study, the intervention was adopted at an organizational level.

A potential limitation of applying such a model in this systematic review is that the Kirkpatrick model was designed for training in industry as opposed to hierarchical application as a critical appraisal tool (Yardley & Dornan, 2012). Despite this, the use of this tool as a conceptual framework is consistent with guidelines (Hammick et al., 2010) and previous systematic reviews in health education (Steinert et al., 2006).

Data was synthesized for analysis by categorizing studies according to the mode of delivery of the intervention (Embedded, Integrated, or Adjunct) and, within these classification, for outcome (Organizational change, Behaviour, Learning, or Reaction).

Results

Fifty-three papers were identified for review. The PRISMA flow chart (Figure 1) details the number of articles identified for inclusion and excluded during phases of the screening process. From these 53 papers, 51 independent studies were identified. It should be noted that each of the two paired papers by Wallace, Shorten, and Crookes (2000) and Shorten et al. (2001); Chester, Burton, Xenos, Elgar, & Denny (2013a) and Chester, Burton, Xenos, & Elgar (2013b); Salamonson, Koch, Weaver, Everett, & Jackson (2010) and Weaver and Jackson (2011), report on findings from the same study and cohort of students. When summarizing these paired papers in this review, extracted data on cohort and types of interventions have only been counted once (that is, per study). Where the individual papers discussed different aspects of data, those findings have been counted independently and discussed accordingly. Furthermore, the paper by Cranney, Morris, Spehar and Scoufis (2008) reports on two independent studies in the one research publication; one targeting information literacy and one the academic skill of working with others.

From the 51 included studies, over 12,000 health science students were provided an opportunity to access support strategies (see Appendices B-D). Participant numbers across studies varied from 8 to 1,841 students. While the majority of studies did not focus on specific equity groupings, five reported on at risk students (n = 345); two on Indigenous students (n = 47); two on students with low English language levels (n = 235) and one on students with learning disability (n = 384). Courses targeted were Nursing (25); Psychology (11); Health Sciences (9); Occupational/Physical therapies (3); and Bioscience and Midwifery (1 each); over half of the targeted students were enrolled in a single discipline. Studies were carried out in Australia (18); England (12); the United States (10); Ireland (3); New Zealand (2); and 1 each in South Africa, Thailand, Canada, and Indonesia. Interventions in 10 of the studies targeted both information literacy and academic skills, 11 targeted information literacy, and 30 targeted academic skills. For Embedded, Integrated and Adjunct interventions, formats included face-to-face, peer mentoring, online, and print.

Embedded Strategies

An embedded mode of delivery was used in 21 studies (23 papers) (Table 1). A number of formats were used to embed support strategies in the curriculum, either singularly or in combination, including workshops, online learning modules, and resources. These were often devised or delivered collaboratively by discipline academics and information literacy and academic language and learning specialists. The majority (11 studies) targeted academic skills, with 5 each addressing information literacy or academic skills and information literacy in combination.

Organizational Change

Corbin and Karasmanis (2010) effected change at an organizational level through an iterative process of implementing an online information literacy support strategy and response to stakeholder feedback. While improvement in information literacy scores was reported, these scores remained poor post-intervention, with only 2 of 11 categories improved to the extent that over half of the respondents were correct. Despite this strategy being institution wide, data on learning indicates limited success.

Behaviour

Eight studies (nine papers) assessed behavioural outcomes, providing evidence of the impact on application of learning, often in the context of curriculum assessment (Table 1 and Appendix B). A control or pseudo–control group featured in three studies that targeted a broad spectrum of academic skills (Chester et al., 2013a; Chester et al., 2013b; Walker et al., 2010; Winstone & Millward, 2012). Winstone and Millward (2012) showed a statistically significant improvement in essay grades and positive student reaction with scaffolded academic skills tutorials; however, this small (2%) increase may not be practically significant (Kirk, 1996). In comparison to their peers, students receiving learning strategies embedded into the curriculum, including formative assessments, were shown to adopt a deeper approach to learning, although surprisingly, higher exam scores were correlated with surface approaches to learning (Walker et al., 2010). The utilization of peer mentoring, to support academic and psychosocial adjustment to university, resulted in improved grades and positive student reaction but did not improve academic progress (Chester et al., 2013a; Chester et al., 2013b).

Several studies investigated student outcomes following embedded support without comparison to a control cohort, with most assessing behaviour based on academic writing outcomes. With a focus on information literacy skills, Boruff and Thomas’ (2011) strategy, involving a lecture, workshops, and a specific assessment, was deemed successful, with 97% of students scoring at least 80% on the assessment. Cassar et al. (2012) investigated a multifaceted academic skills and information literacy support strategy. Outcomes showed improvement in writing grade levels as the semester progressed and pre/posttest numeracy significantly improved. Improved writing outcomes, for consecutive written assessments, were also demonstrated by Fallahi, Wood, Austad, and Fallahi, (2006) with following in-class lessons targeting academic writing and providing peer editing and extensive feedback. While improvement for referencing was immediate, other writing skills did not improve until the fourth assessment (Fallahi et al., 2006). Targeting Indigenous Australian health science students, Rose, Rose, Farrington, and Page (2008) showed scaffolding of academic writing skills significantly improved academic literacy skills. Finally, with a specific focus on teamwork, Cranney et al. (2008) looked at the development of this skill through a strategy built around a group assessment reporting positive outcomes based on mean assessment grades (87%) and a strong relationship between higher marks and group cohesiveness.

Figure 1

PRISMA flow chart for identifying and selecting articles to be included in the systematic review. Reprinted from "Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement,” by D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, & The PRISMA Group (2009). PLoS Medicine 6(6): e1000097 p. 3. http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1000097. Copyright 2009 by PRISMA. Reprinted with permission under the Creative Commons Attributions Licence.

Learning

Learning outcomes were the highest level of evaluation in four studies (reported in five papers, Table 1 and Appendix B). Several of these papers, investigating information literacy, used strategies employing a control or pseudo-control. In a study published by both Wallace et al. (2000) and Shorten et al. (2001), within group improvement for searching and locating resources, as well as interpretation of bibliographic citations, were identified following a structured information literacy strategy. While these improvements were superior to non-program students’ scores for locating and interpreting resources, this was not the case for citation skills. Additionally, students reported higher levels of skill confidence than non-program students. Van Moorsel (2005) reported that a computer literacy strategy significantly improved computer and information literacy of those students in the intervention group, whereas scores for controls remained unchanged. Despite the use of a control, however, between-group statistical comparisons were not reported. Using the data from Van Moorsel (2005, Table 4, column 2 & 3), we calculated 95% confidence intervals for between-group differences (Appendix B), demonstrating the intervention group performed better than controls at both immediate and 5-week follow-up periods, supporting the reported finding.

In the absence of a control group, Pryjmachuk et al. (2012), using a combined information literacy and academic skills strategy, reported significant (13%) improvement in knowledge and in confidence. Likewise, Mandleco, Bohn, Callister, Lassetter, and Carlton (2012) reported within group improvement in 12 of 26 categories of grammar as well as improvement in writing confidence following the provision of three embedded writing modules.

Reaction

Eight papers (Table 1) evaluated outcomes based only on reaction. Consistently, these papers showed that overall, embedded academic skills strategies are viewed positively by students for providing practical tools for improving skills, confidence, and being useful or helpful.

Integrated Strategies

Integrated strategies were delivered by 14 studies, with the majority using module style interventions (Table 1 and Appendix C). Most commonly, the focus was on information literacy skills (Brettle & Raynor, 2013; Craig & Corrall, 2007; Cranney et al., 2008 (Study 1); Lalor, Clarke, & Sheaf, 2012; Weiner, Pelaez, Chang, & Weiner, 2011; Xiao, 2010), followed by academic skills (Betts et al., 2012; Elander, Pittam, Lusher, Fox, & Payne, 2010; Griffiths & Nicolls, 2010; Wray et al., 2013) and then academic and information literacy skills in combination (Hendricks et al., 2014; Hooley, Morrison, Thomas, & Marrs, 2011; Turnbull, Royal, & Purnell, 2011). Modules were made available to students online, although face-to-face delivery was also utilized (Brettle & Raynor, 2013; Craig & Corrall, 2007; Lalor et al., 2012). In the case of Xiao (2010), face-to-face delivery was blended with self-paced resources and online support.

Organizational Change

A pilot study by Hooley et al. (2011), to normalize library and academic skills support access, led to extension of the strategies within the organization. This was despite ambiguous evidence on student success. While there were fewer failures (3%) and similar mean essay grades during the pilot phase compared to a previous cohort, retention was 10% lower in the intervention group. As noted by the authors, the impact on student performance is tentative because of the possible influence of many uncontrolled variables between the two comparison groups; however, authors suggest that the aims of raising the profile of support services and normalizing access to support was achieved.

Table 1.

Summary of Studies Included in the Review Based on Their Mode of Delivery and Outcome Measure.

Mode of Delivery^a	Study	Outcome measure^b
*Embedded*	Study	*Organizational change*	*Behaviour*	*Learning*	*Reaction*
	Arpanantikul et al. (2006)				·
	Beatty et al. (2014)				·
	Beccaria et al. (2014)				·
	Boruff et al. (2011)		·
	Cassar et al. (2012)		·
	Chester et al. (2013a); Chester et al. (2013b)		·
	Corbin et al. (2010)	·
	Cranney et al. (2008)		·
	Fallahi et al. (2006)		·
	Hegarty et al. (2010)				·
	Mandleco et al. (2012)			·
	McMillan et al. (2011)				·
	Pryjmachuk et al. (2012)			·
	Rose et al. (2008)		·
	San Miguel et al. (2013)				·
	Thies et al. (2014)				·
	Van Moorsel (2005)			·
	Walker et al. (2010)		·
	Wallace et al. (2000); Shorten et al. (2001)			·
	Webster et al. (2014)				·
	Winstone et al. (2012)		·
*Integrated*	Betts et al. (2012)		·
	Brettle et al. (2013)			·
	Craig et al. (2007)			·
	Cranney et al. (2008) [Study 1]		·
	Elander et al. (2010)				·
	Griffiths et al. (2010)		·
	Hendricks et al. (2014)			·
	Hooley et al. (2011)	·
	Lalor et al. (2012)			·
	Rolfe (2011)		·
	Turnbull et al. (2011)				·
	Weiner et al. (2011)				·
	Wray et al. (2013)		·
	Xiao (2010)			·
*Adjunct*	Bailey et al. (2007)		·
	Balch (2001)		·
	Brown et al. (2008)			·
	Edwards et al. (2011)			·
	Fleming et al. (2005)		·
	Golding et al. (2012)		·
	Hammond et al. (2010)				·
	Hoyne et al. (2013)	·
	Igbo et al. (2011)				·
	Jorgensen et al. (2013)			·
	Kartika (2008)			·
	Palmer et al. (2014)		·
	Salamonson et al. (2010); Weaver et al. (2011)		·
	Sikhwari et al. (2012)		·
	Silburn et al. (2012)		·
	Sopoaga et al. (2011)		·

Behaviour

In five papers, behaviour was the highest outcome assessed (Table 1 and Appendix C). Studies by Betts et al. (2012) and Rolfe (2011), involving Turnitin (an originality checking software), compared student receiving the strategy to controls. Betts et al. found a significant reduction in plagiarism incidents compared with the previous cohort (1 compared to 7), whilst in contrast, Rolfe noted no overall reduction in plagiarism even though the incidents of poor paraphrasing were decreased (22 compared to 7 incidents). Student cohorts in each of these studies reacted positively to the strategy. Wray et al. (2013) studied students with specific learning difficulties (SpLD) and, following study skills sessions, compared outcomes to students with SpLD from previous cohorts and to peers without identified learning difficulties. Findings showed higher rates of progression (87% of students) compared to SpLD students from previous cohorts who did not receive the study sessions (62%) and progression rates were comparable to student peers. Furthermore, student reactions to sessions were positive.

Following access to online information literacy modules, Cranney et al. (2008 [Study 1]) showed statistically significant, within group, improvements for students’ pre/posttest scores specific to information literacy skills and positive student reaction. Furthermore, specific to behaviour, knowledge application showed significant positive correlation of posttest scores with assignment grades. Online learning support was also provided by Griffiths and Nicolls (2010) where relevant e-tivities and scaffolded phases of academic writing were investigated. Here, 100% of students passed the reflective essay assignment and all responses were positive for online academic support. Neither of these papers had the benefit of a comparison cohort to evaluate the direct effect of the online modular study support intervention on student learning.

Learning

Learning was the highest level of outcome assessed in five papers (Table 1 and Appendix C). Brettle and Raynor (2013), in a well-designed paper, compared student learning following an online information literacy tutorial with a control group who participated in a face-to-face tutorial. A validated pre/posttesting of scored search histories was used and found no between group difference in scores post-intervention, but statistically significant within group improvements for both groups. Despite significant improvements, mean scores for literature searching were still poor for both groups (less than 25%).

Four non-controlled studies assessed information literacy strategies. One delivered the strategy face-to-face in small groups (Craig & Corrall, 2007) and two were computer based (Lalor et al., 2012; Xiao, 2010), all demonstrating a positive impact on learning based on pre/posttesting. The final study, with a broader focus, combined academic and information literacies in a 10 module strategy and showed student knowledge improved, on average, by 7.6% (Hendricks et al., 2014).

Reaction

The effectiveness of support strategies based solely on reaction outcomes was assessed in three papers (Elander et al., 2010; Turnbull et al., 2011; Weiner et al., 2011; Table 1 and Appendix C). While these studies provide evidence that integrated information literacy and academic skills support strategies are viewed positively by students, they do not contribute directly to evidence about improved student learning outcomes.

Adjunct Strategies

Adjunct support strategies (Table 1) were used in 17 papers (16 studies), with 14 targeting academic skills and 2 targeting academic skills in combination with information literacy skills (Appendix D). The papers by Salamonson et al. (2010) and Weaver and Jackson (2011) both reported on the same intervention implemented with the same cohort, but each reported on different outcomes (Behaviour and Reaction). The majority of adjunct strategies were optional for students, with several using diagnostic screening to refer students for academic skills support (Bailey et al., 2007; Hoyne & McNaught, 2013; Palmer et al., 2014). In this context, problematically, outcomes were often compared between students who self-selected to attend strategies and those who did not (Cook & Beckman, 2010). The majority of studies, with the exception of Salamonson et al. (2010) and Weaver and Jackson (2011), did not use a formal control group. The format of these strategies included face-to-face workshops focused on study skill development (Bailey et al., 2007; Hoyne & McNaught, 2013; Palmer et al., 2014; Salamonson et al., 2010; Sikhwari, Selepe, & Maluleke, 2012; Sopoaga & Van der Meer, 2011), peer-assisted learning (Hammond, Bithell, Jones, & Bidgood, 2010), and synchronous (Silburn, Flack, Bridgeman, & Warwick, 2012) or asynchronous online workshops (Brown, Dickson, Humphreys, McQuillan, & Smears, 2008).

Organizational Change

Hoyne and McNaught (2013) investigated a reading and writing program delivered by academic support specialists for students not meeting post-entrance literacy assessment benchmarks. This initiated graduated change in policy at an organizational level, making the program compulsory for at risk students. Once the program was compulsory, there was a 50% reduction in fail rates in a core literacy unit for at risk students.

Behaviour

Behaviour was the highest level of outcome for 10 papers (Bailey et al., 2007; Balch, 2001; Fleming & McKee, 2005; Golding, Wasarhaley, & Fletcher, 2012; Palmer et al., 2014; Salamonson et al., 2010; Sikhwari et al., 2012; Silburn et al., 2012; Sopoaga & Van der Meer, 2011; Weaver & Jackson, 2011; Appendix D). Provision of learning support was offered to students where poor academic skills were evident (Bailey et al., 2007; Palmer et al., 2014). At risk students were identified by Bailey et al. (2007) using a diagnostic essay, and Palmer et al. (2014) via a diagnostic academic literacy tool (Measuring the Academic Skills of University Students [MASUS]). Palmer et al. (2014) showed that following early feedback and compulsory learning support, 73% of at risk students achieved a pass grade, with 77% of students, who were in the lowest band on the MASUS (< 9), improving their score, with 40% moving to the highest band. The analysis by Bailey et al. reported students who attended at least one of four academic skills or information literacy workshops improved their grade. The size of the improvement was not reported in the Bailey study, nor was a control comparison provided. Of importance, however, when considering the value of add-on support, the authors reported that only 50% of those identified as needing additional support accepted it.

Face-to-face workshops on varied academic skills content were the focus of several studies (Fleming & McKee, 2005; Salamonson et al., 2010; Sikhwari et al., 2012; Sopoaga & Van der Meer, 2011; Weaver & Jackson, 2011). Generally, outcomes were reported as positive, based on reported improvements in participants’ behaviour. Neither Fleming and McKee (2005), nor Sikhwari et al. (2012) made use of a control group, and whilst Sopoaga and Van der Meer (2011) compared the academic results of those students who attended with those who did not, the students self-selected to participate in the intervention. While a key strength of the study reported by Salamonson et al. (2010), and Weaver and Jackson (2011) was random allocation of participants into an intervention and control group, unfortunately, participants that were allocated to the intervention and did not attend were then evaluated in the control group.

An external, synchronous academic skills strategy was implemented by Silburn et al. (2012). Students self-selected into the study strategy and results showed these students had a higher mean assessment mark of 8% or greater (adjusted for baseline scores) than students in any of the 3 comparison groups. Furthermore, 90% of students viewed this strategy positively and most of the few negative responses were related to technology.

Print-style interventions for developing academic skills were investigated in the studies by Golding et al. (2012) and Balch (2001). Golding found students who used flash cards more frequently performed significantly better on exams than students who used the cards less. Balch, however, found no correlation between overall degree-of-use ratings and course performance, despite a significant positive correlation between degree-of-use and helpfulness.

Learning

Four papers used Learning as the highest level of evaluation. Three of these delivered academic skills strategies (Brown et al., 2008; Jorgensen & Marek, 2013; Kartika, 2008) and demonstrate mixed results for the skills sets tested (Appendix D). While the study by Jorgensen and Marek (2013) had students self-select attendance at workshops, findings showed that those attending had significantly greater improvement for recognizing academic writing errors compared to non-attending controls. In the absence of a control for comparison, study skills delivered as part of an orientation program were shown to significantly improve general study, concentration, and exam preparation skills, but had no benefit on time management, writing, and note taking based on a within group pre/posttest study skills inventory (Kartika, 2008). A non-compulsory online learning intervention to develop referencing skills resulted in no significant change in pre/posttesting but a significant increase in skill confidence (Reaction) on four of seven items (Brown et al., 2008). Also of note here was the poor uptake of this non-compulsory strategy, with less than 36% of students accessing resources. Edwards and O'Connor, (2011) investigated a compulsory combined online learning intervention, involving computer literacy, research skills, and referencing, finding a 17% improvement in skill pre/post-intervention. Furthermore, students also responded positively to the value of the strategy.

Reaction

Two papers based their evaluation on Reaction (Hammond et al., 2010; Igbo et al., 2011; Table 1 and Appendix D). Both papers offered face-to-face academic skills interventions on a self-selection basis, delivered to three separate cohorts over a three year period. In the study by Hammond et al. (2010), while students agreed peer assisted learning improved social aspects of learning, they felt it did not improve study skills. Additionally, there was generally low attendance for the program (Appendix D). A multifaceted, face-to-face program was reported by Igbo et al. (2011) to be helpful and appreciated.

Discussion

This systematic review has identified very little unbiased, evaluative evidence on the best approach for developing either the academic or information literacy skills of first year health science undergraduates. This is despite the widely reported benefits of study skill support from studies included in this review, irrespective of mode of delivery. While some studies compared formats of intervention, for example, handout compared to workshop and handout (Jorgensen & Marek, 2013); online to face-to-face (Brettle & Raynor, 2013); online, synchronous to asynchronous (Silburn et al., 2012), none of the studies aimed to compare the effect of the support strategy under different modes of delivery (Embedded, Integrated, or Adjunct).

Embedded strategies were investigated most frequently (21 studies), with more than half of these (13) delivering outcomes at the level of Learning or higher, thereby providing evidence based on student learning rather than the student’s perception of the strategy on their learning. Of these 13 studies, all viewed the strategies positively. This included embedded strategies being adopted at an organizational level, even in the absence of empirical findings demonstrating the strategy’s efficacy (Corbin & Karasmanis, 2010). Others report outcomes such as assessment results (for example, Boruff & Thomas, 2011; Cassar et al., 2012; Chester et al., 2013a; Chester et al., 2013b) and improvement in skill domains (for example, Fallahi et al., 2006; Mandleco et al., 2012; Rose et al., 2008) without comparison to a control group. This is problematic because without a control group, this improvement could be attributed to other variables, such as natural progression (Cook & Beckman, 2010), rather than the interventions themselves. Where between group comparisons were made, studies used previous or similar student cohorts who had not undertaken the intervention (for example, Chester et al., 2013a; Chester et al., 2013b; Shorten et al., 2001; Van Moorsel, 2005; Walker et al., 2010; Wallace et al., 2000; Winstone & Millward, 2012). While the reported findings in such studies were positive, due to inherent biases associated with non-random allocation of participants (Cook & Beckman, 2010), such comparisons can be limited. Overall, from the studies included in this review, learning outcomes associated with embedded strategies are positive; however, the effect on learning cannot be clearly attributed to the intervention strategies themselves.

Overall, integrated strategies are also viewed positively for supporting the development of first year health science students’ academic and information literacy skills. Consistent with embedded delivery, one study showed organizational uptake of a strategy with ambiguous quantitative evidence of a beneficial impact on student success and retention (Hooley et al., 2011). Eleven studies reported that learning (via behavioural or learning outcomes) occurred subsequent to integrated support strategies. Again, the majority of studies only looked at outcomes for an intervention group, or provided comparison to a pseudo-control group, limiting the capacity to conclude about the learning effects of the intervention directly (Cook & Beckman, 2010). One study did randomly allocate participants (Brettle & Raynor, 2013), providing strong evidence that the delivery of integrated information literacy support was equally as effective via either face-to-face or online formats.

For adjunct strategies, again, positive learning and behaviour outcomes are typically reported. Similar to other modes of delivery, findings here are commonly based on outcomes from an intervention cohort alone (for example, Bailey et al., 2007; Balch, 2001; Golding et al., 2012; Palmer et al., 2014; Sikhwari et al., 2012). Where comparison to other student groups is provided, this is usually a cohort of students who chose not to attend the intervention (for example, Fleming & McKee, 2005; Jorgensen & Marek, 2013; Silburn et al., 2012; Sopoaga & Van der Meer, 2011) and thereby, such findings are subject to selection bias (Cook & Beckman, 2010). While, by nature, adjunct strategies usually depend on students self-selecting support, there may be possible systematic differences between students who self-select support and those that do not. Factors related to self-selection may result in differences in learning outcomes regardless of the intervention strategy (Cook & Beckman, 2010). Nonetheless, the studies investigating adjunct strategies included in this review provide insight into their potential limitation of low uptake, even where a need for skill development is identified (Bailey et al., 2007; Brown et al., 2008). Of practical relevance, where adjunct strategies were made compulsory for at risk students, the beneficial impact on student success was evident (Hoyne & McNaught, 2013).

A major limitation to finding unbiased evidence on best practice for supporting student skill development is that the majority of studies’ findings, despite reporting positive outcomes, are confounded by the absence of a control. Only 11 of the 50 studies included in this review provided evidence on student outcomes in comparison to a “control” or alternative intervention. Most frequently, studies provided only a pseudo-control whereby students self-selected into an intervention or alternative group (for example, Fleming & McKee, 2005; Golding et al., 2012; Jorgensen & Marek, 2013; Silburn et al., 2012) or a previous cohort was used (Betts et al., 2012; Hoyne & McNaught, 2013; Walker et al., 2010). While it is not uncommon for educational research to lack rigorous design, unfortunately, where students self-select into groups, selection bias is likely to confound results, or comparison to previous cohorts may also be confounded by other uncontrolled or systematic variables unrelated to the intervention (Cook & Beckman, 2010).

Two studies in this review attempted to control for biases related to group allocation by randomizing participants. Brettle and Raynor (2013) compared the effectiveness of integrated support strategies via either online tutorials or face-to-face delivery for developing information literacy skills. As previously discussed, this well-designed study found no between group differences in learning scores post-intervention, with both groups, although still scoring poorly, improving. Similarly, random allocation was adopted in the study reported in the papers by Salamonson et al. (2010) and Weaver and Jackson (2011). Here, ESL students were randomly allocated to an adjunct intervention targeting academic learning and writing support or to a usual support that included an invitation to attend a generic skills program offered by the university. Assignment scores for those attending the intervention were significantly higher than controls. While a strength of this study design was the random allocation of participants, only data for the 47% of the participants who were allocated to the intervention and actually attended were included in the analysis, thereby confounding the study’s findings on the basis that the data was not analyzed on intention to treat principles (Sainani, 2010).

Reaction was the only level of evaluation for 13 studies in this review. Eight of these investigated embedded strategies. Evaluating the effectiveness of an intervention only on outcomes at the level of reaction is problematic. While the overall positive reactions to the majority of interventions is commendable, liking a strategy, or reporting an increase in confidence after completion does not necessarily translate to learning or increases in skill level, or improved grades and retention. For example, Brown et al. (2008) showed no significant change in referencing performance but a significant increase in skill confidence. Likewise, there was a notable mismatch between student success and reaction where Hooley et al. (2011) showed positive reaction outcomes, yet a 10% decline in retention. In further support of this inconsistency, Sikhwari et al. (2012) found a one-day study skills workshop resulted in improved academic achievement but a reduction in scores on a study skills inventory test.

Where individual studies use surveys to evaluate student reaction, response rates are frequently low. Poor response rates were noted in several studies included in this review (for example see, Beccaria et al., 2014 [34%]; Craig and Corrall, 2007 [38%]; Elander et al., 2010 [30%]; Turnbull et al., 2011 [19%]). Systematic non-response can, in part, contribute to non-response bias (Nishimura, Wagner, & Elliott, 2016). It is possible that findings on reaction outcomes, measured via surveys, are subject to such bias and this may also contribute to incongruence of reaction with actual learning and student success.

Low uptake of non-compulsory interventions external to the curriculum is another issue highlighted by the review. In health science disciplines, such interventions are reported as the most common type of support strategies implemented in practice, as they do not encroach on “credit-bearing class time” (Fenton-Smith & Frohman, 2013, p. A-61). Providing evidence to support poor uptake of non-compulsory strategies, less than 36% of health science students accessed online referencing modules (Brown et al., 2008) and similarly, only 47% of students allocated to a targeted academic learning and writing program attended (Salamonson et al., 2010; Weaver & Jackson, 2011). The literature reports that time demands (May, Hodgson, & Marks-Maran, 2005) and lack of relevance and misperception of skill levels (Kimmins & Stagg, 2009), as well as negative stigma (Goldingay et al., 2014), are associated with students not attending adjunct support strategies. In this review, where support programs were made compulsory for all students, completion rates were high (for example, Weiner et al., 2011) and this has significant impact, particularly for students academically at risk (Hoyne & McNaught, 2013).

Students identified as being academically at risk yet failing to engage with support offered was an issue identified in this review. Wray et al. (2013) found that 48% of students identified as at risk of having a specific learning difficulty did not pursue further support on offer. Likewise, only 50% of nursing students identified on a diagnostic essay accepted additional academic support (Bailey et al., 2007). Even lower uptake was identified by Beatty, Collins, and Buckingham (2014), where only 20% of at risk students engaged with available support. Making attendance at support programs for those identified as at risk compulsory, however, was found to significantly reduce failure rates (Hoyne & McNaught, 2013). While this may be perceived as ethically inappropriate, evidence from this review suggests that for students to access support, interventions either need to be compulsory or embedded as a way of increasing the likelihood of engagement.

Limitations

A key limitation to this review is that it has not been able to clearly achieve our aim. Results of studies included are not based on stringent methodology and quantitative analysis; therefore, unbiased evidence about the best approach to support the academic and information literacy needs of first year health science students cannot be provided. This issue has previously been acknowledged by Pryjmachuk et al. (2012) regarding effectiveness of study skills support and commenting on the lack of evaluative evidence in the literature and the need for more robust research. It is important to acknowledge, however, that frequently, individual papers are written as practice reports and are not intended to be experimentally designed research reports.

The currency of the literature search may also be viewed as problematic. In considering the absence of the most current literature (beyond August 2014) as a potential limitation, a further search was undertaken to determine the impact on findings. The search was re-run in selected databases (CINAHL, Academic Search Premier, and PsycINFO) to account for the most recent literature. After duplicates, theses, and book reviews were removed, 62 articles remained and were checked for relevance. Four articles (Kavšek, Peklaj, & Žugelj, 2016; Lin, 2015; Moreton & Conklin, 2015; Sego & Stuart, 2016) were retrieved in full for further consideration. The study by Kavšek et al. (2016) was found to be relevant to the review criteria; however, whilst the study used a control group, it was not randomized. It therefore provided no additional, higher quality evidence and the updated search has not altered the overall findings of this systematic review.

Conclusion

This review has synthesized evidence on academic and information literacy support strategies for first year health science students. The majority of the studies included have reported positive outcomes following the implementation of such strategies, irrespective of their mode of delivery (Embedded, Integrated or Adjunct). Despite the contemporary view and rationalization for delivery of support strategies within curricula, approaches frequently continue to require extracurricular engagement. In terms of identifying the best practice for developing academic and information literacy skills, a major limitation to reaching an unbiased conclusion is that, typically, findings of individual studies are confounded by the absence of suitable control groups. Without suitable control groups for comparison, learning outcomes for the student cohort of interest cannot necessarily be attributed to the intervention itself. This is not necessarily meant to be a criticism of individual papers, as frequently they are written as practice reports. Of further note, articles in this review were not aimed at comparing different modes of delivery, and therefore, do not provide direct evidence on what mode is best. In this context, higher quality research is required to provide increased certainty on what strategies are most effective for developing information literacy and academic skills of first year health science students.

The problem of low student uptake by first year health science students with non-compulsory interventions, particularly when they are targeting those at risk, has also been identified. With this considered, and despite the absence of clear, unbiased evidence of superior learning outcomes for embedded support, there is a strong rationale for academic and information literacy support strategies for first year health science students to be fully embedded into the curriculum. Strategies need to be fully inter-woven in a disciplinary context as a way of maximizing student uptake of the support strategy in a meaningful way to the discipline, and thereby provide an opportunity to impact learning.

Due to lack of suitably designed research providing evidence on which mode of support delivery is most effective, this review has not been able to clearly achieve its aim of identifying the best practice for developing first year health science students’ information literacy and academic skills. However, when considering the nature of students accessing support, embedding strategies into the curriculum is recommended.

References

Ambery, D., Manners, J., & Smith, K. (2005, November). Embedded information literacy: A collaborative approach. Paper presented at the 7th Biennial Conference of the Association for Academic Language and Learning, Critiquing and reflecting: LAS profession and practice, Australian National University, Canberra. Retrieved from http://www.aall.org.au/sites/default/files/las2005/Ambery_Manners_%26_Smith.pdf

Arpanantikul, M., Thanooruk, R., & Chanpuelksa, P. (2006). Self-directed learning readiness, critical thinking skill, and self esteem in nursing students studying through problem based learning. Thai Journal of Nursing Research, 10(1), 59-72.

Bailey, P., Derbyshire, J., Harding, A., Middleton, A., Rayson, K., & Syson, L. (2007). Assessing the impact of a study skills programme on the academic development of nursing diploma students at Northumbria University, UK. Health Information and Libraries Journal, 24(s1), 77-85. http://dx.doi.org/10.1111/j.1471-1842.2007.00741.x

Balch, W. R. (2001). Study tips: How helpful do introductory psychology students find them? Teaching of Psychology, 28(4), 272-291. http://dx.doi.org/10.1207/S15328023TOP2804_09

Bearman, M., Smith, C. D., Carbone, A., Slade, S., Baik, C., Hughes-Warrington, M., & Neumann, D. L. (2012). Systematic review methodology in higher education. Higher Education Research & Development, 31(5), 625-640. http://dx.doi.org/10.1080/07294360.2012.702735

Beatty, S., Collins, A., & Buckingham, M. (2014). Embedding academic socialisation within a language support program: An Australian case study. The International Journal of the First Year in Higher Education, 5(1), 9-18. http://dx.doi.org/10.5204/intjfyhe.v5i1.180

Beccaria, L., Kek, M., Huijser, H., Rose, J., & Kimmins, L. (2014). The interrelationships between student approaches to learning and group work. Nurse Education Today, 34(7), 1094-1103. http://dx.doi.org/10.1016/j.nedt.2014.02.006

Betts, L. R., Bostoek, S. J., Elder, T. J., & Trueman, M. (2012). Encouraging good writing practice in first-year psychology students: An intervention using Turnitin. Psychology Teaching Review, 18(2), 74-81. Retrieved from http://files.eric.ed.gov/fulltext/EJ991412.pdf

Birmingham, E., Chinwongs, L., Flaspohler, M. R., Hearn, C., Kvanvig, D., & Portmann, R. (2008). First-year writing teachers, perceptions of students' information literacy competencies, and a call for a collaborative approach. Communications in Information Literacy, 2(1), 6-24. Retrieved from http://www.comminfolit.org/index.php?journal=cil&page=article&op=view&path[]=Spring2008AR1&path[]=64

Bonanno, H. (2002, July). Standing the test of time: Revisiting a first year diagnostic procedure. Paper presented at the 6th Pacific Rim Conference: First Year in Higher Education, University of Canterbury, Christchurch, New Zealand. Retrieved from http://fyhe.com.au/past_papers/papers02/BonannoPaper.doc

Boruff, J. T., & Thomas, A. (2011). Integrating evidence-based practice and information literacy skills in teaching physical and occupational therapy students. Health Information and Libraries Journal, 28(4), 264-272. http://dx.doi.org/10.1111/j.1471-1842.2011.00953.x

Brettle, A. (2003). Information skills training: A systematic review of the literature. Health Information and Libraries Journal, 20(Suppl. s1), 3-9. http://dx.doi.org/10.1046/j.1365-2532.20.s1.3.x

Brettle, A., & Raynor, M. (2013). Developing information literacy skills in pre-registration nurses: An experimental study of teaching methods. Nurse Education Today, 33(2), 103-109. http://dx.doi.org/10.1016/j.nedt.2011.12.003

Brown, C. A., Dickson, R., Humphreys, A.-L., McQuillan, V., & Smears, E. (2008). Promoting academic writing/referencing skills: Outcome of an undergraduate e-learning pilot project. British Journal of Educational Technology, 39(1), 140-156. http://dx.doi.org/10.1111/j.1467-8535.2007.00735.x

Bundy, A. (Ed.). (2004). Australian and New Zealand information literacy framework: Principles, standards and practice (2nd ed.). Adelaide: Australian and New Zealand Institute for Information Literacy. Retrieved from http://www.caul.edu.au/content/upload/files/info-literacy/InfoLiteracyFramework.pdf

Cassar, A., Funk, R., Hutchings, D., Henderson, F., & Pancini, G. (2012). Student transitions: Evaluation of an embedded skills approach to scaffolded learning in the nursing curriculum. International Journal of the First Year in Higher Education, 3(1), 35-47. http://dx.doi.org/10.5204/intjfyhe.v3i1.102

Chanock, K. (2013). Teaching subject literacies through blended learning: Reflections on a collaboration between academic learning staff and teachers in the disciplines. Journal of Academic Language and Learning, 7(2), A106-A119. Retrieved from http://journal.aall.org.au/index.php/jall/article/viewArticle/256

Chanock, K., Horton, C., Reedman, M., & Stephenson, B. (2012). Collaborating to embed academic literacies and personal support in first year discipline subjects. Journal of University Teaching & Learning Practice, 9(3), 1-13. Retrieved from http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1356&context=jutlp

Chester, A., Burton, L., Xenos, S., Elgar, K., & Denny, B. (2013a). Transition in, transition out: A sustainable model to engage first year students in learning. A practice report. The International Journal of the First Year in Higher Education, 4(2), 125-130. http://dx.doi.org/10.5204/intjfyhe.v4i2.174

Chester, A., Burton, L. J., Xenos, S., & Elgar, K. (2013b). Peer mentoring: Supporting successful transition for first year undergraduate psychology students. Australian Journal of Psychology, 65(1), 30-37. http://dx.doi.org/10.1111/ajpy.12006

Cook, D. A., & Beckman, T. J. (2010). Reflections on experimental research in medical education. Advances in Health Sciences Education, 15(3), 455-464. http://dx.doi.org/10.1007/s10459-008-9117-3

Corbin, J., & Karasmanis, S. (2010, June). Evidence into practice: Evaluation of an innovative information literacy model for first year Health Sciences students. Paper presented at the 13th Pacific Rim First Year in Higher Education Conference, Adelaide, Australia. Retrieved from http://hdl.handle.net/1959.9/526237

Craig, A., & Corrall, S. (2007). Making a difference? Measuring the impact of an information literacy programme for pre-registration nursing students in the UK. Health Information and Libraries Journal, 24(2), 118-127. http://dx.doi.org/10.1111/j.1471-1842.2007.00688.x

Cranney, J., Morris, S., Spehar, B., & Scoufis, M. (2008). Helping first year students think like psychologists: Supporting information literacy and teamwork skill development. Psychology Learning & Teaching, 7(1), 28-36. http://dx.doi.org/10.2304/plat.2008.7.1.28

Dubicki, E. (2013). Faculty perceptions of students' information literacy skills competencies. Journal of Information Literacy, 7(2), 97-125. http://dx.doi.org/10.11645/7.2.1852

Edwards, J., & O'Connor, P. A. (2011). Improving technological competency in nursing students: The Passport Project. Journal of Educators Online, 8(2), 1-20. Retrieved from https://www.thejeo.com/archive/archive/2011_82/edwardsandoconnorpaperpdf

Einfalt, J., & Turley, J. (2009). Developing a three-way collaborative model to promote first year student engagement and skill support. e-Journal of Business Education & Scholarship of Teaching 3(2), 41-48. Retrieved from http://www.ejbest.org/upload/eJBEST_EinfaltTurley2009.pdf

Elander, J., Pittam, G., Lusher, J., Fox, P., & Payne, N. (2010). Evaluation of an intervention to help students avoid unintentional plagiarism by improving their authorial identity. Assessment & Evaluation in Higher Education, 35(2), 157-171. http://dx.doi.org/10.1080/02602930802687745

Fallahi, C. R., Wood, R. M., Austad, C. S., & Fallahi, H. (2006). A program for improving undergraduate psychology students' basic writing skills. Teaching of Psychology, 33(3), 171-175. http://dx.doi.org/10.1207/s15328023top3303_3

Fenton-Smith, B., & Frohman, R. (2013). A scoping study of academic language and learning in the health sciences at Australian universities. Journal of Academic Language and Learning, 7(1), A61-A79. Retrieved from http://journal.aall.org.au/index.php/jall/article/view/253/164

Fergy, S., Heatley, S., Morgan, G., & Hodgson, D. (2008). The impact of pre-entry study skills training programmes on students' first year experience in health and social care programmes. Nurse Education in Practice, 8(1), 20-30. http://dx.doi.org/10.1016/j.nepr.2007.02.003

Fleming, S., & McKee, G. (2005). The mature student question. Nurse Education Today, 25(3), 230-237. http://dx.doi.org/10.1016/j.nedt.2005.01.006

Goldfinch, J., & Hughes, M. (2007). Skills, learning styles and success of first-year undergraduates. Active Learning in Higher Education, 8(3), 259-273. http://dx.doi.org/10.1177/1469787407081881

Golding, J. M., Wasarhaley, N. E., & Fletcher, B. (2012). The use of flashcards in an introduction to psychology class. Teaching of Psychology, 39(3), 199-202. http://dx.doi.org/10.1177/0098628312450436

Goldingay, S., Hitch, D., Ryan, J., Farrugia, D., Hosken, N., Lamaro, G., Nihill, C., & Macfarlane, S. (2014). "The university didn't actually tell us this is what you have to do": Social inclusion through embedding of academic skills in first year professional courses. The International Journal of the First Year in Higher Education, 5(1), 43-53. http://dx.doi.org/10.5204/intjfyhe.v5i1.194

Griffiths, L., & Nicolls, B. (2010). e-Support4U: An evaluation of academic writing skills support in practice. Nurse Education in Practice, 10(6), 341-348. http://dx.doi.org/10.1016/j.nepr.2010.02.005

Gunn, C., Hearne, S., & Sibthorpe, J. (2011). Right from the start: A rationale for embedding academic literacy skills in university courses. Journal of University Teaching & Learning Practice, 8(1), 1-10. Retrieved from http://ro.uow.edu.au/jutlp/vol8/iss1/6

Hafford-Letchfield, T. (2007). Factors affecting the retention of learners following the degree in social work at a university in the south-east of England. Learning in Health and Social Care, 6(3), 170-184. http://dx.doi.org/10.1111/j.1473-6861.2007.00159.x

Hammick, M., Dornan, T., & Steinert, Y. (2010). Conducting a best evidence systematic review. Part 1: From idea to data coding. BEME Guide No. 13. Medical Teacher, 32(1), 3-15. http://dx.doi.org/10.3109/01421590903414245

Hammond, J. A., Bithell, C. P., Jones, L., & Bidgood, P. (2010). A first year experience of student-directed peer-assisted learning. Active Learning in Higher Education, 11(3), 201-212. http://dx.doi.org/10.1177/1469787410379683

Harris, A., & Ashton, J. (2011). Embedding and integrating language and academic skills: An innovative approach. Journal of Academic Language and Learning, 5(2), A73-A87. Retrieved from http://www.journal.aall.org.au/index.php/jall/article/view/158

Hegarty, N., & Carbery, A. (2010). Piloting a dedicated information literacy programme for nursing students at Waterford Institute of Technology libraries. Library Review, 59(8), 606-614. http://dx.doi:10.1108/00242531011073137

Hendricks, J., Andrew, L., & Fowler, A. C. (2014). The piloting of an Academic Literary Education Course (ALEC) to improve academic literacy of first semester undergraduate students in a Western Australian university. Journal of Nursing Education and Practice, 4(4), 19-27. http://dx.doi.org/10.5430/jnep.v4n4p19

Hitch, D., Goldingay, S., Hosken, N., Lamaro, G., Macfarlane, S., Nihill, C., Ryan, J., & Farrugia, D. (2012). Academic skills and beyond: A resource based approach to support student success in higher education. Journal of Academic Language and Learning, 6(2), A29-A41. Retrieved from http://www.journal.aall.org.au/index.php/jall/article/viewArticle/197

Hooley, M., Morrison, L., Thomas, M., & Marrs, L. (2011, June). Enhancing the FYHE experience through professional partnership: Embedding academic and library skills in a first-year psychology unit. Paper presented at the First Year in Higher Education Conference, Fremantle. Retrieved from http://fyhe.com.au/past_papers/papers11/FYHE-2011/content/pdf/11E.pdf

Hoyne, G. F., & McNaught, K. (2013). Understanding the psychology of seeking support to increase Health Science student engagement in academic support services. A practice report. The International Journal of the First Year in Higher Education, 4(1), 109-116. http://dx.doi.org/10.5204/intjfyhe.v4i1.149

Igbo, I. N., Straker, K. C., Landson, M. J., Symes, L., Bernard, L. F., Hughes, L. A., & Carroll, T. L. (2011). An innovative, multidisciplinary strategy to improve retention of nursing students from disadvantaged backgrounds. Nursing Education Perspectives, 32(6), 375-379. Retrieved from http://journals.lww.com/neponline/Abstract/2011/11000/An_Innovative,_Multidisciplinary_Strategy_to.7.aspx

Jorgensen, T. D., & Marek, P. (2013). Workshops increase students’ proficiency at identifying general and APA-style writing errors. Teaching of Psychology, 40(4), 294-299. http://dx.doi.org/10.1177/0098628313501037

Kartika, A. (2008). Study skills training: Is it an answer to the lack of college students' study skills? International Journal of Learning, 14(9), 35-43. http://dx.doi.org/10.18848/1447-9494/CGP/v14i09/45480

Kavšek, T., Peklaj, C., & Žugelj, U. (2016). Information literacy training evaluation: The case of first year psychology students. Journal of Academic Librarianship, 42(4), 293-299. http://dx.doi.org/10.1016/j.acalib.2016.06.008

Kimmins, L., & Stagg, A. (2009, 28–30 September). Creating confidence: Developing academic skills and information literacy behaviours to support the precepts of tertiary academic performance. Paper presented at the 4th Asia Pacific Conference on Educational Integrity (4APCEI), Wollongong, Australia. Retrieved from https://eprints.usq.edu.au/6150/1/Kimmins_Stagg_AV.pdf

Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56(5), 746-759. http://dx.doi.org/10.1177/0013164496056005002

Kirkpatrick, D. (1996). Great ideas revisited. Training and Development, 50(1), 54-59. Retrieved from http://www.astd.org/TD/

Koufogiannakis, D., & Wiebe, N. (2006). Effective methods for teaching information literacy skills to undergraduate students: A systematic review and meta-analysis. Evidence Based Library and Information Practice, 1(3), 3-43. http://dx.doi.org/10.18438/B8MS3D

Lalor, J. G., Clarke, M., & Sheaf, G. (2012). An evaluation of the effectiveness of information literacy training for undergraduate midwives to improve their ability to access evidence for practice. Nurse Education in Practice, 12(5), 269-272. http://dx.doi.org/10.1016/j.nepr.2012.06.005

Lin, K. (2015). Evaluating the effect of a clicker in an information literacy course for college nursing students in Taiwan. CIN: Computers, Informatics, Nursing, 33(3), 115-121. http://dx.doi.org/10.1097/CIN.0000000000000129

Mandleco, B. L., Bohn, C., Callister, L. C., Lassetter, J., & Carlton, T. (2012). Integrating advanced writing content into a scholarly inquiry in nursing course. International Journal of Nursing Education Scholarship, 9(1), 1-16. http://dx.doi.org/10.1515/1548-923X.2213

May, S., Hodgson, D., & Marks-Maran, D. (2005, December). Feet under the table: Students' perceptions of the effectiveness of learning support provided during their first year of study on health and social care programmes. Paper presented at the Society for Research into Higher Education Conference, University of Edinburgh, Edinburgh. Retrieved from http://www.leeds.ac.uk/educol/documents/151488.htm

McMillan, L. R., & Raines, K. (2011). Using the "write" resources: Nursing student evaluation of an interdisciplinary collaboration using a professional writing assignment. Journal of Nursing Education, 50(12), 697-702. http://dx.doi:10.3928/01484834-20110930-01

McWilliams, R., & Allan, Q. (2014). Embedding academic literacy skills: Towards a best practice model. Journal of University Teaching & Learning Practice, 11(3), 1-20. Retrieved from http://ro.uow.edu.au/jutlp/vol11/iss3/8

Moore, D., Brewster, S., Dorroh, C., & Moreau, M. (2002). Information competency instruction in a two-year college: One size does not fit all. Reference Services Review, 30(4), 300-306. http://dx.doi.org/10.1108/00907320210451286

Moreton, E. O., & Conklin, J. L. (2015). Closing the loop on nursing library instruction: Using student performance to improve outcomes. Medical Reference Services Quarterly, 34(1), 113-121. http://dx.doi.org/10.1080/02763869.2015.986805

Nishimura, R., Wagner, J., & Elliott, M. (2016). Alternative indicators for the risk of non-response bias: A simulation study. International Statistical Review, 84(1), 43-62. http://dx.doi.org/10.1111/insr.12100

Oermann, M. H., Leonardelli, A. K., Turner, K. M., Hawks, S. J., Derouin, A. L., & Hueckel, R. M. (2015). Systematic review of educational programs and strategies for developing students' and nurses' writing skills. Journal of Nursing Education, 54(1), 28-34. http://dx.doi.org/10.3928/01484834-20141224-01

Ooms, A., Fergy, S., Marks-Maran, D., Burke, L., & Sheehy, K. (2013). Providing learning support to nursing students: A study of two universities. Nurse Education in Practice, 13(2), 89-95. http://dx.doi.org/10.1016/j.nepr.2012.07.011

Palmer, L., Levett-Jones, T., Smith, R., & McMillan, M. (2014). Academic literacy diagnostic assessment in the first semester of first year at university. The International Journal of the First Year in Higher Education, 5(1), 67-78. http://dx.doi.org/10.5204/intjfyhe.v5i1.201

Pryjmachuk, S., Gill, A., Wood, P., Olleveant, N., & Keeley, P. (2012). Evaluation of an online study skills course. Active Learning in Higher Education, 13(2), 155-168. http://dx.doi.org/10.1177/1469787412441298

Rae, S., & Hunn, M. (2015). Assessing the impact of embedding online academic and information literacy resources into a first year business course. Evidence Based Library and Information Practice, 10(4), 95-112. http://dx.doi.org/10.18438/B80C76

Rolfe, V. (2011). Can Turnitin be used to provide instant formative feedback? British Journal of Educational Technology, 42(4), 701-710. http://dx.doi.org/10.1111/j.1467-8535.2010.01091.x

Rose, D., Rose, M., Farrington, S., & Page, S. (2008). Scaffolding academic literacy with indigenous health sciences students: An evaluative study. Journal of English for Academic Purposes, 7(3), 165-179. http://dx.doi.org/10.1016/j.jeap.2008.05.004

Sacre, S., & Nash, R. (2010). Assessing and developing academic literacy in first year health undergraduates. The International Journal of Learning, 17(9), 189-196. Retrieved from http://ijl.cgpublisher.com/product/pub.30/prod.2933

Sainani, K. L. (2010). Making sense of intention-to-treat. PM & R, 2(3), 209-213. http://dx.doi.org/10.1016/j.pmrj.2010.01.004

Salamonson, Y., Koch, J., Weaver, R., Everett, B., & Jackson, D. (2010). Embedded academic writing support for nursing students with English as a second language. Journal of Advanced Nursing, 66(2), 413-421. http://dx.doi.org/10.1111/j.1365-2648.2009.05158.x

Sego, S. A., & Stuart, A. E. (2016). Learning to read empirical articles in General Psychology. Teaching of Psychology, 43(1), 38-42. http://dx.doi.org/10.1177/0098628315620875

Shorten, A., Wallace, C., & Crookes, P. A. (2001). Developing information literacy: A key to evidence-based nursing. International Nursing Review, 48(2), 86-92. http://dx.doi.org/10.1046/j.1466-7657.2001.00045.x

Sikhwari, T. D., Selepe, M. C., & Maluleke, M. (2012). Assessing the impact of a study skills workshop on the academic achievement of first-year nursing students at the University of Venda, South Africa. African Journal for Physical Activity and Health Sciences, 18(Supplement 2), 83-94. Retrieved from https://www.ajol.info/index.php/ajpherd/article/view/83918

Silburn, J., Flack, M., Bridgeman, S., & Warwick, R. (2012). Embedded academic language and learning support via an e-learning tool. Journal of Academic Language & Learning, 6(3), A36-A47. Retrieved from http://journal.aall.org.au/index.php/jall/article/view/191/153

Sopoaga, F., & Van der Meer, J. (2011). Building a Pacific health workforce in New Zealand: Initial findings from a transition project in first year health sciences at university. A practice report. The International Journal of the First Year in Higher Education, 2(2), 61-68. http://dx.doi.org/10.5204/intjfyhe.v2i2.88

Steinert, Y., Mann, K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., & Prideaux, D. (2006). A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Medical Teacher, 28(6), 497-526. http://dx.doi.org/10.1080/01421590600902976

Stevens, S., & Miretzky, D. (2012). Faculty's perceptions of students' characteristics: A for effort please. Current Issues in Education, 15(2), 1-17. Retrieved from https://cie.asu.edu/ojs/index.php/cieatasu/article/viewFile/875/330

Turnbull, B., Royal, B., & Purnell, M. (2011). Using an interdisciplinary partnership to develop nursing students' information literacy skills: An evaluation. Contemporary Nurse, 38(1/2), 122-129. http://dx.doi.org/10.5172/conu.2011.38.1-2.122

Thies, L. C., Wallis, A., Turner, A., & Wishart, L. (2014). Embedded academic literacies curricula: The challenges of measuring success. Journal of Academic Language and Learning, 8(2), A43-A59. Retrieved from http://journal.aall.org.au/index.php/jall/article/view/301/197

Van Moorsel, G. (2005). Library-sponsored instruction improves core informatics competencies among allied health students: A research-based case study. Journal of Allied Health, 34(3), 145-152.

Walker, R., Spronken-Smith, R., Bond, C., McDonald, F., Reynolds, J., & McMartin, A. (2010). The impact of curriculum change on health sciences first year students' approaches to learning. Instructional Science: An International Journal of the Learning Sciences, 38(6), 707-722. http://dx.doi.org/10.1007/s11251-009-9092-y

Wallace, M. C., Shorten, A., & Crookes, P. A. (2000). Teaching information literacy skills: An evaluation. Nurse Education Today, 20(6), 485-489. http://dx.doi.org/10.1054/nedt.1999.0439

Weaver, R., & Jackson, D. (2011). Evaluating an academic writing program for nursing students who have English as a second language. Contemporary Nurse, 38(1/2), 130-138. http://dx.doi.org/10.5172/conu.2011.38.1-2.130

Webster, S., Harding, T., Robinson, J., Yeboah, C., Hutchison, C., & Mountain, D. (2014). Improving academic success: Evaluation of the BN Transition to Clinical Practice module. Journal of Nursing Education and Practice, 4(5), 95-106. http://dx.doi:10.5430/jnep.v4n5p95

Weiner, S. A., Pelaez, N., Chang, K., & Weiner, J. (2011). Biology and nursing students' perceptions of a web-based information literacy tutorial. Communications in Information Literacy, 5(2), 187-201. Retrieved from http://www.comminfolit.org/index.php?journal=cil&page=article&op=view&path[]=v5i2p

Wilkes, J., Godwin, J., & Gurney, L. J. (2015). Developing information literacy and academic writing skills through the collaborative design of an assessment task for first year engineering students. Australian Academic & Research Libraries, 46(3), 164-175. http://dx.doi.org/10.1080/00048623.2015.1062260

Winstone, N., & Millward, L. (2012). The value of peers and support from scaffolding: Applying constructivist principles to the teaching of psychology. Psychology Teaching Review, 18(2), 59-67. Retrieved from https://shop.bps.org.uk/psychology-teaching-review-vol-18-no-2-autumn-2012.html

Wray, J., Aspland, J., Taghzouit, J., & Pace, K. (2013). Making the nursing curriculum more inclusive for students with specific learning difficulties (SpLD): Embedding specialist study skills into a core module. Nurse Education Today, 33(6), 602-607. http://dx.doi.org/10.1016/j.nedt.2012.07.009

Xiao, J. (2010). Integrating information literacy into Blackboard: Librarian-faculty collaboration for successful student learning. Library Management, 31(8/9), 654-668. http://dx.doi.org/10.1108/01435121011093423

Yardley, S., & Dornan, T. (2012). Kirkpatrick's levels and education 'evidence'. Medical Education, 46(1), 97-106. http://dx.doi.org/10.1111/j.1365-2923.2011.04076.x

Zhang, L., Watson, E. M., & Banfield, L. (2007). The efficacy of computer-assisted instruction versus face-to-face instruction in academic libraries: A systematic review. The Journal of Academic Librarianship, 33(4), 478-484. http://dx.doi.org/10.1016/j.acalib.2007.03.006

Appendix B

Studies Reporting on Academic and Information Literacy Skill Development Strategies Embedded into Curriculum^a

Authors	Cohort	Intervention	Outcome measure	General Findings
Organizational Change
Corbin et al. (2010)	Health Science; Australia. Approximately 1700 students, 1000 usable pre-experience survey responses; 1085 post-experience surveys.	Online IL modules offered in a core unit.	Pre/post difference on literacy quizzes; student and staff feedback on: modules; use of library services and resources; and library discussion board.	Improvement in IL skills (mean score pre- to posttest 26% to 37%); just over ¾ responses were positive – positive student reaction to modules; positive staff feedback; organizational change, responding to stakeholder feedback, with the program implemented faculty-wide.
Behaviour
Boruff et al. (2011)	Physical and Occupational Therapy; Canada. 104 students.	Lecture, workshop, targeted assessment targeting IL and evidence based practice.	Evaluation of learning via curriculum assessment task.	97% of students scored at least 80% on assessment.
Cassar et al. (2012)	Nursing; Australia. Number of enrolled students not reported. Data for 2009 & 2010 students, interviews: 35 students; focus groups: 11 students; surveys: 544 students.	Scaffolded learning approaches in a core, foundational unit including an online learning directory, a numeracy package, and IL and essay writing sessions aligned to assessments.	Mixed methods evaluation for two cohorts in consecutive years. Results on numeracy and writing tasks over time; program evaluation via semi-structured interviews/focus groups; usefulness surveys.	Writing: grade levels improved over the semester; numeracy improved; majority (85%) of students agreed embedding skills was essential or useful.
Chester et al. (2013a); Chester et al. (2013b)	Psychology; Australia. 241 students, 231 provided data.	Peer mentoring program, Transition in Transition Out model (into first year; out of university), supporting academic (particularly learning approaches) and psychosocial adjustment. Focus of paper on on-campus FY students at a one university.	Academic performance (compared to previous cohort; pre/post self-report measures for students’ learning approach (Approaches and Study Skills Inventory for Students [ASSIST]); program evaluation questionnaire.	Higher percentage of students achieved grades > 60%; learning approaches significantly changed (p < .001) to more strategic, deeper (moderate effect size) and less surface based (small effect size); majority enjoyed program (70%) and perceived it to positively influence their academic work (59%).
Cranney et al. (2008)	Psychology; Australia Study 2: 1^st implementation, 533 students (383 analyzed); 2^nd implementation 561 students.	(Study 2): Team work skill development program built around a group assessment task. Upon reflection, specific strategies were implemented in the 2^nd roll out to improve the program.	Student rating of group process (productivity and cohesiveness); evaluation of the program by survey and focus group; group project assessment mark.	Marks indicated that groups functioned effectively (mean score 87%); higher marks were predictive of group cohesiveness (β = .159, p < .05), students responded positively to the program.
Fallahi et al. (2006)	Psychology; USA. 109 students, 78 at follow-up.	Lessons involving writing, peer editing, and extensive feedback.	5 assignments graded and separately rated for basic writing skills (grammar, writing style, writing mechanics, and referencing) by blind assessor.	Improvement on 4 writing skill domains (p < .001), immediately for referencing and by the fourth paper for other writing skills.
Rose et al. (2008)	Health Science; Australia. 8 (FY) Indigenous students (Also, 25 other students in higher years of study not considered in this review).	Scaffolding of AS in the curriculum where teachers guide students through reading, critical understanding, and note taking.	Pre/posttesting of AS.	Significant overall improvements in academic literacy skills (for FY Bachelor cohort).
Walker et al. (2010)	Health Science; New Zealand. 1,841 students enrolled in the subject, 705 included completing both pre/posttests; 599 subjects from a previous cohort [control group] in equivalent subject who completed test.	Study and learning approaches in a new curriculum including self-directed learning modules; more formative assessment tools; website for the discussion of difficult concepts.	ASSIST administered pre/post-intervention and compared to a control group. Correlations for exam scores and learning approaches determined.	Post-assessment, students were taking a deep and more strategic approach to their studies (p < .001) and surface approach to a lesser extent; students adopted a deep approach, to a greater extent, compared to controls (p < .001); high performance on final exam was significantly correlated with a surface approach (r = .16, p < .0001).
Winstone et al. (2012)	Psychology; England. 125 students (47 in the intervention group; 78 from a prior (control) cohort).	20 AS tutorials delivered using a scaffolded approach (compared to a control group who did not receive a scaffolded approach to support AS).	Improvement in essay grades compared between groups; student feedback rating usefulness.	Increase in average essay grades for the scaffolded group was higher (2%) than for the non-scaffolded group (p = .02); the scaffolded approach was rated significantly more useful than the non-scaffolded approach.
Learning
Mandleco et al. (2012)	Nursing; Australia. 176 students (82 providing data for learning and 47 on confidence).	Unit consisting of 3 modules embedded in the curriculum on writing in the disciplines; writing to learn and writing across the curriculum.	Grammar test (CLIPS) pre/post-intervention; writing confidence rated monthly for 4 months.	Significant improvement in 12 of 26 categories on the CLIPS test; mean scores improved each month for writing confidence from a mean score of 3.48 to 4.02 out of 5 over the 4 month period.
Pryjmachuk et al. (2012)	Nursing; England. 260 students (63 complete sets of survey data; 12 interviewees).	Online, blended learning unit delivering 8 AS/IL skills topics incorporating compulsory learning tasks.	Pre/post-surveys evaluating knowledge, skills (via confidence), and attitude; focus groups interviews.	Improvement in knowledge (median score 58% compared to 71%, p < .001) and confidence for a variety of study related skills; the strategy was reported to be “fit for purpose.”
Van Moorsel (2005)	Occupational, Physical and Respiratory Therapy; USA. 189 students (179 usable data pairings); control group of 64 physician assistant students (48 usable data pairings).	Computer literacy for healthcare professionals (3 hrs/wk for 7 wks.).	Pre/posttest (post-intervention and at 5 wk follow-up) measuring acquisition of literature searching skills; change in student confidence for literature searching.	Difference in skill between groups, mean (95% CI): Post = 3.42 (2.27 to 4.56), Post at 5wks = 4.93 (3.79 to 6.07) (calculated from mean (SD); Van Moorsel, 2005, Table 4, column 2 & 3). Within group improvement in skill at both posttest intervals (p < .001) and improved confidence for literature searching (p < .001).
Wallace et al. (2000); Shorten et al. (2001)	Nursing; Australia. 138 interventions (55 sets of complete pre/post data); Control (non-program) 88, 200-level health and behavioural science (including nursing) students.	Structured IL program involving lecture and laboratory/tutorial sessions taught collaboratively with three learning activities and three related assessment tasks in the context of nursing.	Pre/post-program questionnaires assessing application of IL and citation skills; confidence with IL skills.	Post-program, student scores superior to non-program students for locating and interpreting resources (68% compared to 27%, p < .001) but not for bibliographic citation skills (93% compared to 90%, p = .70); within group improvement for searching and locating resources and interpreting bibliographic citations (p < .001). Students’ IL confidence higher than “non-program” for 7 of 10 skills (p < .001).
Reaction
Arpanantikul et al. (2006)	Nursing; Thailand. 136 students (129 qualified to participate and 124 with usable data pairings).	Problem based learning (PBL) method for course delivery (utilizing PBL skills, searching skills, concept mapping, and learning plans).	Pre/post-program surveys: Self-directed Learning Readiness Scale; Self Esteem Scale; critical thinking.	Pre-program > 95% scored in the high category for self-directed readiness and self-esteem; significant improvement in self-directed readiness (mean increase 4%, p < .01); post-program, 88% scored moderate and 11% low for critical thinking.
Beatty et al. (2014)	Health Science; Australia. 111 students (51 students identified at academic risk and targeted for support).	English language and AS embedded in a core unit; scaffolded assessment; contextualized examples; in-class collaborations between discipline staff and learning advisers; opportunity for support. At risk students recommended to seek help.	Generic unit feedback form; number of students who accessed support following referral.	> 79% agreement that the unit had improved communication and writing skills; 89% reported having a clear understanding of what was required in the unit; limited success in encouraging at risk students to seek additional support (10 of 51 at risk students sought support).
Beccaria et al. (2014)	Nursing; Australia. 301 students at 2 campuses (92 responses for pre-survey 102 responses for post-survey).	3 targeted learning and teaching activities embedded in a core unit, focusing on group work skills.	Pre/post-surveys; students’: (1) perception of group work; (2) approaches to learning (two-factor Study Process Questionnaire).	Significant increase to both surface (p = .02, effect size = .04); and deep (p = .04, effect size = .2) approaches to learning. A surface learning approach more likely to be associated with a discomfort for group work.
Hegarty et al. (2010)	Nursing; Ireland. 350 students (number in intervention for FYs not specified).	IL training workshops across the 4 year degree. 2 hour session for FYs delivered in a research skill module.	Online survey on reaction to the program.	FY: 98% agreement that workshop was practical and useful; 100% valued the program as good or above.
McMillan et al. (2011)	Nursing; USA. 46 students.	1 hour IL in-class session for: writing tutorials; classroom peers for feedback; student tutors at the writing centre for editing and proofreading assignment.	Writing Assignment Resource Evaluation (author developed tool) for effectiveness of activities to improve student writing, learning and quality of work.	≥ 70% agreed that IL was instructive; time with tutors valuable; class time for writing assignment & peer review beneficial; ≥ 50% agreed draft revisions with tutor contributed to learning; working with writing centre improved understanding of writing.
San Miguel et al. (2013)	Nursing; Australia. 176 students with low English language proficiency (2 cohorts over consecutive years).	Diagnostic screening and specific tutorial program with collaboratively developed materials and assessment tasks to respond to student diversity targeting students identified.	Focus group: themes identified & clustered into 5 major categories.	Students felt more comfortable and confident; deeper explanations of information; tutor is very important; learning about reading and writing; mixed responses regarding how helpful this was; moving on helped them to adjust.
Thies et al. (2014)	Health Science; Australia. 1,152 students.	Various AS embedded across 3 core health units involving: online modules to improve study techniques; learning resources for researching and report writing; use of feedback to help students be reflective learners.	Mixed methods approach looking at reaction to program. Student surveys and focus groups; staff interviews and surveys; use of a student AS reflective tool.	Findings for 1 of the 3 units (HBS109): 82% accessed at least one module; 76% found them very helpful or quite helpful; staff report students having good or excellent understanding of AS.
Webster et al. (2014)	Nursing; Australia. 400 students (380 at follow-up).	Transition to Clinical Practice Module program that includes a 5 week compulsory transition module in a core FY unit that includes English literacy, library research skills, and clinical competencies.	Pre/post-intervention questionnaire to evaluate knowledge and confidence for transition into clinical practice.	82% rated program better than expected; 96% indicated it had helped to develop skills; significant increase (p < .01) in knowledge and confidence ratings post-program in most areas except confidence with AS and accessing support.

^aStudies categorized according to highest level of learning outcome (Kirkpatrick, 1996).

KEY: AS = Academic skills; CI = Confidence Interval; FY = First year; IL = Information literacy; SD = Standard Deviation

Appendix C

Studies Reporting on Academic and Information Literacy Skill Development Strategies Integrated into Curriculum^a

Authors	Cohort	Intervention	Outcome measure	General Findings
Organizational change
Hooley et al. (2011)	Psychology; Australia. 80 (off campus) students (20 completed the questionnaires).	AS and library staff resided as guest lecturers in the LMS answering questions; help sheets provided.	Organizational change; student academic performance and retention; students surveyed on usage, intention to use, attitudes, and awareness.	Students reported: increased awareness of resources and likelihood of use (100%); skills support more accessible (90%); helpful (85%). 10% decline in retention (95% vs. 85%); fewer failures (1.4% vs. 4.6%); essay grades unchanged. Extension of program in organization including: earlier presence of advisors on LMS; online module developed as an early assessment.
Behaviour
Betts et al. (2012)	Psychology; England. 116 students; 71 completed the evaluation.	Lecture on academic writing; presentation on referencing conventions; report submitted via Turnitin with assistance provided on interpreting the originality report.	Plagiarism rates in subsequent assessments; online survey on effectiveness and learning experience.	Reduction in occurrences of plagiarism (7) compared with the previous cohort (1, no p value reported); majority agree Turnitin reassured them their work was their own (68%) and that it helped to understand plagiarism (58%).
Cranney et al. (2008) (Study 1)	Psychology; Australia. 752 psychology students.	Five IL skills modules (via LMS platform).	(Study 1): Pre/post IL tests; questionnaire and focus groups.	Within group improvements for students’ pre/posttest IL scores (p < .05). Higher average posttest performance significantly associated with higher grades (p < .01). Program viewed positively for attitudes, usefulness and being liked.
Griffiths et al. (2010)	Nursing; England. 17 students.	E-Support4U: a series of relevant e-tivities, closely related to the students’ academic course, scaffolding phases of academic writing.	Reflective-practice assignment results; online self-assessment quiz; LMS access data.	100% pass rate in assignment; 100% positive response to online academic support; perceived barriers related to: accessibility, finding time for computer use on wards; limited IT skills; the placing of the e-Support4U as a stand-alone module rather than having it embedded.
Rolfe (2011)	Bioscience; England. 76 students (80 control subjects from previous cohort).	Using Turnitin; instruction on interpreting originality reports.	Student draft and final essays analyzed against criteria; staff/student questionnaires; small sub-group interviews.	No significant between group difference in overall plagiarism. Significant: reduction in plagiarism due to poor paraphrasing (22 incidences compared to 7, p < .05); increase in students not providing in-text citations (25 to 45 students, p < .05); poorer essay performance (62 compared to 53%, p < .001). Turnitin use rated positively; staff reported it raised plagiarism awareness.
Wray et al. (2013)	Nursing; England. 384 students (2 consecutive cohorts) with SpLD, 300 completed the evaluation survey); (control group: previous cohort).	Nine study skills sessions delivered in a core unit.	Progression data; feedback questionnaire; time to disability registration.	Progression rates 25% higher than previous cohort with SpLD and comparable to peers with no learning difficulty; sessions viewed positively; students contacted disability services 4–6 weeks earlier than previous cohort.
Learning
Brettle et al. (2013)	Nursing; England. 93 students with 77 randomized attending program (intervention n = 40; comparison group, n=37 and 55 students at follow-up).	Randomized controlled trial comparing an online IL tutorial to a F2F IL tutorial.	Search histories scored using a validated checklist; skill retention measured at 1 month.	No between group differences for posttest scores (p < .05); both groups improved (OL mean scores from 3% to 18%, p < .001; F2F 4% to 22% p < .001); skills retained 1 month later.
Craig et al. (2007)	Nursing; England. 76 students (29 completing both pre/posttest; 9 interviewed).	IL program 3 x 3 hr F2F practical sessions to 8 small groups – whole group induction and information searching	Paired pre/posttest for IL skills and confidence; interviews on confidence outcomes.	72% improved their skill scores; 97% self-rated confidence as “Fairly” or “Very” confident compared with 76% at pretest.
Hendricks et al. (2014)	Nursing, Paramedicine; Australia. 214 nursing (n = 143) and paramedicine (n = 71) students.	Academic Literacy Information Course: Ten learning modules related to referencing, sourcing information, essay writing, and paraphrasing.	Paired pre/posttesting related to understanding of academic literacy concepts.	Significant improvement (7.6%, p < .001) in understanding of academic literacy concepts.
Lalor et al. (2012)	Midwifery; Ireland. 63 FY (from a total of 108) students (49 FY students with complete data).	In the first year: 4 hrs of computer based sessions focussed on IL skills.	Pre/posttest on search histories rated by researchers as poor, fair or good.	Improvement in IL skills with only 3% (from 79% at baseline) remaining “poor” posttest.
Xiao (2010)	Nursing; USA. 356 students from 2005 – 2008 FY cohorts.	IL integrated into unit via: 1 hr library orientation; self-paced online tutorials; online resources; and librarian support on discussion forum in LMS.	2008 pre/post true/false test on IL and APA referencing knowledge. 2005-2008 post-intervention evaluation survey.	Pre/posttest (2008): students improved understanding for some IL concepts; greater confidence in completing research assignment. High levels of agreement that course was helpful and improved skills.
Reaction
Elander et al. (2010)	Psychology; England. 364 students (111 completed survey).	Education regarding authorship, writing, and avoidance of plagiarism integrated into existing modules. Delivered at various institutions prior to assignment submission.	Student Authorship Questionnaire (18 item Likert response); Questionnaire re: usefulness of the intervention; Focus groups.	Confidence in writing, understanding of authorship, knowledge to avoid plagiarism, and top-down approaches to writing increased significantly, with greatest improvements for FY undergraduates; 86% believed it helped them avoid plagiarism; 66% believed it helped them write better; changed understanding about authorial identity and academic writing.
Turnbull et al. (2011)	Nursing; Australia. 174 students.	Online tutorials – 6 modules covering IL, academic integrity, and referencing.	Online survey.	83% agreed to being more confident in using library resources post-tutorial.
Weiner et al. (2011)	Nursing; USA. 60 students (48 FY students completed the modules & questionnaire). (Biology student outcomes not reported in this review).	Compulsory online IL tutorial for nursing students that formed an assignment in their course.	Module completion; self-report questionnaires.	Nursing students: 97% completed all modules; 75% liked the intervention as they learned important information; 91% of these indicated they did not know the information prior (p = .005).

^aStudies categorized according to highest level of learning outcome (Kirkpatrick, 1996).

KEY: AS = Academic skills; F2F = Face-to-face; FY = First year; IL = Information literacy; LMS = Learning management system; SpLD = Specific learning difficulties.

Appendix D

Studies Reporting on Academic and Information Literacy Skill Development Strategies Adjunct to the Curriculum^a

Authors	Cohort	Intervention		Outcome measure	General Findings
*Organization*
Hoyne et al. (2013)	Health Science; Australia. 76 students identified as “at risk” using a post-entrance literacy assessment.	Support program, for students diagnosed at risk, in reading and writing delivered by the Academic Enabling Support Centre.		Pass/fail rates for a core literacy unit.	Reduction in fail rate by 50% for at risk students; (10% compared to 21%) when support was made compulsory.
*Behaviour*
Bailey et al. (2007)	Nursing; England. 46 students identified from a diagnostic essay (at risk).	Four remedial workshops: 2 on IL skills; 1 on essay writing; 1 on referencing outside of class tutorial times.		Focus groups; questionnaire; assignment grade.	50% of students identified as needing additional support accepted it (only 1 student attended all 4 sessions). Students attending at least one workshop improved their academic grades in next assignment. Students increased their confidence and perceived an improvement in IL and referencing.
Balch (2001)	Psychology; USA. 114 students.	Series of Study Tips sheets.		Students rated each tip for helpfulness and amount of use on a scale from 0-10.	Most helpful tips were for lecture notes; degree-of-use ratings were correlated with overall helpfulness r(113) = .42, p < .001 ratings, but not with course performance; no correlation between overall degree-of-use ratings and course performance, r(113) = –.05, p > .10).
Fleming et al. (2005)	Nursing; Ireland. 67 mature age students invited with 44 attending the intervention and 33 returning questionnaires.	One week, 2 part, pre-course program: 1) socialization to university life; 2) study skills.		Questionnaires about program; progression rates.	Better progression through course of mature students who attended compared to those who did not p < .05; high agreement for: program achieving objectives; successful and very helpful in preparing for the course; giving confidence and information needed to start course.
Golding et al. (2012)	Psychology; USA. 415 psychology students with 60% FYs.	Flash cards (both written and computer generated) to prepare for exams.		Flashcard Survey; Exam results.	Students who used flashcards on all exams performed better than other students (Mean ±SD) (41.34±5.16) compared to those who used flashcards on one exam (38.67±4.86, p = .0001); and two exams (38.57±5.21, p = .002); and no exams (40.03±5.23, p = .043).
Palmer et al. (2014)	Nursing; Australia. 569 students (513 completed preliteracy screening test – 92 targeted (at risk) based on test results).	Early feedback on academic literacy skill levels from unit tutors and provision of non-compulsory learning support (from the learning development unit in the University) targeted at students who scored in the lowest band (<9) on Measuring the Academic Skills of University Students (MASUS). All students could attend.		MASUS scores classified into 3 bands; course grade.	Of those in the lowest band (<9): 77% improved their MASUS score; 40% shifted to the highest band (>12); and 73% achieved a pass grade.
Salamonson et al. (2010); Weaver et al. (2011)	Nursing; Australia. 106 ESL students (with low to medium ELAS score <19) randomly allocated into intervention group: n = 59 (28 attending); and usual support n = 47).	4-day targeted academic learning and writing support workshops; one-on-one sessions providing individual feedback on academic writing.		Assignment results; open-ended questionnaire (pre/post) about assistance they wanted and perceptions of the program and support; informal feedback sought by group discussion in final intervention session.	Better assignment scores for intervention group (Mean ±SD = 70.8±6.1) compared to control group (58.4±3.4, p = .002) and to non-attendees (48.5±5.5, p = .001). Provision of individual feedback identified as a key benefit.
Sikhwari et al. (2012)	Nursing; South Africa. 33 students.	One day study skills workshop developed by the Student Counselling and Career Development Unit, focused on motivation, time management, learning skills, concentration, exam techniques.		The Learning and Study Strategies Inventory (LASSI) as a pre/posttest; academic results between semesters and years.	On average, LASSI scores poorer at follow-up (p < .025); increase in academic achievement between semesters (Mean increased scores ±SD) (7.10±4.21, p = .000 and between years 5.53± 4.57, p = .000).
Silburn et al. (2012)	Health Science; Australia. 86 (29 internal, 57) students. Self-selected into 1 of 4 study groups.	External, synchronous academic language and learning support (4 1 hour online workshops in LMS) on essay writing and referencing compared to Internal – no academic language and learning support; External – no online academic language and learning support; External asynchronous.		Online survey; assessment marks pre/post- intervention.	Synchronous online academic language and learning students: 90% of positive responses, most negative responses related to technology; highest mean grades (8% or greater) post-intervention compared to other 3 groups (adjusted for baseline scores, p < .05).
Sopoaga et al. (2011)	Health Science; New Zealand. 39 Pacific Islander students.	6 week structured program: peer educators met with students weekly; general guidance provided, academic support, information about support services and university systems.		Comparison of academic results for attending and non-attending students.	Superior grades for attending students: A or B grades for 39% compared to 0% of non-attending students; Fail grades for 36% compared to 44% of non-attending students.
*Learning*
Brown et al. (2008)	Health Science; England. 57 students; 20 accessed intervention.	Learning to Reference Project (6 online, archived audio-visual learning presentations).		Pre/posttesting of knowledge related to referencing, attitudes, and computer skills irrespective of module completion (n = 52 at follow-up); paired data analysis.	No change in performance on a referencing quiz; significant increase in referencing skill confidence on 4 of 7 items (p < .005). < 36% accessed the resources.
Edwards et al. (2011)	Nursing; USA. 90 students.	PASSPORT Project for Nursing Success: 7 online learning modules in the LMS to improve student orientation, computer literacy, research and APA format knowledge.		Pre/posttest computer literacy survey; qualitative evaluation.	Mean computer literacy score increase on posttest of 17%; positive responses in terms of value of modules, some negative responses relating to access to academic advisor and lack of time to complete PASSPORT.
Jorgensen et al. (2013)	Psychology; USA. 105 students; 58 students attended workshops and 47 students acted as controls.	Students self-selected into groups. Participant groups attended 1 of 3, 20-30 minute workshop topics on grammar, mechanics, or referencing. Control group attended one discussion session about APA style rules and received a handout.		Paired pre/posttesting; Pretest, items relating to error recognition on topics delivered. Follow up at 2 and 7 days and 2-4 weeks.	Workshop participants improved more (Mean difference±SE = 9.91±0.69) than handout-only (control) participants (3.19±1.17, p < .0001). Workshop participants improved for all topics (grammar, p < .0001; mechanics, p < .0002; references p < .0001) and retained their proficiency in follow-up tests. Control group participants did not improve (p = .15).
Kartika (2008)	Psychology; Indonesia. 155 students.	Study skills training embedded in the University’s orientation program involving 6 x 3 hr sessions during a 14 week semester.		Pre/posttesting using The Study Skills Inventory (SSI).	Significant improvement in SSI item scores for concentration (p < .05); exam preparation (p < .001). No improvement for time management and writing/note taking skills.
*Reaction*
Hammond et al. (2010)	Physiotherapy; England. 90 students (3 cohorts over 3 years (26 in 2003; 39 in 2004; 25 in 2005).		PAL – Voluntary, timetabled sessions, encouraging cooperation, team work, and active problem solving through student-directed activities around class content.	Questionnaire evaluating student perception and satisfaction with PAL sessions.	Overall student agreement that PAL improves social aspects of learning but does not improve study skills or assignment preparation. Generally low attendance with at >3 sessions 20 – 59% (2003-5).
Igbo et al. (2011)	Nursing; USA. 105 (76% high risk) students (3 cohorts over 3 years (27 in 2004; 39 in 2005; 39 in 2006).		F2F study skills, critical thinking, communication, professional socialization, medical terminology, and career coaching activities 2 hours, 1 afternoon/week for the first academic year.	Progression/retention rates of students. Student feedback; reporting of grade point average increase and confidence levels medical terminology pre/posttests.	76.8% average retention rate over the 3 years. Feedback indicates that the program was helpful and students were appreciative of it.

^aStudies categorized according to highest level of learning outcome (Kirkpatrick, 1996).

KEY: ESL = English as second language; F2F = Face-to-face; FY = First year; IL= Information literacy; LMS = Learning management system; PAL = Peer assisted learning.