464 Measuring Association between Library Instruction and Graduation GPA Shun Han Rebekah Wong and Dianne Cmor Shun Han Rebekah Wong is Assistant Librarian in the Multimedia Services Section and Dianne Cmor is Assistant Librarian in the Information Services Section at Hong Kong Baptist University Library; e-mail: rebekahw@hkbu.edu.hk, dcmor@hkbu.edu.hk. © Shun Han Rebekah Wong and Dianne Cmor Academic libraries devote considerable human resources in delivering library instruction programs. This study attempts to determine if these instructional efforts have any measurable effect on student performance in terms of overall grades. Library workshop attendance and gradua- tion GPA of over 8,000 students was analyzed at Hong Kong Baptist University. It was found that, if more than one or two library workshops were offered to students within the course of their program, there was a higher tendency of workshop attendance having a positive impact on final GPA. The results indicate that library instruction has a direct cor- relation with student performance, but only if a certain minimum amount of instruction is provided. lthough most academic librar- ies are dedicated to the provi- sion of quality library instruc- tion programs in response to changing information and institutional environments, very little progress has been made in measuring the actual effect of these programs on student perfor- mance. The Hong Kong Baptist University (HKBU) Library has provided one-shot, skills-centric library sessions for many years until 2008. Starting from that time, following institutional trends, we gradual- ly adopted an integrated, outcomes-based teaching and learning approach to our library instruction programs. Before we completely move to a new era, we think that a thorough and empirical assessment of previous instructional efforts may bring insight for our next move(s). In April 2010, the HKBU Library embarked on a large-scale analysis to investigate if a correlation exists between student cumulative grade point aver- age (GPA) and their library workshop attendance, making use of 2007 to 2009 graduates as the sample. Cases totaling 8,701 were involved. This is an extension of an earlier project conducted a few months ago, which successfully uncov- ered a meaningful correlation between student GPAs and their loans of books and audiovisual materials.1 Literature Review Much of the literature on assessing li- brary instruction programs focuses on student perceptions of the instruction or on isolated skills improvement in the form of pre- and post-testing in relation crl-151 Measuring Association between Library Instruction and Graduation GPA 465 to library instruction. Recently, there have been more attempts to move beyond perception and skills-testing to assess deeper learning of information literacy concepts, including the retention and application of such knowledge and skills. Still, only a small portion of the literature examines the impact of library instruction on students’ academic performance in a course or curriculum, with even a smaller portion looking at overall student success indicators such as the GPA or graduation rates. This literature review will focus on the effect of library instruction on student performance and success at the course assignment, course, or program level in higher education settings. In the effort to assess how library instruction impacts student course per- formance (defined here as performance needed beyond the library instruction session), most of the attention to date has been on assessing a specific research assignment or the bibliography of such an assignment. Content analysis of bib- liographies wherein student success is defined in terms of numbers of citations, breadth of sources, scholarliness of ma- terials used, accuracy of citations, and so on have been undertaken (Ackerson and Young;2 Hovde;3 Webster and Reilly;4 Mohler;5 Knight;6 Hurst and Leonard;7 Edzan;8 Knight-Davis and Sung9). Though the literature generally reports that stu- dents who receive library instruction show improved bibliographies in terms of these metrics, the more difficult ques- tion is whether library instruction and/ or improved bibliographies will lead to better papers and better grades. The results of studies attempting to address this question have been mixed. In assessing term papers, Kohl and Wilson found no significant relation- ship between bibliography ratings (by researchers) and grades given by course instructors.10 Emmons and Martin at- tempted to assess beyond the bibliog- raphy, by using a rubric to determine if students were able to make use of the information they found in the context of supporting arguments and construct- ing knowledge in their papers.11 They found that library instruction had some positive effects on information seeking as evidenced in bibliographies, but students were rarely able to evaluate, analyze, or synthesize effectively in their writing. Dogden et al found no evidence that library instruction had a positive impact on the grades of a sociology assignment.12 Hurst and Leonard found that students receiving library instruction were more likely to cite scholarly sources in their papers; however, there was no difference in paper or course grades.13 The authors suggest that the grading of research pa- pers needs to include the explicit grading of research itself for the impact of library instruction and improved information literacy to become evident. Though several studies found no re- lationship between library instruction or high-quality bibliographies and assign- ment grades, other studies did find some relationship. Dykeman and King assessed a small sample of term papers both on research skills and on writing skills and found that students who had received instruction from a librarian and a writing instructor performed better than those who did not receive such instruction.14 Robinson and Schegl found a statistically significant correlation between citation behavior and assignment grade but sug- gested that the correlation might have been based more on quantity of citations rather than quality.15 Wang found that stu- dents who took an elective library credit course produced better bibliographies in terms of scholarly sources and correct ci- tations and that they also received higher grades for their papers and their courses than did their matched counterparts who did not take the library course.16 Looking more broadly than a particu- lar assignment, studies that aim to show a relationship between library skills and/ or instruction and overall student success have been few. An early study by Corlett found scores on a library skills test to be a valid forecaster of freshman GPA at the 466 College & Research Libraries September 2011 end of the first semester.17 Breivik found students who took a library orientation course alongside their English course had higher term paper scores and course completion rates than those students who took the English course alone.18 Selegean, Thomas, and Richman used matching control variables (major, level, and SAT scores) to determine if students who took a library credit course had higher GPA, persistence, and graduation rates than those who did not.19 They found that the study group had a significantly higher GPA and persistence rate but found no significance on graduation rates. Moore et al found a positive association be- tween library class attendance and GPA, but the sample size was too small to be significant.20 They did, however, find a statistically significant association be- tween workshop attendance and overall pass rate. Coulter, Clarke, and Scamman compared course grades between sections in courses that received library instruction and sections those that did not and found no clear association between library in- struction and course grade overall.21 Broader than the scope of this literature review, Matthews summarized much of the research literature relating to library instruction (including opinion surveys, skills testing, and academic performance), characterizing it as supportive or nonsup- portive of the idea that instruction has a positive effect on student performance.22 Excluding opinion surveys, results showed a fairly even split between stud- ies that found a positive association and those that found no association. Of impor- tance to note, however, is the sample sizes of these studies, which were all relatively small; the largest sample was 512. Experimental Design As this study was an extension of an earlier project, the HKBU administration had already approved for the library to be given the necessary student academic data from the Academic Registry (AR). Nevertheless, to protect the privacy of the graduates, library colleagues were not allowed to have the GPA data with the student’s identification, which was the student number in this case. There- fore, the library prepared Excel tables with student names, student numbers, programs of study, graduation years, and library workshop attendance data; AR replaced student numbers and student names with corresponding GPA data and shuffled the order of rows before sending them to the library. Student attendance at library workshops had been systemati- cally recorded for several years, allowing for this type of analysis. Samples & Populations The subjects of this study were all HKBU students who graduated within the last three years (from 2007 to 2009). A total of 8,701 students (cases) were identified. As Hong Kong provides three-year tertiary education, most of the subjects carried out their study at our university within the period from 2004 to 2009. Library workshop attendance data was retrieved from the workshop attendance sheets over the period of these six years. The two independent variables in this study were: 1. Graduation GPA (denoted as “GPA”), ranging from 1.82 to 4.00 2. The number of times these students attended library instruction work- shops (denoted as “WKS”). These data ranged from 0 to 5. Cases (pairs of data) were first divided into 53 sample groups, according to their study major (for example, Translation, Bi- ology) and level of study (undergraduate or graduate level). This arrangement was based on the belief that different subject disciplines had different criteria or internal guidelines to assign GPA values, so cases across programs of study were not compa- rable. (Please refer to table 1.) Among the fifty-three groups that we identified, eight of them were invalid for this analysis. (They are shaded in grey in table 1.) These invalid groups matched one of three circumstanc- es: (a) no library instruction programs were offered for the group during that period; (b) library instruction programs were offered Measuring Association between Library Instruction and Graduation GPA 467 but no one attended; or (c) the sample size was too small (n<30). The sample size of the remaining 45 valid sample groups ranged from 31 to 1,223. The corresponding population of each sample group is all HKBU current students and graduates who share the same study major and level of study. For Sample Group No. of Workshops Offered Throughout Their StudyDiscipline Level* FACULTY OF ARTS Chi Lang & Lit UGS 2 GS 1 Eng Lang & Lit UGS 5 GS — Humanities UGS 3 Language Std GS 2 Music UGS 3 GS 1 Religion & Phil UGS 3 GS 1 Translation UGS 2 GS 1 SCHOOL OF BUSINESS Accountancy UGS 3 GS 1 Economics UGS 2 GS — Finance UGS 2 Human Res Mgt UGS 3 GS 2 Marketing UGS 2 China Business UGS 2 Business Mgt GS 1 SCHOOL OF COMMUNICATION Film UGS 2 GS — Communication Std UGS 2 GS 4 Journalism UGS 2 GS 2 Sample Group No. of Workshops Offered Throughout Their StudyDiscipline Level* Biology UGS 1 GS 2 Chemistry UGS 2 GS 1 Computer Sc UGS 1 GS 1 Mathematics UGS 1 GS — Physics UGS 1 FACULTY OF SOCIAL SCIENCES Education GS 1 Geography UGS 2 GS — Gov’t & Int’l Std UGS 4 History UGS 3 GS — Physical Ed UGS 1 GS 1 Sociology UGS 2 GS — Social Work UGS 2 GS 2 OTHER Chi Medicine UGS 3 GS 2 Visual Arts UGS 1 GS — * UGS denotes undergraduate students and GS denotes graduate students Table 1 Sample Groups with the Number of library Instruction Programs Offered 468 College & Research Libraries September 2011 example, the sample group is intended to provide inferences, whenever possible, to all HKBU students/ graduates who take/took journalism as their major in their undergraduate study. Statistical Methods Used For each valid sample group, we used the Chi-Square (χ2) Test for Independence to determine whether there was a statistical association between the two variables (GPA and WKS). A nonparametric pro- cedure was adopted instead of the more common parametric tests such as Pear- son’s Correlation Test. First, the variable WKS of nearly all sample groups was not normally distributed. In 16 out of 45 valid sample groups, WKS was also found to be “dichotomous”23 (that is, having only two values that were either “0” or “1”). These two factors made parametric cor- relation tests impossible for this analysis. Among the common nonparametric tests, we chose the Chi-Square (χ2) Test for In- dependence, since the variable WKS fit the definition of categorical data, which “are represented by counting the number of times a particular event or condition occurs.”24 To perform the test, GPA data was also transformed into categorical data based on AR’s grading system (refer to table 2). The null hypothesis (H0) and the re- search hypothesis (HA) were: H0: In the general population, there is no association between student GPA and the number of library sessions at- tended. HA: In the general population, there is a predictable relationship between student GPA and the number of library sessions attended. We followed common statistical prac- tice and set the level of significance (α) as 0.05. In other words, there was a 95 Table 2 Transformation Guideline of GPa Data Category GPa A 3.71–4.00 B 3.31–3.70 C 3.01–3.30 D 2.71–3.00 E 2.31–2.70 F 2.01–2.30 G ≤ 2.00 Table 3 The Sample Groups That are Proven to Have Positive association in Their Population Sample Group Association Strength Affiliated Faculty/School Discipline Level Eng Lang & Lit UGS 0.50 (Strong) Faculty of Arts Mathematics UGS 0.35 (Medium) Faculty of Science Film UGS 0.33 (Medium) School of Communication Chi Medicine UGS 0.31 (Medium) School of Chinese Medicine Computer Sc UGS 0.27 (Mild) Faculty of Science Human Res Mgt UGS 0.26 (Mild) School of Business Social Work UGS 0.26 (Mild) Faculty of Social Sciences Sociology UGS 0.23 (Mild) Faculty of Social Sciences Human Res Mgt GS 0.21 (Mild) School of Business Accountancy UGS 0.15 (Mild) School of Business Communication Std GS 0.13 (Mild) School of Communication Measuring Association between Library Instruction and Graduation GPA 469 percent chance that any observed statis- tical differences were real and not due to chance. Since the Chi-Square (χ2) Test for Independence is limited to identify- ing an association’s presence or absence, Cramer’s V statistics were then used to determine the strength of the association if it existed. The most commonly used interpretation of Cramer’s V value was established by Cohen in 1988: mild=0.10, medium=0.30, and strong=0.50.25 Findings Among the 45 valid sample groups, 11 of them (24.5%) were statistically proven to have a positive relationship between GPA and WKS in the corresponding popula- tion. They are listed in table 3 in order of association strength. No sample groups were found to have a negative association between the two variables. We found a very interesting result after we classified the sample groups based on the number of library sessions offered. (Please see figure 1.) Results showed that if more library sessions were offered to a sample group of students, there was a higher chance for that sample group to have a positive correlation between GPA and WKS. In fact, there was only one sample group provided with five library sessions; it was . This sample group was not only proven to have a positive correlation, but it also had a unique “strong” correlation. Before we move on to the discussion, we want to restate that correlation analy- sis can only reflect whether or not a rela- tionship exists between two sets of data, as well as the strength of that relationship; it cannot determine or show whether a cause-and-effect relationship exists. The purpose of this study was to investigate if a meaningful correlation existed between library workshop attendance and GPA, but not to identify the cause of any such relationship. Discussion A discussion of the results requires an understanding of the local library in- struction environment during the years under analysis. Within each program (for instance, or ), sample groups are offered a certain number of library workshops, some of which are required and some of which are not. For example, an under- graduate student must take the library orientation session offered to all incom- ing students as part of their University Life course in first year (makeup sessions FIGure 1 The Percentage of Sample Groups That Have Positive association in Their Population, Grouped by the Number of library Sessions Offered 100% 50% 43% 22% 15% 50% 57% 78% 85% 0% 20% 40% 60% 80% 100% 5 Library Sessions Offered 4 Library Sessions Offered 3 Library Sessions Offered 2 Library Sessions Offered 1 Library Session Offered Without Associa�on With Posi�ve Associa�on 470 College & Research Libraries September 2011 must be attended if the initial workshop is missed for some reason). Beyond this common requirement, some undergradu- ate programs have a required research methods course that might include library workshops integrated into the course. Elective courses in the program might also integrate library workshops as requested by the instructor. Finally, library work- shops might be offered outside a course, such as the “Final Project” workshops that are organized for various depart- ments on request, and are recommended but not required for students to attend. Within this instructional environment, the number of relevant workshops offered to students will depend on their program, such that an English student might have the opportunity to attend five workshops throughout the course of his or her study, while a Biology student might only have the opportunity to attend two workshops. Our findings, at a surface level, were disappointing in that only 24.5 percent of the sample groups (11 of 45) showed a positive association between workshops attended and GPA. Finding that, for 75.5 percent of our sample groups (34 of 45), library workshops did not have any positive associations with GPA made us ask many questions. Are our workshops ineffective? Should we radically revise them? Should we move away from library workshops and move toward some other instructional model? Should we abandon our instructional efforts altogether, given their seemingly limited impact? Do some sample groups (based on program) ben- efit from library instruction because the nature of the program itself grades library research skills in some more explicit way, thus affecting GPA? The questions we asked came from many angles and are certainly worthy of further consideration and investigation. However, once we clas- sified our sample groups based on the number of library workshops that were offered to each of the groups, we found that the results of our study were neither surprising nor particularly disturbing. We found that, with an increased number of workshops offered to a group, there is a higher tendency of having a positive im- pact on student GPA. This is no surprise to instructional librarians who forever lament that one or two instructional ses- sions are simply not adequate. Our findings showed that, for pro- grams that offered only one library work- shop to students throughout their time at HKBU, only 15 percent of these programs showed a positive association between students attending a workshop and their final GPA. When two workshops were offered to a program, there was still little impact, with only 22 percent of programs showing a positive association. However, once three or four workshops were of- fered in a program, we started to see that almost 50 percent of programs were showing a positive association between their students’ workshop attendance and their final GPA. Finally, when five workshops were offered to one sample group, there was not only a positive as- sociation, but it was the only group that had a “strong” association. Though different programs surely have different needs when it comes to the nature and extent of library and in- formation skills, and some programs may benefit less than others from increased instruction, this overall pattern is of in- terest. It is not surprising that literature students would benefit from more library instruction with respect to their GPA, and that math or visual arts students might not see their information skills reflected in their GPA as clearly. Still, as all of higher education becomes more inquiry-based with the goal of producing students with an ability to independently stay current in their fields, information literacy is be- coming more important to all disciplines, especially as we expand our definition of information literacy beyond traditional literature searching. The large sample size of this study should be emphasized. Most assessment efforts looking to find associations be- tween library instruction and academic performance (such as those described Measuring Association between Library Instruction and Graduation GPA 471 in the literature review section of this paper) analyze one sample group in a given course or program, of a size that generally does not exceed a few hundred. This study has 45 sample groups and over 8,000 students; we could treat it as 45 individual studies, each of them similar in size and scope to many of the studies published in the previous literature. For example, the group had 171 students in the population and showed a strong positive association between workshop attendance and over- all GPA, having been offered five relevant library workshops throughout the course of their program. had 514 students with a mild association strength, having been offered four work- shops; had 114 students and no association, having been offered one workshop. An analysis of previous studies look- ing for relationships between library in- struction and student performance show inconclusive results, with some studies finding a positive association and some studies finding no such association. This study is unique in its comprehensive- ness and can be viewed as encompassing many smaller studies in one. Within this scenario, we would find inconclusive re- sults as well, with some groups showing a positive association and some groups showing no association. However, with the broad view that this study provides, we have been able to identify one factor that has a predictable effect on the likeli- hood of a group having a positive associa- tion between workshop attendance and GPA: number of workshops provided. We have identified that, for groups with one or two workshops provided, as a group, they have little likelihood of showing a positive association; and the likelihood of association, and strength of association, increases with the number of workshops provided. This provides evidence that instruction librarians can use as they work to build instruction programs that have an actual and measurable impact on student performance and success. Limitations The major limitation of the study relates to the statistical method itself. Correla- tion analysis can study and “consider” the chosen variables only, but ignores all other possible factors such as student motivation, instructor requirements for assignments, or variability in grading among programs. As instruction librar- ians, we are fully aware that the extent to which faculty members consider library research skills or citation skills as important in their programs will definitely affect the association results. Take an extreme example: if the GPA of a whole program depends solely on tests and examinations (with no assignments, term papers, or other factors), then library instruction programs can hardly affect their students’ academic performance. Unfortunately, this correlation analysis cannot take these factors into consider- ation. However, the overall picture that this study provides can inform future studies that might wish to differentiate between programs that are more depen- dent or less dependent on information literacy skills. (We should be careful not to assume that information literacy is equivalent to traditional literature-based research in today’s environment. For ex- ample, visual arts students may not need traditional literature searching skills, but they may need to find a wide variety of text- and non–text-based information to help contextualize their creative works.) Another limitation of this study relates to the mix of required and voluntary at- tendance in the library workshops. The instructional offerings in a given program might range from absolutely mandatory (no graduation without attendance), to course-based workshops (as mandatory, or not, as any other lecture or tutorial), to voluntary attendance at a recommended workshop outside classroom time. This makes it difficult to ascertain how great the effect of self-selection might be on our results. It should be noted, however, that, in the Hong Kong environment, students generally attend course-based workshops 472 College & Research Libraries September 2011 that do make up for the majority of the workshops offered. Conclusion The findings of this study suggest that one or two discrete library workshops offered within a program have very little impact on overall student success as measured by final GPA. Three or four library workshops offered within a program are more likely to result in a positive association with GPA, with ap- proximately half of the programs offer- ing this number of workshops showing a positive association with GPA. Though only one sample group had five library workshops offered, this sample group was particularly convincing, as it con- sisted of a sample of 171 students and showed a “strong” association between higher number of workshops attended and higher GPA. As noted in the Introduction, our instructional program is currently in the midst of a fundamental shift. This study has made use of data from an era of discrete, one-shot, skills-centric work- shops. This is not the ideal instructional scenario, and yet we still found a positive association between library instruction attendance and GPA, as long as a certain number of instructional workshops were offered within a program. In the last two years we have gradually moved to an integrated, outcomes-based model of instruction that incorporates different types of instruction, at different points, in a scaffolded and coordinated approach to developing information literacy. We believe that this new model of library in- struction will have an even greater impact on student learning, and thus academic performance, in more program areas (sample groups) in the future. We hope to replicate this study in the future to determine if our new instruc- tional program brings positive associa- tions between library instruction and GPA to more sample groups and/or to see if the strength of associations is increased. We would welcome replications of this broad study at other institutions to see if a similar pattern would emerge: that is, that when there are “enough” library instruc- tion interactions throughout the course of a program, there is a higher tendency that participation in such interactions will impact positively on student GPA. Notes 1. Shun Han Rebekah Wong and T.D. Webb, “Uncovering Meaningful Correlation between Student Academic Performance and Library Material Usage,” College and Research Libraries 72, no.4 (2011): 361-70. 2. Linda G. Ackerson and Virginia E. Young, “Assessing the Relationship between Library Instruction Methods and the Quality of Undergraduate Research,” Research Strategies 9, no. 3 (1991): 139–41. 3. Karen Hovde, “Check the Citation: Library Instruction and Student Paper Bibliographies,” Research Strategies 17, no. 1 (2000): 3–9. 4. Janet Webster and Loretta Rielly, “A Library Instruction Case Study: Measuring Success from Multiple Perspectives,” Research Strategies 19, no. 1 (2003): 16–32. 5. Beth A. Mohler, “Citation Analysis as an Assessment Tool,” Science & Technology Libraries 25, no. 4 (2005): 57–64. 6. Lorrie A. Knight, “Using Rubrics to Assess Information Literacy,” Reference Services Review 34, no. 1 (2006): 43–55. 7. Susan Hurst and Joseph Leonard, “Garbage In, Garbage Out: The Effect of Library Instruc- tion on the Quality of Students’ Term Papers,” Electronic Journal of Academic & Special Librarianship 8, no. 1 (2007). Available online at http://southernlibrarianship.icaap.org/content/v08n01/hurst_s01. htm. [Accessed 11 May 2010]. 8. N.N. Edzan, “Analysing the References of Final Year Project Reports,” Journal of Educational Media & Library Sciences 46, no. 2 (2008): 211–31. 9. Stacey Knight-Davis and Jan S. Sung, “Analysis of Citations in Undergraduate Papers,” College & Research Libraries 69, no. 5 (2008): 447–58. 10. David F. Kohl and Lizabeth A. Wilson, “Effectiveness of Course-Integrated Bibliographic Measuring Association between Library Instruction and Graduation GPA 473 Instruction in Improving Coursework,” RQ 27, no. 2 (1986): 206–11. 11. Mark Emmons and Wanda Martin, “Engaging Conversation: Evaluating the Contribution of Library Instruction to the Quality of Student Research,” College and Research Libraries 63, no. 6 (2002): 545–61. 12. Lynda Dodgen, Sarah Naper, Olia Palmer, and Adrian Rapp, “Not so SILI: Sociology Information Literacy Infusion as the Focus of Faculty and Librarian Collaboration,” Community & Junior College Libraries 11, no. 4 (2003): 27–33. 13. Hurst and Leonard, “Garbage In, Garbage Out.” 14. Amy Dykeman and Barbara King, “Term Paper Analysis: A Proposal for Evaluating Bib- liographic Instruction,” Research Strategies 1, no. 1 (1983): 14–21. 15. Andrew M. Robinson and Karen Schlegl, “Student Bibliographies Improve when Profes- sors Provide Enforceable Guidelines for Citations,” portal: Libraries and the Academy 4, no. 2 (2004): 275–90. 16. Rui Wang, “The Lasting Impact of a Library Credit Course,” portal: Libraries and the Academy 6, no. 1 (2006): 79–92. 17. Donna Corlett, “Library Skills, Study Habits and Attitudes, and Sex as Related to Academic Achievement,” Educational and Psychological Measurement 34, no. 4 (1974): 967–69. 18. Patricia Senn Breivik, Open Admissions and the Academic Library (Chicago: American Library Association, 1977). 19. John Cornell Selegean, Martha Lou Thomas, and Marie Louise Richman, “Long-Range Effectiveness of Library Use Instruction,” College & Research Libraries 44, no. 6 (1983): 476–80. 20. Deborah Moore, Steve Brewster, Cynthia Dorroh, and Michael Moreau, “Information Competency Instruction in a Two-Year College: One Size Does Not Fit all,” Reference Services Review 30, no. 4 (2002): 300–06. 21. Priscilla Coulter, Susan Clarke, and Carol Scamman, “Course Grade as a Measure of the Effectiveness of One-Shot Information Literacy Instruction,” Public Services Quarterly 3, no. 1 (2007): 147–63. 22. Joseph R. Matthews, Library Assessment in Higher Education (Westport, Conn.: Libraries Unlimited, 2007). 23. Barry H. Cohen and R. Brooke Lea, Essentials of Statistics for the Social and Behavioral Sciences (Hoboken, N.J.: John Wiley & Sons, 2004), 199. 24. Gregory W. Corder and Dale I. Foreman, Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach (Hoboken, N.J.: John Wiley & Sons, 2009), 155. 25. Jacob Cohen, Statistical Power Analysis for the Behavioral Sciences (Hillsdale, N.J.: Lawrence Erlbaum Associates, 1988).