Gilstrap.indd Assessing Learning, Critical Reflection, and Quality Educational Outcomes: The Critical Incident Questionnaire Donald L. Gilstrap and Jason Dupree This research study incorporates Brookfield’s Critical Incident Question- naire (CIQ) as a qualitative instrument to assess the ACRL Information Literacy Competency Standards for Higher Education in one library’s instructional curriculum. A sample (n=348) of English Composition II stu- dents was studied over the course of two semesters during a four-session instructional program. A methodological framework of critical reflection, incidents, and events was incorporated, as well as reflection on practice. Results of the study showed the CIQ was effective in supporting qualita- tive methods for assessment of critical reflection in general and the ACRL Standards specifically during the research and learning process. n 1989, the American Li- brary Association first main- streamed the importance of critical thought in informa- tion literacy.1 This focus on thinking critically was later developed further by the implementation of the Association of College & Research Libraries (ACRL) Information Literacy Competency Stan- dards for Higher Education in 2000.2 As a result, critical thinking skills have become increasingly important in library and information literacy programs over the past 15 years. Recognizing the importance and complexity of this pedagogical con- struct, commi ee members of the ACRL did not define the term critical thinking as a static and prerequisite “thing” that exists in classroom se ings waiting to be discovered. Rather, thinking and discern- ing critically incorporates a process of experience and analysis toward which we guide our students as they become information-literate consumers as well as producers of information. Encompassing the related concepts of critical reflection and critical incidents, the goal of this re- search study was to facilitate and assess the process of critical thinking in a library instruction program, as well as to reflect on our own practice of teaching informa- tion literacy. In 2002, the Southwestern Oklahoma State University (SWOSU) Libraries re- structured its entire library instruction curriculum to integrate specifically the Donald Gilstrap is Associate Dean of Libraries for Technical Services, University of Oklahoma Libraries; e-mail: gilstrap@ou.edu. Jason Dupree is Head of Public Services, Southwestern Oklahoma State University Libraries; jason.dupree@swosu.edu. The authors wish to thank Janet Grabeal, Assistant to the Director, SWOSU Libraries, for her assistance in compiling data used in this study. 407 mailto:jason.dupree@swosu.edu mailto:gilstrap@ou.edu 408 College & Research Libraries September 2008 new ACRL Standards. The new cur- riculum identified three tiers of instruc- tional levels. Tier 1 targets first-year undergraduates in Freshman Orientation and/or English Composition I courses, where library instruction focuses on the library’s services, resources, and Web site during a single session. Tier 2 consists of multiple sessions, integrated into the English Composition II courses that traditionally consist of first- and second- year students. Library instruction in the second tier elaborates on the library’s services and resources, emphasizes searching techniques, details the process of evaluating information, and allows students time to work with resources and research concepts through self-directed and instructor-mediated environments. Research-based, upper-division, and graduate courses constitute Tier 3. This tier serves graduate and undergraduate students who are focusing on research within their academic major. At this level, library instruction a empts to expand a student’s knowledge of informational resources within his or her degree field, stresses advanced searching techniques and elaborates on critical thinking and reflection by asking students to consider legal, social, and ethical uses of informa- tion. Each of the Tier levels in the SWOSU Libraries Instruction Program utilizes a curriculum outline, following the ACRL Information Literacy Competency Stan- dards for Higher Education. A er we had developed the curriculum, we wanted to identify a method of assessing student learning, as well as to help us continually improve our teaching. We have, therefore, chosen a method that assesses the ACRL Standards specifically and critical reflec- tion during learning in general. Literature Review In our development of a theoretical frame- work for assessing the critical thinking process, this study relies on the works of Brookfield,3 Tripp,4 and Woods.5 During the 1980s, Stephen Brookfield6 produced a seminal work, The Skillful Teacher. This book has risen to a level of wide embrace in teaching circles as a means for reflect- ing on practice. It has been incorporated into library science and educational the- ory courses, and it has become a staple in many university faculty professional development collections. Brookfield developed his theoretical framework for the CIQ, used as the assessment instru- ment for this study, by borrowing from Daloz’s7 focus on transformational learn- ing, Tripp’s8 research on critical incidents, and Woods’9 development of critical edu- cational events. Consequently, Brookfield argues for a more personalized approach to teaching that moves the instructor to- ward the role of active participant in the learning process, much like the teaching of manipulative and cognitive processes in information literacy as described by Henri and Dillon.10 In Brookfield’s view, those who teach can draw upon both their students’ and their own sources of criti- cal reflection through journaling, group work, and discussion of classroom events. The development of Brookfield’s work in this area seems to have coincided with the emphasis placed on critical thinking in the ACRL Standards. Understanding Brookfield’s devel- opment of the CIQ as the assessment instrument used for the study begins with a brief overview of Tripp’s11 seminal work on critical pedagogy. Tripp defines critical incidents, which emerge through the critical reflection process, in the fol- lowing way: Critical incidents are not ‘things’ which exist independently of an observer and are awaiting discovery like gold nuggets or desert islands but like all data, critical incidents are created. Incidents happen, but critical incidents are produced by the way we look at a situation: a critical incident is an interpretation of the significance of an event. To take something as a critical incident is a value judgment we make, and the basis of that judgment is the sig- http:Dillon.10 Assessing Learning, Critical Reflection, and Quality Educational Outcomes 409 nificance we a ach to the meaning of the incident.12 Tripp also contends that critical in- cidents do not have to be dramatic or unusual. O entimes, critical incidents seem nothing more than common ev- eryday events “but are rendered critical through analysis,”13 as is perhaps the case in many of the information literacy courses we teach. Brookfield’s development of the CIQ also relies on the work of Woods,14 who argues for a more radical approach to studies of critical reflection. In Woods’s view, critical educational events are cata- lysts for transformative development of both students and teachers. In case study research, Woods and others have found that critical incidents have the potential to be “highly charged moments and epi- sodes that have enormous consequences for personal change and development.”15 Segal has also proposed a complimentary perspective on critical incidents and criti- cal reflection during the learning process.16 Segal describes how assumptions and habitual practices that are disrupted lead to reflection or defensiveness during learning. Equally, the more recent works of Mezirow17 on perspective transformation show how a disorienting dilemma serves as a catalyst for critical reflection, usually involving something dramatic. Conse- quently, Segal notes that the very process of critical reflection can cause feelings of uneasiness among learners and can lead to apparent defensiveness in responses during the process of critical reflection, a phenomenon that is shown in more detail in the findings section of this article. Several scholars in the social and be- havioral sciences have performed studies of critical incidents through reflection using this framework for investigation in recent years. Drawing from Schön,18 the works of Thiel19 and Miller20 have utilized critical incident techniques in biographi- cal writing on classroom experiences to reflect on teaching practice. Weinreich’s21 study incorporated Brookfield’s CIQ to assess the impact service learning had on collaboration among students during team projects. Equally, Angelides and Ainscow22 and Jackson and Wasson23 develop Tripp’s framework of critical incidents, addressing culture and how it impacts the learning environment. Additionally, Farmer24 focuses on the underlying social contexts that drive the critical reflection process, while Pugh and Bergin’s25 studies show how students reflect on knowledge gained in the classroom and apply it in the outside world. Moreover, Walton and Ne leton26 borrow from Schön’s27 work on reflect- ing on practice to develop a conceptual model that focuses on critical reflection in the classroom through the integration of students’ classroom experiences with the research process.28 And, finally, Brookfield has drawn on the use of critical incidents in numerous autobiographical and eth- nographical studies of critical reflection in areas ranging from teacher education to the health professions.29 These types of research studies, there- fore, add further support for the use of critical incident methods to assess the teaching and learning processes of infor- mation literacy programs. Additionally, these studies relate uniquely to the ACRL Standards’ focus on applying the knowl- edge gained from library instruction to influence events outside the classroom se ing.30 However, a empts to assess this based on the ACRL Standards provide us with unique challenges and oppor- tunities. Although it is argued critical incidents emerge in different ways, it is agreed that they are unpredictable, are difficult to control, contain simultane- ous problems and solutions, and can last from a very short period to many weeks.31 Therefore, we feel that a qualitative tool of discovery, such as has been developed by Brookfield with the CIQ, presents educa- tors with an opportunity to incorporate new assessment methods that seek to develop further understanding of the critical reflection process in general, while identifying ACRL Standards, Outcomes, http:weeks.31 http:se�ing.30 http:professions.29 http:process.28 http:process.16 http:incident.12 410 College & Research Libraries September 2008 and Performance Indicators in informa- tion literacy programs specifically. Research Design For the purposes of this study, we have chosen a qualitative method as a basis for our research design. Brookfield’s exten- sive investigation into the roles of critical reflection in teaching and learning pro- cesses led to the development of the CIQ.32 The primary purpose of this instrument is to assess student critical thinking and subsequently reflect on these findings as a source of professional development by teachers. This study uses a modification of the CIQ to collect data on the critical reflection process during library instruc- tion sessions at our library. All sections of semester-long English Composition II courses at this university were chosen for study for a period of one full academic year. We determined that the entire population (P=752) of English Composition II students at this university would be advantageous for a number of reasons. Primarily, we felt the population would provide more robust and compre- hensive data when analyzing the complex- ities of the critical reflection process in our new, ACRL Standards–based curriculum. Equally, we had already introduced the tier approach to library instruction into the English Department on campus, focusing on the Tier 2 level of our Library Instruc- tion Program for these courses. A very productive and collegial rela- tionship had been developed between English and Library faculty, and we had integrated a four-session curriculum into most English Composition II courses. Students would come to the library dur- ing four different class periods, receiving instruction on information literacy and academic research. Library Instruction sessions included traditional lecture, group work, active learning activities, and a time for critical reflection at the end of each class. Additionally, due to the typically one-session duration of Tier 1 freshman orientation and Tier 3 upper division and graduate classes, we believed at the time it would be harder to integrate the CIQ instrument and still come away with rich and descriptive data that incorporated the complexities of a multisession approach. A er students completed a short de- mographic questionnaire (see Appendix A) and informed-consent documents, the first instruction session focused on intro- ducing the research process and applying it within the lecture material. Additionally, we incorporated a theoretical component on database architectures: showing how databases store and retrieve informa- tion, introducing database searching techniques, and applying this knowledge through the use of the online catalog to search for books. The second session included a review of the first session, lecture and discussion on the differences between popular magazines and scholarly journals, and instructor demonstrations and student application of both serials aggregation and journal article databases. The third session included a recursive review, focused on news sources and the use of relevancy ranking, and utilized newswire services and government in- formation. And the fourth session began with review and then integrated group activities and discussion on the strengths of the open versus the hidden Web, an examination of search engines and tools used to locate information on the Web and how they operate, the process of evaluat- ing information sources, and a concluding discussion on examples where authors failed to conduct thorough research. At the end of each of the four sessions, CIQs were distributed to students with five basic questions (see Appendix B). Stu- dents had time to reflect on critical inci- dents they felt had taken place during the learning process in that session and write brief responses to describe these ideas. Li- brarians teaching the classes would then collect the CIQs at the end of the class and place them in an envelope. Before the next day’s session, each instructor would glean the CIQs and look for issues or ideas they perceived to be critical incidents in their Assessing Learning, Critical Reflection, and Quality Educational Outcomes 411 teaching of information literacy. The next session would incorporate follow-up on the previous lesson to ensure that compo- nents were being taught clearly. In Tripps’s view, reflective teaching is just as important as reflective learning among participants. Tripp also argues that, when studying critical reflection, it is difficult to “move from a conclusion to data rather than the other way,”33 and that continual analysis of the data aids in more accurate representations of critical incidents. This idea supports our choice of qualitative investigation for this study and complements the framework from which Brookfield originally developed the CIQ. Equally, Denzin and Lincoln34 have been cited frequently in the literature in their description of data triangulation as a va- lidity procedure in qualitative research. To decrease the influence of the researchers on the study findings, we have followed their approach, using multiple methods and multiple researchers throughout the study. During different stages of this re- search, instructors performed journaling activities where we reflected on learning exchanges in dialogue between teachers and students—or procedural issues deal- ing with the study—and logged these in individual journals. We also relied on the methods encouraged by Brookfield,35 Hoover,36 and Schön37 to reflect critically on our practice of teaching by meeting periodically to discuss each other’s ob- servations while integrating our views into a coherent “master journal.” Library instructors were given full data sets that contained all of the CIQ responses. All student IDs had been stripped from these files, and any in- formation that identified the instructor teaching the class or the corresponding instructor was also removed. Instructors then worked independently with the data sets to develop clustering of the general themes that emerged from each of the sessions, similar to the approach chosen by Kracker and Wang.38 Equally, a er instructors had developed their general themes, we met to compare and discuss each other’s themes as additional meth- ods for data checks suggested by Denzin and Lincoln.39 Notes were recorded of the observations of each of these meetings, and group clustering took place to syn- thesize the major themes we observed for the findings section of this article. Results and Discussion of Findings Group interpretation of the data led to clusters of themes that emerged through- out this study for each question. Out of the population surveyed (P=752), we received a total response rate of 58% (N=433), and a study response rate of 46% (n=348) for those students who completed informed-consent forms. We viewed this as a very successful response rate, particularly for a qualitative study.40 A portion of the data reflected shallow description for each of the questions. Answers ranged from “I understood everything” in Question 1 to “None” or “N/A” in Question 2 , Question 4, and Question 5. However, inferences can be made about these types of responses: students are apathetic about the grow- ing amount of survey research in higher education, or that students’ perceptions of their research abilities are higher than testing of these abilities would sug- FIGURE 1 Participated in Library Instruction Session Previously Yes 36% No 61% Didn't respond 3% http:study.40 http:Lincoln.39 412 College & Research Libraries September 2008 FIGURE 2 Responses by Age 17 18 19 20 21 22 23 24 25 26 27 28 32 33 34 38 45 Didn't Respond 0 20 40 60 80 100 120 140 17 19 21 23 25 27 32 34 45 gest, as has been shown by Dunn41 and Maughan.42 Therefore, arguments could be made that the critical reflection pro- cess is not taking place for these students, the process is not apparent in their de- scriptions, or there has not been enough background content knowledge on the critical reflection process. Conversely, it can also be inferred that librarians have performed content delivery well, FIGURE 3 Responses by Sex Didn't respond 3% Male Female 56% 41% which supports ACRL Information Lit- eracy Outcomes, and, as a result, fewer students related descriptions of critical incidents. Equally, we expected that the number of students identified as having participated previously in Tier 1 sessions (see figure 1) might have contributed to this phenomenon. Further research would be necessary to investigate the questions surrounding the lack of depth in these particular responses. However, since we chose an entire population for our data collection, we viewed the fact that a large portion of students responded with rich descrip- tion as a success for qualitative studies, as Denzin and Lincoln43 have shown. As most researchers conducting qualitative studies will note, the richness of these data cannot comprehensively be ex- plained within the confines of an article. Some might suggest that qualitative data analysis provides avenues for infinite ob- servations and descriptions of participant responses.44 Descriptive statistics have also been used for exploratory purposes and to generate future hypotheses (see http:responses.44 http:Maughan.42 Assessing Learning, Critical Reflection, and Quality Educational Outcomes 413 FIGURE 4 Responses by Identified GPA 0 20 40 60 80 100 120 3.5-4.0 3.0-3.4 2.5-2.9 2.0-2.4 1.9 or below Didn't respond figures 1–6). Through the process of cod- descriptive vigne es of student responses ing and recoding data, we have tried to that help support analysis of our observa- identify general themes that emerged tions and reflection on our teaching prac- during this study and provide very short, tice. Standards, Performance Indicators, FIGURE 5 Responses by Academic Level 0 50 100 150 200 250 Freshman Sophomore Junior Senior Didn't Respond 414 College & Research Libraries September 2008 FIGURE 6 Transfer Student Yes 9% No 88% Didn't respond 3% and Outcomes45 from the ACRL Infor- mation Literacy Competency Standards for Higher Education will be referred to throughout the results of this study, and we suggest referencing this document while reading the findings. Results for Question 1 Learning The main themes that emerged from re- searchers coding responses for this ques- tion deal with learning, searching, time, and evaluation of resources. The learning process seemed to be exemplified as the main source of critical reflection. Pri- marily, students responded most o en to issues surrounding the application of knowledge. Many students referenced their feelings of confidence about under- standing the content of the day’s session when they were able to apply what the instructor taught. This proclivity toward kinesthetic learning by the students came in the form of applying new knowledge to worksheet activities, interacting in group exercises, teaching the teacher, and hands-on computer activities, which reflect standards 1.1, 2.2, and 3.6. Students noted this through statements such as, “I understood what we learned when we did our activity,” “when we got to practice what we were being taught,” and “I think everything is becoming clearer now.” Statements such as these reinforce the power of in-class exercises to help students synthesize the concepts they are learning and apply them to the actual research expected of them in their coursework. Visual learning also served as a cata- lyst for many of the students’ reflections on learning. Graphical representations and models used in PowerPoint presen- tations appeared to be an effective form of conveying information theory–based concepts through visual media. As one student noted, “I really understood the power point slide. It allows me to see what you’re talking about rather than to just hear it.” Moreover, generative metaphors were used in the teaching process that helped bring out deeper criti- cal reflection on student learning. When teaching components of the research process, a ladder was used as a visual representation. When describing knowl- edge acquisition, this metaphor returned repeatedly. One student commented, “I felt like I most understood the slides, es- pecially the ‘research ladder’ slide.” Using visual metaphors to teach poetry research, TABLE 1 Results For Question 1: At What Moment in the Class Today Did You Feel Like You Most Understood the Instructional Content? Question 1 Theme Responses, n = All/everything 95 Learning 76 Boolean/search strategies 67 Time 41 Evaluation 35 N/A or illegible 14 Miscellaneous 20 TOTAL 348 Assessing Learning, Critical Reflection, and Quality Educational Outcomes 415 another student noted, “while looking at the poetry, I realized what the instructor wanted us to do in our research.” Another recurring concept in the learning theme was the use of review during closing and subsequent sessions, as students noted this technique increased their recall of previously learned knowl- edge. Students commented particularly that the incorporation of role-playing, where students teach the teacher, was an effective method to increase recall. And, conversely, reflections on feelings of confusion emerged. Albeit these feel- ings were infrequent for this question, students showed that, although they did not understand all of the content, they were empowered in the classroom to re- spond to these phenomena. Equally, these feelings of confusion o en accompanied observations about the complex nature of the research process while using the Internet, supporting Segal, Brookfield, Trip, and Woods.46 Searching Issues surrounding searching also emerged as a major theme for this ques- tion. We recognize the emergence of Web 2.0 technologies and do not necessarily encourage or force students to use Bool- ean searching when more effective natu- ral-language searching is available. How- ever, Boolean logic is an integral theoretical component of this library instruction cur- riculum to build foundational knowledge for database architectures, and the issue of search construction came up most fre- quently for this theme. Librarians felt this is one of the more difficult—or perhaps unexciting—aspects of the curriculum to teach. However, perhaps it is because of this focus on teaching Boolean logic that instructors spend more time on it. Although we expected criticism of teach- ing Boolean logic, much to our surprise, students responded with descriptions of clarity on this subject above many other curriculum components in the searching techniques session perceived by instruc- tors to be less difficult. This was evidenced by comments such as, “I felt like I under- stood the instructional content when the instructor was talking about Boolean operators,” and “talking about Boolean logic is what made sense to me.” Again, we noticed the importance of standards 2.2 and 3.6 to which students alluded in their statements. Other related student reflections on searching dealt with a be er understanding of when to use keyword and when to use subject searching, and how to use article databases and Internet search engines effectively. Time Identification of time periods during the session—such as “beginning,” “middle,” “end”—were also considered moments of understanding by the students. However, since descriptions were limited in these responses, it was difficult to identify exactly which part of the instructional session to which students were refer- ring. Equally, since each instructor taught components at different times during the session, generalization about the time periods could not be made. As a result, this theme had to be downplayed due to insufficient data for analysis. However, some comments alluded to ideas that were usable for pedagogical reflection for the individual instructor such as, “I un- derstood it all a er she completed [his or her] lecture,” in this case before applying knowledge; or “the instructor explained the topics really well so I felt I understood everything at the end of the lesson,” in this case a er applying knowledge. Evaluation of Resources The evaluation of resources emerged as another common theme in the curricu- lum. Part of the curriculum emphasizes the key differences between magazines and journals and their uses for academic research. Based on our feedback from them, students tended to know relatively li le about scholarly journals before par- ticipating in library instruction. However, a er completing the session on scholarly information sources, students commonly http:Woods.46 416 College & Research Libraries September 2008 TABLE 2 Results For Question 2: At What Moment in the Class Today Did You Feel Most Confused About the Instructional Content? Question 2 Theme Responses, n= None/never 182 Boolean/search strategies 44 Learning 37 Time 33 Evaluation 22 N/A or illegible 14 Miscellaneous 16 TOTAL 348 reported a be er understanding of how popular and scholarly publications dif- fer—emphasizing standard 1.2.d—and how journals contribute to the research activities of university faculty. This same type of reflection came from analysis of Web-based sources. A er comparing government and education domains with propaganda and parody sites, such as martinlutherking.org and whitehouse. net, students commented on their realiza- tion that there are indeed vast differences between popular and research-based Web sites available for free access. We felt the students’ observations confirmed the importance of standard 3.2.c, realizing prejudice and deception in information resources. Although the publication of scholarly information is critical to the advancement of academic thought and progress in higher education, we recog- nized how commonly we in university se ings take for granted that students do not fully understand how very different popular and scholarly literature can be. The students’ concrete identification of the importance of evaluation of sources in research emphasized their positive reaction and deeper understanding of these differences when provided with this curriculum. Results for Question 2 Searching Although 67 students responded that the session on Boolean logic and search strat- egies was the moment where they most understand the instructional content, 44, or 13%, of the students also responded that this led to their most confusing mo- ment. However, this confusion seemed to emerge among students for different reasons than in Question 1. One student implied that Boolean logic was not a user-centered form of searching, noting: “…the and, or, and not I understood, but it seems like too much work.” Standard 3.4.e addresses this statement, particularly in regard to determining the limitations of information-gathering strategies. Another student identified that, in spite of the com- plexity of user interfaces, they are not able to infer what the user means: “…when it came to using the Boolean operators in the actual search, I couldn’t think of synonyms for Oklahoma and statehood.” Another ap- proach instructors used was to provide a mathematical metaphor to explain Boolean logic, but one student noted: “…when she started applying it to math with the FOIL method, I just got lost.” These comments helped us to reflect not only on our teach- ing strategies but also on the limitations in the design of systems with which we access information. Time For this question, we were able to distin- guish more information regarding time segments during the class period. Rather than the brief answers that appeared in Question 1, students included informa- tion in Question 2 that helped us identify what section of the class session or why that section stood out, such as “…at the beginning of class because I didn’t know any of the content” or “…at the beginning talking about [word games].” Moreover, students addressed issues of time, deal- ing with the instructional content. Some students commented there was a lot of information presented in a short amount of time; a dilemma with which most of http:martinlutherking.org Assessing Learning, Critical Reflection, and Quality Educational Outcomes 417 us teaching library instruction can sym- pathize. “S/he went too fast on the slides. I didn’t have enough time to write down one of the last slides.” We found reflections such as these to be telling statements. On one hand, we real- ized we might need to adjust the content and approach to teaching this section of library instruction. Yet, on the other hand, we found that these types of comments showed engagement in the instructional session at a level where she or he felt frustrated by not being able to record all of the knowledge being shared. We also feel it is important to note that these phenomena emerged in Question 2 where confusion is identified. Supporting the works of Brookfield, Tripp, and Woods, critical reflection takes place as a result of critical incidents and events.47 Critical reflection by the students on these issues seemed to take place at a deeper level, as a result of their confusion or frustration. Equally, they expanded on why these feelings were important to them. Evaluation of Resources Evaluation of resources was another ma- jor theme that emerged in both Question 1 and Question 2. “I am kind of confused about all of the Web sites as far as finding the correct information I may need,” and “I was most confused about using bal- anced or biased resources. It looks like it would be hard to distinguish.” Another telling statement was “I was most con- fused when we talked about magazines and journals, since I don’t read either of those.” We recognized that, although the content being taught in the curriculum is important, statements such as these might reflect a much different methodology used to access information, principally from the World Wide Web, by new gen- erations of college students and that we might need to adjust the way we approach the section on evaluating magazines and journals to reflect how online resources change student perspectives. The central focus of standard 3.4.e speaks to the generational differences between the Net Generation and previous generations in their determinations of the nature of information by questioning its origins and limitations. Learning For some students, the application of knowledge learned in the sessions was not as easy as it appeared. We found this was particular to difficult topics where keywords are so specific or narrow that databases produced few results. Review- ing search strategies and informational sources, and subsequently integrating new knowledge, is the core methodology of standard 3.7.b–c stressed in our cur- riculum. Other students noted feelings of information overload such as, “exactly where to go when I find what I am look- ing for. There was a lot of information in this session, and I kind of got it all mixed up.” This supports the idea that research is a difficult process, but we felt that, for some students, the amount of informa- tion might be too much. Despite this possibility, standard 1.1.c–d is designed to alleviate anxiety and frustration by en- couraging students to explore resources, develop familiarity with their topics, and achieve a manageable focus regarding their research tasks. Some students also did not understand the group activities TABLE 3 Results For Question 3: What Was Your Most Rewarding Experience in Today’s Class? Question 3 Theme Responses, n = Confidence 97 Boolean/search strategies 77 Evaluation 63 Learning 61 N/A or illegible 13 Library resources 24 Miscellaneous 13 TOTAL 348 http:events.47 418 College & Research Libraries September 2008 or metaphors used to convey meaning of the curriculum concepts. We found, however, that comments of confusion on the types of pedagogy used in the sessions were spread evenly between kinesthetic, auditory, and visual learning. This tends to agree with the reliability of using multiple methods when teaching to ac- commodate diverse learning styles while focusing on experience in learning.48 This phenomenon of diverse learning styles supported our original curriculum objec- tive of including several different learning activities and teaching methods to sup- port the multiple ways in which students diffuse knowledge. Results for Question 3 Confidence Results from Question 3 detailed student perceptions of their most rewarding ex- perience in the day’s library instruction session. By far, the most commonly emer- gent theme for this question surrounds students’ descriptions of improved con- fidence in their library research skills, exemplifying standards 1.2.a, 1.2.c, and 1.4.b. Responses included confidence in library research in general: “I feel more confident about doing research in the library” and “I feel be er prepared to perform library research”; confidence in database searching: “I feel confident I can perform searches more efficiently,” underscoring standard 2.1.d; and stan- dards 3.4.a and 3.7.a develop the students’ confidence in synthesizing and applying knowledge reflected in several statements such as: “When I was able to complete the worksheets by myself!” Evaluation of Resources Evaluation of resources was also iden- tified by students as being their most rewarding experience. These responses supported aspects of standard 3, where students learn to evaluate information, think critically, and incorporate new information into their knowledge base. The following student reflections are well founded in 3.2.a: “learning where to find credible sources for references on the In- ternet” and “that when you use some do- mains, you want to check for accuracy and authority of the author, because some are credible and some aren’t.” But responses on this question also identified common misperceptions we might have about students: “going through the Web sites and seeing how a professional opinion would be be er than a college student’s opinion.” Standard 3.4 highlights the possibilities for students to compare new information with prior knowledge while formulating new conclusions, and stan- dard 3.5 encourages students to reconcile divergent viewpoints and judge what is appropriate or inappropriate. Clearly, these students reflected their understand- ing of these concepts a er engaging in the instructional sessions. Although at face value, this might ap- pear obvious to librarians, we realized how important a concept this has become in the age of blogging and facebook.com. We noted it was interesting that so many students identified evaluation of resources as their most rewarding experience. When we might be too apt to write off newer generations of students in their use of sus- pect resources, we were reinforced that, if librarians provide instruction on evalua- tion—and utilize methods that generate meaning for students—then the students will come to value it to a high degree. The Learning Experience The learning experience was identified as another rewarding theme of the instruc- tional session. One aspect of the learning experience students noted as important was the focus on recursive learning. De- scribed by Doll49 as a continuous return to previous content and experiences in order to create new knowledge and meaning, we actively included review and discussion of the previous session’s content to provide a foundation for the new session. This was evidenced through responses such as “the review” and “be- ing able to understand what we did in the last class be er.” In standard 4.1.b, http:facebook.com http:learning.48 Assessing Learning, Critical Reflection, and Quality Educational Outcomes 419 knowledge and skills are revisited from previous experience, and, more specifi- cally, standard 1.1.d–e addresses the in- formation requirement of the student by defining and modifying that need while standard 2.4.b refines this need based on examination of the data or their absence. Other responses identified visual, kines- thetic, and auditory learning as the most rewarding experiences, with comments ranging from students’ appreciation of the well-constructed PowerPoint pre- sentations, the lecture content, and the worksheet and group activities. A few students noted their future application of this knowledge through statements such as, “when I perform searches anywhere else, I will know how to improve their quality,” a close association to standard 3.7.c; “learning valuable information that will benefit me not only with this research but with other research in the future”; and “I actually learned something that is go- ing to be helpful in the future.” Again, we found support for the ACRL Information Literacy outcomes objective of applying information literacy skills outside the classroom—specifically in standards 1.1.f, 2.2.a, 2.4, and 4.1.b. As a result, we felt TABLE 4 Results For Question 4: What Was Your Most Surprising Experience in Today’s Class? Question 4 Theme Responses, n = None, N/A, or illegible 93 Learning 52 Boolean/search strategies 38 Evaluation 36 Complexity of research 26 Confidence 24 Engagement in class 22 Library resources 17 Miscellaneous 40 TOTAL 348 we had accomplished one of our primary curriculum objectives. Results for Question 4 Learning: Synthesis and Application When evaluating responses for this ques- tion, we noted that learning was identified as the most surprising experience. Stu- dents responded in a way that focused on their newly found abilities to synthesize and apply the knowledge of the lecture and group activities when constructing search strategies and undertaking the research process. Many students ex- pressed their surprise a er recognizing the similarities and differences in data fields that existed among databases, such as thesauri and descriptors. Those responses reflected the beneficial use of standards 2.2.c–f and 2.5.d. Students also commented on how the use of Boolean operators, truncation and wildcards, and limiters could refine their searches, refer- ring back to standard 2.2.d. Students also expressed feelings of surprise in the idea that their research improved greatly by applying these new concepts of searching. During the first session, many students identified, both in their CIQ responses and through comments recorded by instructors a er classes, that they felt confident in their ability to perform research with search engines and did not see a priority for at- tending library instruction classes. A er completing the sessions, however, stu- dents noted surprise in how much more effective their research strategies had become. Search engine proselytes seem- ingly became library database converts! Reflections such as “[I was surprised] how many books are available on my topic,” or “how much information I found on my topic through the databases here in the library,” exhibited the students’ recogni- tion of the value of library resources. This value assigned by the students highlights standard 2.1.c, where students examine the scope and organization of information retrieval systems, drawing conclusions based upon their findings as stressed 420 College & Research Libraries September 2008 in standard 3.4. And, finally, standard 5.1.b facilitates contemplation of free vs. fee-based accessible information and the subsequent issues students do not realize exist. Other students expressed happiness or gratitude in being able to apply this new knowledge to future research. One student noted that “[a er this class] I will be able to look up information for other papers, because now I think it is much easier to do.” As educators, we were also surprised and encouraged that we were making a difference and contributing to the learning process in ways that were unexpected by the students. Evaluation Responses on this question also revealed deeper reflections on the evaluation of Internet resources. Again, at the begin- ning of the library instruction curriculum, several students had commented on their confidence in performing research—using World Wide Web search engines—sup- porting the University of California and California State University system libraries information literacy studies.50 However, librarians provided lecture and group ac- tivities, incorporating a map of the World Wide Web and an evaluation of falsified research, parody, and propaganda Web sites.51 Many students expressed their TABLE 5 Results For Question 5: Was There Information Presented in Today’s Class That is Still Unclear to You? If So, Please Describe Question 5 Theme Responses, n = None, N/A, or illegible 318 Learning 9 Boolean/search strategies 7 Library resources 4 Evaluation 3 Miscellaneous 7 TOTAL 348 surprise at the amount of misinformation and disinformation on the Internet, stress- ing the importance of standards 3.2.a, 3.2.c, and 3.6.a. Equally, as a result of group activities in the classroom and subsequent application of these criteria for analysis, students commented that they now felt more confident about evaluating Internet resources due to a be er understanding of domain use and identifying author credentials, reflecting standard 1.3.b. Results for Question 5 The majority of responses for Question 5 were either left blank, or students responded with “no” or “none.” Those students who did respond, however, ranged in their comments from specific to complex aspects of the research process. Many students discussed specific aspects of library resources, such as limiters in the online catalog or full-text retrieval options in subscription databases as still being unclear. Understandably, students stressed some confusion over when to use particu- lar resources, such the use of CQ Researcher over EBSCO’s Academic Search Premier and vice versa. Standards 1.2.c, 2.2.e–f, and 2.3.a are designed to guide students to differentiate between resources, apply search strategies in various information retrieval systems to increase success, and incorporate discipline-specific techniques when appropriate. Other students, how- ever, noted confusion over Boolean search strategies or the evaluation of resources. We found it difficult to develop themes for this question, as the students’ respons- es related to specific aspects of the day’s class and were o en contingent on their own perceptions of the day’s events in relation to their student peers, instructors, and/or teaching methods. However, we identified this phenomenon as supporting Tripp’s argument that critical reflection cannot be socially and culturally extri- cated from our students and that the class- room realities created by students can be socially constructed.52 Additionally, the comments that students chose to record on information that was still unclear at the http:constructed.52 http:sites.51 http:studies.50 Assessing Learning, Critical Reflection, and Quality Educational Outcomes 421 Ta bl e 6 In se rt T ab le H ea de r H er e Q ue st io n 1 T he m e R es po ns es Q ue st io n 2 T he m e R es po ns es Q ue st io n 3 T he m e R es po ns es Q ue st io n 4 T he m e R es po ns es Q ue st io n 5 T he m e R es po ns es L ea rn in g 76 L ea rn in g 37 L ea rn in g 61 L ea rn in g 52 L ea rn in g 9 B oo le an /s ea rc h st ra te gi es 67 B oo le an /s ea rc h st ra te gi es 44 B oo le an /s ea rc h st ra te gi es 77 B oo le an /s ea rc h st ra te gi es 38 B oo le an /s ea rc h st ra te gi es 7 E va lu at io n 35 E va lu at io n 22 E va lu at io n 63 E va lu at io n 36 E va lu at io n 3 T i m e 10 5 Ti m e 85 L ib ra ry re so ur ce s 24 L ib ra ry re so ur ce s 17 L ib ra ry re so ur ce s 4 C on fid en ce 97 C on fid en ce 24 end of the day’s session were incorporated by instructors to review for the next day’s class, highlighting the recursive learning process integrated into the curriculum. Moreover, we found that those students who did choose to respond seemed to note consistently their realization that the classroom content was not as easy as they had expected and/or that they would need to practice what they had learned outside of class when they had more time to apply their new knowledge. Those reflections reinforce aspects of standards 1.1, 1.3, 2.2, 3.4, and 4.1. Conclusion For the methodological framework of the study, we feel the use of the Critical Inci- dent Questionnaire supported the works of Brookfield53 and Tripp54 throughout this research. Responses generated by students consistently seemed to uphold the theory that critical reflection takes place as a result of critical incidents and events. We also identified Segal’s55 work in the responses of students who noted disorienting or disrupting incidents in the classes and how these led to rich descriptions in the critical reflection process. As an example, when students were confused, critical reflection by the students on particular is- sues seemed to take place at a deeper level as a result of their confusion. Through the process of writing their responses, they expanded on why they were confused, exhibiting an iterative learning cycle that developed through the course of the entire block of instructional sessions. At another level, the things students most understood in Question 1 also came up as emergent themes for concepts they found most dif- ficult to comprehend in Question 2 and/or Question 5. Moreover, their reasons for confusion seemed to differ significantly from those who identified the same ideas as most understandable. We contend this further supported Tripp’s concept of the social creation of knowledge during the process of learning. Furthermore, we suggest that, when evaluating students, using methods that focus on quality in 422 College & Research Libraries September 2008 education, critical reflection analysis tends to support why and how learning is tak- ing place as opposed to measuring only learning content. Another phenomenon that developed during this study is the emergence of themes across questions (see Table 6). Learning, search strategies, and evalua- tion were themes that remained constant in each of the questions. Library resources as a theme spanned Questions 3, 4, and 5, while time was identified in both Questions 1 and 2. Confidence equally emerged as a theme in both Questions 3 and 4. We found it particularly compel- ling that the concepts of learning and evaluation spanned all of the questions and helped lend support for: 1) the use of the CIQ instrument as a method for assessing the critical reflection process, and 2) that learning and evaluation, two themes one could infer would elicit criti- cal reflection, emerged as a result of the curriculum and pedagogy. We are unsure why the themes of library resources, time, and confidence emerged consistently in their respective questions and believe more research would be necessary to investigate these phenomena. It is inferred that Woods56 was sup- ported in some aspects of the study through reflections that were more radical in nature. However, we do not believe this support was strong for this study. This could be due to the limitations of the CIQ instrument and the limited time frame for observation, reflection, and discovery of more radical critical events. An equally frustrating dilemma we faced was the large number of “nonresponses” from respondents. These ranged from “N/A” to “yes” or simply blank responses for some of the questions. We can a empt to make assumptions about why this hap- pened: apathy toward survey research by students, a lack of engagement in the class, or full learning engagement and un- derstanding of the curriculum outcomes. However, we feel further research would need to be conducted to help explain why students responded in this way. Another aspect of this study that became somewhat problematic for the researchers was the size and scope of the population. On the one hand, we felt it important to extend participation to an entire population to gain description of reflection at a level and scope extensive enough to analyze the critical reflection process. Although this was not an insur- mountable a empt, we quickly realized that the size of the data sets from the sample (n=348) were quite large and presented a lengthy period of analysis. We feel that it might be beneficial to in- corporate the use of the CIQ only once, at the end of the summation of sessions. However, we first addressed this pos- sibility at the beginning of the study, and we believed that a true assessment of the curriculum would require the use of the CIQ a er each session. Otherwise, student recall of the previous sessions would be minimized, skewing the data primar- ily toward reflection on the last session when the CIQ would be performed. In a empting to apply the CIQ as an as- sessment instrument at other institutions that incorporate multisession informa- tion literacy or instructional technology curricula, we might recommend the use of stratified sampling among different courses to increase the timeliness of data analysis. Additionally, we believe the use of the CIQ might be valuable as an effec- tive assessment method for information literacy curricula that incorporate single sessions for each course. The main limitation of this study sur- rounds the concept of critical reflection in general. The theoretical framework relies on the qualitative interpretations of the researchers. Although multiple methods were utilized to limit the influence of the researchers’ subjectivity throughout this study, it cannot be discounted.57 Equally, it should be recognized that each campus and each student brings with them unique sociocultural perspectives and academic experiences that influence student re- sponses in a study such as this. Although generalizations about the findings of http:discounted.57 Assessing Learning, Critical Reflection, and Quality Educational Outcomes 423 this study to other populations might be inferred, it is not our intent to apply this knowledge across undergraduate students in library instruction. Moreover, descriptive statistics were used only for exploratory purposes for our own reflection on practice and to assist in de- veloping hypotheses for future studies of information literacy. In conclusion, we believe the CIQ proved to be an effective qualitative instrument to assess critical reflec- tion and critical incidents during the process of learning. Although this study focuses on information literacy curriculum, similar research could be performed in any program that is complementary in nature in terms of time segments and instructional objectives, such as in writing centers, instructional technology programs, or computer literacy courses. Equally, we feel this assessment approach would p r o v e e f f e c t i v e f o r s e m e s t e r - l o n g courses in information literacy, Library Science, or other social and behavioral sciences disciplines. As was detailed in the responses to each of the ques- tions, themes that emerged showed the process of critical reflection taking place consistently between instructors and throughout each of the sessions. These themes identified the key cur- riculum components of the program and expanded upon them, suggesting that the curriculum was effective in both content and design to support the ACRL Information Literacy Compe- tency Standards for Higher Education specifically and to elicit critical thought during learning in general. Perhaps just as important, the themes we identified were instrumental in our reflections on practice. The knowledge that we gained from this study helped us to further de- velop the curriculum while continuing to focus on both critical reflection and the ACRL Standards in new ways. Appendix A Student ID #: ________________ Please check or write in your responses below: Age _____ 1 Year in school _____ Freshman _____ Sophomore _____ Junior _____ Senior 1 Current or estimated GPA : 1 _____ 3.5–4.0 _____3.0–3.4 _____2.5–2.9 _____2.0–2.4 _____ 1.5 or below–1.9 1 Sex: _____M _____ F 1 Have you participated in a Library Instruction session at the SWOSU Libraries during a previous semester? _____ Yes _____ No Are you a transfer student from a college or university other than SWOSU? _____ Yes _____ No 424 College & Research Libraries September 2008 Appendix B Student ID #: ________________ Instructor #: _______________ In one or two sentences, please respond to each of the questions below about your experiences in today’s Library Instruction class: 1. At what moment in the class today did you feel like you most understood the in- structional content? 2. At what moment in the class today did you feel most confused about the instruc- tional content? 3. What was your most rewarding experience in today’s class? 4. What was your most surprising experience in today’s class? 5. Was there information presented in today’s class that is still unclear to you? If so, please describe. Notes 1. “Presidential Commi ee on Information Literacy: Final Report,” The American Library Association, January 1989. Available online from www.ala.org/ala/acrl/acrlpubs/whitepapers/ presidential.htm. [Accessed 1 July 2008]. 2. “Information Literacy Competency Standards for Higher Education: Standards, Outcomes, and Performance Indicators,” Association of College & Research Libraries, January 2000. Available online from www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm. [Accessed 1 July 2008]. 3. Stephen Brookfield, Becoming a Critically Reflective Teacher (San Francisco: Jossey-Bass, 1995). 4. David Tripp, Critical Incidents in Teaching: Developing Professional Judgment (London: Rout- ledge, 1993). 5. Peter Woods, Critical Events in Teaching and Learning (Washington, D.C.: Falmer Press, 1993). 6. Stephen Brookfield, The Skillful Teacher: On Technique, Trust, and Responsiveness in the Class- room (San Francisco: Jossey-Bass, 1990). 7. L.A. Daloz, Effective Teaching and Mentoring: Realizing the Transformational Power of Adult Learning Experiences (San Francisco: Jossey-Bass, 1986). 8. Tripp, Critical Incidents in Teaching. 9. Woods, Critical Events in Teaching and Learning. 10. James Henri and Ken Dillon, “Learning to Learn: Reflections upon Enquiry, Information Literacy and Critical Thinking,” Australian Library Journal 41 (May 1992): 103–17. 11. Tripp, Critical Incidents in Teaching. 12. Ibid., 29. 13. Ibid., 25. 14. Woods, Critical Events in Teaching and Learning. 15. Patricia J. Sikes, Lynda Measor, and Peter Woods, Teacher Careers: Crises and Continuities (Philadelphia: Falmer Press, 1985): 230. 16. Stephen Segal, “Existential Conditions of Explicitness: An Heideggerian Perspective,” Studies in Continuing Education 21, no. 1 (1999): 73–90. 17. Jack Mezirow, “Postmodern Critique of Transformation Theory: A Response to Pietrykows- ki,” Adult Education Quarterly 49, no. 1 (1998): 65–67; Jack Mezirow, “Transformation Theory out of Context,” Adult Education Quarterly 48, no. 1 (1997): 60–62; Jack Mezirow, Transformative Dimensions of Adult Learning (San Francisco: Jossey-Bass, 1991). 18. Donald Schön, Beyond the Stable State (New York: Random House, Inc., 1971); Donald Schön, www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm www.ala.org/ala/acrl/acrlpubs/whitepapers Assessing Learning, Critical Reflection, and Quality Educational Outcomes 425 The Reflective Practitioner: How Professionals Think in Action (Averbury, U.K.: Ashgate Publishing Limited, 1991). 19. Teresa Thiel, “Reflections on Critical Incidents,” Prospect 14, no. 1 (Apr. 1999): 44–52. 20. Stacy Miller, “Students as Agents of Classroom Change: The Power of Cul- tivating Positive Expectations,” Journal of Adolescent & Adult Literacy 48, no. 7 (2005): 540–46. 21. Donna M. Weinreich, “Service- Learning at the Edge of Chaos,” Education- al Gerontology 29, no. 3 (2004): 181–95. 22. Panayiotis Angelides and Mel Ainscow, “Making Sense of the Role of Culture in School Improvement,” School Effectiveness and School Improvement 11, no. 2 (2000): 145–63. 23. Mary Jackson and Diane Wasson, “Critical Thinking Requires Critical Do- ing: An Analysis of Students’ Multicul- tural Experiences within Freire’s Frame- work,” paper presented at the American Educational Research Association, Chi- cago, Ill. (ERIC Document Reproduction Service, No. ED478164, Apr. 2003). 24. Lesley S.J. Farmer, “Narrative In- quiry as Assessment Tool: A Course Case Study,” Journal of Education for Library and Information Science 45, no. 4 (Fall 2004): 340–51. 25. Kevin J. Pugh and David A. Bergin, “The Effect of Schooling on Students’ Out-of-School Experience,” Educational Researcher 34, no. 9 (2004): 15–23. 26. Graham Walton and Sarah Ne le- ton, “Reflective and Critical Thinking in User Education Programmes,” British Journal of Academic Librarianship 7, no. 1 (1992): 31–43. 27. Donald Schön, The Reflective Prac- titioner. 28. Walton and Ne leton, “Reflective and Critical Thinking.” 29. Stephen Brookfield, “Critically Reflective Practice,” Journal of Continuing Education in the Health Professions 18 (1998): 197–205; Stephen Brookfield, “Experien- tial Pedagogy: Grounding Teaching in Students’ Learning,” Journal of Experiential Education 19, no. 2 (1996): 62–68. 30. “Information Literacy Competency Standards for Higher Education.” 31. Woods, Critical Events in Teaching and Learning; Tripp, Critical Incidents in Teaching; Brookfield, Becoming a Critically Reflective Teacher. 32. Brookfield, Becoming a Critically Reflective Teacher. 33. Tripp, Critical Incidents in Teaching, 39. 34. Norman K. Denzin and Yvonne S. VICTORIAN ­ FICTION ­ An Online Research Guide ­ Lionel Stevenson’s Victorian Fiction: A Guide to Research (1964) covered eleven novelists. George H. Ford’s Victorian Fiction: A Second Guide (1978) added six more. But the lesser Victorian novelists still remained unknown. Their exclusion, as Sheila M. Smith pointed out, “gives a distorted view of the literary activity of the nineteenth century.” Victorian Fiction: An Online Research Guide offers comprehensive coverage of fifty-one major novelists, such as Dickens, George Eliot, the Brontes, Elizabeth Gaskell, and Thomas Hardy; minor novelists, such as Edward Bulwer-Lytton, Charles Reade, Samuel Butler, and George Gissing; and underread novelists, such as George Egerton, Vernon Lee, Amy Levy, Eliza Lynn Linton, Ouida, and John Oliver Hobbes It includes editions of primary works and contemporary reviews and critical works from 1830 onward, and provides chronologies and information on biographies of novelists and their manuscript holdings Conduct sophisticated search of the entire 35,000+ entries on books, articles, and dissertation abstracts, each with linked & cross-referenced subject words and phrases on areas of interest Or narrow down your search to a limited number of recommended and annotated essential entries on each novelist or novel selected from hundreds of peer- reviewed books and articles * Coverage from 1830 to the present ­ *Information on primary works & ­ manuscript holdings * Chronologies and ­ editions *Annotated entries with ­ subject keywords ­ * Peer-reviewed books and articles identified ­ * Searching, printing & saving ­ of essential entries ­ * Fast, accurate & relevant ­ For more information, please check www.victorianfiction.com http:www.victorianfiction.com 426 College & Research Libraries September 2008 Lincoln, Handbook of Qualitative Research (Thousand Oaks, Calif.: Sage Publications, 1994). 35. Brookfield, The Skillful Teacher; Brookfield, Becoming a Critically Reflective Teacher. 36. Clara Hoover, “We Don’t Have to Learn Anything; We Just Have to Find the Answer,” School Library Media Activities Monthly XXII, no. 2 (2005): 26–28. 37. Schön, The Reflective Practitioner; Schön, Beyond the Stable State. 38. Jacqueline Kracker and Peiling Wang, “Research Anxiety and Students’ Perceptions of Research: An Experiment, Part II, Content Analysis of Their Writings on Two Experiences,” Journal of the American Society for Information Science and Technology 53, no. 4 (2002): 295- 307. 39. Denzin and Lincoln, Handbook of Qualitative Research. 40. Ibid. 41. Kathleen Dunn, “Assessing Information Literacy Skills in the California State University: A Progress Report,” Journal of Academic Librarianship 28, no. 3 (2002): 26–35. 42. Patricia Davi Maughan, “Assessing Information Literacy among Undergraduates: A Discussion of the Literature and the University of California-Berkeley Assessment Experience,” College & Research Libraries 26, no. 1 (2001): 71–85. 43. Denzin and Lincoln, Handbook of Qualitative Research. 44. Denzin and Lincoln, Handbook of Qualitative Research; Joe L. Kincheloe and Kathleen S. Berry, Rigour and Complexity in Educational Research: Conceptualizing the Bricolage (Berkshire, U.K.: Open University Press, 2004). 45. “Information Literacy Competency Standards for Higher Education.” 46. Segal, “Existential Conditions of Explicitness”; Tripp, Critical Incidents in Teaching; Brook- field, Becoming a Critically Reflective Teacher; Woods, Critical Events in Teaching. 47. Tripp, Critical Incidents in Teaching; Brookfield, Becoming a Critically Reflective Teacher; Woods, Critical Events in Teaching. 48. David A. Kolb, Experiential Learning: Experience as the Source of Learning and Development (Englewood Cliffs, N.J.: Prentice-Hall, 1984). 49. William E. Doll, Jr., A Post-Modern Perspective on Curriculum (New York: Teachers College, Columbia University, 1993). 50. Dunn, “Assessing Information Literacy Skills”; Maughan, “Assessing Information Literacy among Undergraduates.” 51. “Dihydrogen Monoxide Research Division,” available online from www.dhmo.org/facts. html. [Accessed 1 July 2008]; “The White House: Officious Website of George W. Bush,” avail- able online from h p://whitehouse.org/. [Accessed 1 July 2008]; “Martin Luther King, Jr. – A True Historical Examination,” Available online from www.martinlutherking.org/. [Accessed 1 July 2008]. 52. Tripp, Critical Incidents in Teaching. 53. Brookfield, Becoming a Critically Reflective Teacher. 54. Tripp, Critical Incidents in Teaching. 55. Segal, “Existential Conditions of Explicitness.” 56. Woods, Critical Events in Teaching and Learning. 57. Denzin and Lincoln, Handbook of Qualitative Research. http:www.martinlutherking.org http:h�p://whitehouse.org www.dhmo.org/facts