532 A Longitudinal Assessment of Graduate Student Research Behavior and the Impact of Attending a Library Literature Review Workshop Hannah Gascho Rempel Hannah Gascho Rempel is Graduate Student Services Coordinator in the Oregon State University Li- braries at Oregon State University; e-mail: Hannah.rempel@oregonstate.edu. © Hannah Gascho Rempel This article discusses findings from a longitudinal research study that examined the way graduate students carry out the literature review and how they were impacted by attending a library literature review workshop. The literature review research process serves as an important gateway for graduate students into their scholarly communities’ practices and can be a logical place for librarians to offer assistance. This study was carried out to gain insights into the ways librarians can better meet graduate students’ needs to shape the services offered to graduate students. Find- ings reveal a lack of a single path through the literature review research process, but they do indicate that attending a literature review workshop can have long-term benefits. hat role do librarians have in the development of graduate student research practices? Comprehending the research habits of graduate students is a necessary first step in providing graduate students with the support needed to become suc- cessful scholars. In particular, gaining an increased understanding of the way graduate students carry out research for their literature reviews can help librar- ians to address problematic areas that arise during this key piece of the research process. Librarians have recognized the importance of providing assistance to graduate students during the literature review process, as the literature review serves as a significant grounding element for students’ research. Gaining new in- sights into graduate students’ approaches to and frustrations with the literature review process will enable academic librarians to more strategically address graduate students’ needs and, as a result, will aid graduate students on the path to becoming scholarly researchers. This article discusses findings from a longitudinal research study that exam- ined the way graduate students carry out the literature review and how they were impacted by attending a library literature review workshop. In addition, I will pres- ent recommendations of how librarians can use this information on graduate student behavior to provide services to graduate students. crl-79 A Longitudinal Assessment of Graduate Student Research Behavior 533 Review of the Literature Becoming grounded within a disciplin- ary community is an important part of graduate students’ transition to becoming scholarly researchers. To facilitate this shift, graduate students must read exten- sively in their field, learn how scholarly discourse happens in their discipline, and learn the specialized vocabulary of their field. In a study by Barbara Lovitts, gradu- ate students who did not complete their degree showed signs of not identifying with their scholarly community or seeing the importance of following disciplinary conventions.1 These students did not see the value of citing other researchers’ work and, instead, wanted to study and write in ways that had worked for them as under- graduates. Richard McNabb discusses the importance of making appropriate schol- arly “gestures,” especially as a necessary step for being published. Making schol- arly gestures includes using disciplinary jargon, demonstrating an understanding of the literature and the field, and using the particular style of the discipline.2 All of these scholarly gestures are elements of conducting and writing a successful literature review. Without making these gestures, graduate students will not be able to make the transition to becoming scholarly researchers and writers. In addition to becoming grounded in their field, graduate students must make the transition to becoming independent researchers. By definition, graduate work, particularly at the doctoral level, requires “individual performance” and independent initiative taking.3 Part of this shift involves moving away from being a generalist, as most students were as un- dergraduates, to becoming specialists, as is required for graduate study.4 Another major difference from doing undergradu- ate work is the move from being a “course taker ” to creating one’s own research agenda.5 One way graduate students can make this leap is by working on a review of the literature. A traditional role of the literature re- view is to help students become rooted within their disciplinary communities, as well as to look for their own research niche within the community. The litera- ture review is typically defined as the pro- cess of writing a summary “that describes the past and current state of information … and documents a need for a proposed study.”6 However, students often lack an understanding of the richness of research and writing that is required for a literature review and, instead, compile a document that looks more similar to an annotated bibliography.7 Students who misunder- stand the purpose of the literature review will miss the opportunity to find a place in their discipline’s scholarly conversations. Graduate students may make the mis- take of underestimating the importance of the literature review because they lack systemic support. It has been widely re- ported that graduate students lack faculty guidance in general. In interviews with graduate students, Ann Austin found that students received little information on the formal or informal requirements of grad- uate school.8 Only half of the 1,500 gradu- ate students Melissa Anderson and Judith Swazey surveyed said that faculty were explicit in their expectations of graduate student performance and requirements.9 In discussing the relationships that faculty build with their students, Lovitts reports that faculty feel that the responsibility for establishing and maintaining a relation- ship with graduate students lies primarily with the graduate students themselves. Faculty will not go out of their way to establish this relationship.10 As a result, peer groups often stand in for faculty members in providing advice and in easing the transition into graduate school and the scholarly research process.11 If faculty advisors are frequently absent in the process of providing general orienta- tion information to graduate school, it is not surprising that faculty frequently do not provide much support for some of the more basic levels of carrying out scholarly research. Specifically, students seeking assis- tance for conducting the literature review 534 College & Research Libraries November 2010 may not find much faculty guidance. A major reason for this problem is likely because faculty do not value the literature review as highly as they do other parts of the thesis. David Boote and Penny Beile found faculty addressed the litera- ture review in a haphazard fashion and placed little emphasis on researching and writing the literature review.12 Laurene Zaporzhetz suggests that faculty down- play the literature review in part because of faculty’s own lack of ability to guide students through the process and because faculty expect that their students should already possess adequate research skills upon entering the graduate program.13 From anecdotal evidence from reference desks across the country, librarians know this is often not the case. Several authors who have written about providing services for graduate stu- dents agree the literature review is a key place to focus library efforts for graduate students. Librarians’ strategies for ad- dressing the literature review include holding workshops, creating rubrics, and integrating instruction focused on the literature review into credit courses.14 Because of the necessity of understanding previous research to make contributions in a field, Boote and Beile advocate for literature review instruction that is inte- grated at the programmatic level rather than just in a single workshop or a single class.15 However, to move beyond just pro- viding programming without more fully considering the user’s needs, Rachel Fleming-May and Lisa Yuro suggest that becoming familiar with graduate stu- dents’ research processes can help librar- ians know when to provide intervention or services.16 Fleming-May and Yuro used focus groups with social science gradu- ate students to reveal some insights into their research process. One of their main findings was that graduate students are caught between the undergraduate and faculty realms and are still looking for their voice within the research process. One way graduate students showed in their study that they had advanced beyond their undergraduate research processes was by their ability to narrow in on specific source types and databases. However, these students still struggled to determine the relevance of sources they consulted, but they were reluctant to ask for assistance, especially from librarians. Fleming-May and Yuro’s work also reveals that librarians are not always well informed about graduate-level research processes.17 In a survey of academic librar- ians in which librarians were asked about specifics of the doctoral process, many librarians lacked knowledge of the depth at which graduate research was carried out. The researchers hypothesized that this lack of information can make it dif- ficult for librarians to provide substantive services to this population. In addition, Boote and Beile noted that library instruc- tion often addresses the mechanics of searching and source types rather than delving into deeper research issues.18 This superficial method of instructing graduate students may result from a lack of understanding of what the graduate research process involves. Little previous research has been done to elucidate how graduate students approach the actual process of carry- ing out the literature review from the early searching phases to writing the literature review itself. However, there are examples that shed some light on how students envision the literature review research process. Christine Bruce analyzed graduate students’ responses to the question, “What is your understand- ing of the literature review?”19 Based on the responses, she developed six hierar- chical categories representing students’ conceptions of the literature review and concluded there is no standard under- standing of the literature review or its level of importance among students. Bruce also conducted a longitudinal study in which a range of graduate students from different disciplines wrote reflection pieces on the literature review process.20 She categorized most of their responses as A Longitudinal Assessment of Graduate Student Research Behavior 535 novice researchers in their approach to the literature and discovered that a primary difficulty for these students was trying to determine an appropriate scope for both their research projects and literature reviews. As with students’ understand- ing of the literature review, there was no shared approach for discovering an appropriate scope. In one of the most illuminating stud- ies, Rosemary Green and Peter Macauley interviewed four doctoral Education stu- dents who described several search habits and evolving work flow processes, such as using citation and reference tracking toward the end of their research as op- posed to more general keyword searching at the beginning.21 The authors describe the initial search process of these gradu- ate students as being more general in nature, such as using basic search tools, and moving on to more sophisticated and scholarly sources toward the end of their career, such as databases. These graduate students conducted searches in a variety of ways depending on what worked best for them at the time; the methods used included browsing, serendipity, and us- ing Internet resources. However, some of the results of this study reflect the specific student population: older adults who have been practitioners in the field and who are not as facile in negotiating electronic resources. Samuel Chu and Nancy Law provide more insight into the ability of graduate students to refine their searching behav- iors over time.22 Their study participants initially misunderstood some of the cata- log or database search options, such as the subject search, and, as a result, would use these functions incorrectly. Students improved their searching with training and became more sophisticated searchers who valued search techniques and used more savvy search methods. Another im- portant finding from this study was that, initially, graduate students did not know the authors and journals in their field and had to do more keyword searching. It took time for these students to become acquainted with the literature in their field, which had the compounding effect of influencing their search strategies. In the late 1970s, Cynthia Corkill et al. conducted longitudinal interviews with five humanities graduate students.23 The authors provide descriptions of these students’ reading and writing habits, as well as students’ frustrations with missing journal issues and the interlibrary loan process. While many of these concerns still exist today, the way library resources are accessed has changed drastically since this research was conducted. Most of the studies described focused on graduate students within particular disciplines. Several of the studies also dis- tinguish between master’s and doctoral students. The distinction between mas- ter’s and doctoral students is important in some contexts (for example, master’s stu- dents who are engaged in a professional master’s or a practice-oriented program, such as business or education students). Students in professional master ’s pro- grams rarely produce a thesis and, as a re- sult, are not required to conduct literature reviews. In fact, only 15 percent of current master’s programs follow a “traditional” curriculum; instead, most programs are practice-oriented degree programs.24 However, when master’s-level students are engaged in empirical research, their research process is often quite similar to that of a doctoral student. In the case where a doctoral student embarks on the degree without previous graduate study, the distinction between a master’s and a doctoral student conducting empirical re- search, especially during the first several years of study, is minimal. At Oregon State University (OSU) Libraries, we recognized a need to im- prove our services to graduate students and created a graduate student services program that included a literature review workshop to address graduate students’ library research needs on this often diffi- cult topic.25 To assess the effectiveness of the literature review workshop for gradu- ate students and to determine whether 536 College & Research Libraries November 2010 or not we were accurately addressing students’ needs surrounding the literature review process, a longitudinal study was conducted to gather this information. In addition, a goal of this research was to gain insights into the ways librarians can better meet graduate students’ needs to shape the services offered to graduate students. Methods This study used a longitudinal approach to explore how graduate students per- ceive the research process, who they look to for support in this process, what difficulties they have using the library while conducting research, and what impact attending the literature review workshop had on their long-term research processes. The longitudinal interviews were conducted over the course of two years to gain insights into how graduate students approach the literature review process over time, as well as to determine if students retained and used informa- tion they learned in a library workshop on the literature review throughout their graduate career. OSU is a Carnegie I land grant re- search institution with a graduate stu- dent population of approximately 3,100 students.26 Students were solicited for the longitudinal interviews during the fall 2007 offering of the literature review workshop. Only participants who were beginning their first year of graduate study at OSU were invited to volunteer for this research project. (The workshop is open to all graduate students regard- less of their length of tenure at OSU.) Eighteen students expressed their interest by filling out a form with their name and contact information. These students were contacted via e-mail, the research project was explained to them in more detail, and they were invited to schedule a time for an interview. Eleven students responded to this invitation; however, one of these students turned out not to be a gradu- ate student. Institutional Review Board approval was granted for this project, and participants filled out appropriate consent forms. Our literature review workshop does not attract students from professional master ’s programs because these pro- grams neither emphasize nor require a literature review. The students attending the literature review workshop are in- volved in empirical research projects; as a result, no effort was made to separate out study participants by whether they were pursuing a master’s or doctoral degree. Eight of the ten participants were master’s students; the two doctoral students had previously completed master’s degrees. All participants were studying in depart- ments that have doctoral programs and were working on research projects that involved original research. Eight of the participants were working on a thesis, and two were working on a project. Four of the participants came from the College of Science, four came from the College of Forestry, one came from the College of Health and Human Sciences, and one came from the College of Liberal Arts. Because OSU is the state’s land grant re- search university, a strong representation from the sciences and natural resources fields provided a fairly accurate reflec- tion of the graduate student body at this campus. Nine of the ten participants were women. This sample of volunteer participants does not precisely match the literature review workshop makeup, as less than 90 percent of workshop attendees are women (based on observational data). The college-level affiliation of the volun- teer participants does provide a relatively accurate portrayal of the workshop reg- istrants, with the exception of the lack of participants from the College of Engineer- ing, who register at the second highest rate for the workshop. Interviews were held at three intervals: at the beginning of winter term 2008, at the beginning of fall term 2008, and at the beginning of spring term 2009 (OSU is on the quarter system). Interviews were spaced approximately two terms apart so A Longitudinal Assessment of Graduate Student Research Behavior 537 participants would have new information to share and to reflect approximate begin- ning, middle, and end stages of graduate students’ work on the literature review. Two participants left the study after the first interview, and one participant left after the second interview, leaving seven students who participated in all three interviews. Interviews were conducted in a semi- structured manner using open-ended in- terview questions. The same ten questions were asked at each interview session. Two additional questions were added at the last interview. The interview questions asked students to reflect on work the stu- dent had done on their literature review in the previous term, what the student learned about doing library research in the previous term, what work the student had done as a result of attending the literature review workshop, from whom the student had learned new library research skills in the previous term, and any problems encountered in their library research in the previous quarter. Students were also given the opportunity to ask any questions they had. Demographic and logistical data were collected, such as departmental affiliation, start and fin- ish time of the student’s degree, whether or not they had previously received a graduate degree, and where they carried out most of their library research. At the last interview, students were also asked if they were familiar with the term “open access” and whether or not they planned on publishing their research. Notes for the interviews were hand- recorded and transcribed immediately after each interview. After all the inter- views were complete, analysis involved multiple readings of the transcripts to determine if there were similar trends across participant interviews, particularly in regard to questions about the literature review research process, the importance of the literature review workshop over time, and reliance on others for assistance. Based on the repeated trends observed, coding categories were created for the question related to the literature review process, as this question elicited the most student input. The remaining questions were analyzed for similarities and trends, but did not require coding due to the rela- tive brevity of the responses. The coding categories assigned to the literature review process responses were acquiring, organizing, reading, and writing. The acquiring category included statements related to searching and find- ing, and it encompassed using library resources, Web resources, and people networks. The organizing category in- cluded statements about using specific bibliographic management tools such as EndNote, creating folders, filing papers, and managing PDFs in an electronic envi- ronment. The reading category included statements about scanning articles, ref- erences to specific article sections read, and the process of rereading. The writ- ing category included statements about prewriting tasks, such as note taking, revising, writing documents related to the thesis literature review (such as proj- ect proposals), and creating annotated bibliographies. The validity of the four categories used in this study is shown in their close overlap with the five steps Creswell out- lines in his recommendations of how to conduct a literature review.27 His advice for researchers beginning the literature review process include the following: 1) Identify key terms to use in your search for literature; 2) Locate literature about a topic by consulting several types of mate- rials and databases; 3) Critically evaluate and select the literature for your review; 4) Organize the literature you have se- lected by abstracting or taking notes on the literature and developing a visual diagram of it; 5) Write a literature review that reports summaries of the literature for inclusion in your research report. His first three steps make up the acquiring category in this study. His fourth step introduces the idea of organizing and prewriting. The fifth step explicitly dis- cusses writing, a category chosen for this 538 College & Research Libraries November 2010 study. While Creswell does not explicitly mention reading as a step, it is hinted at in the evaluation step and is a key part of the process of conducting a literature review. Findings Literature Review Process Participants’ responses to the question “what work have you done on your lit- erature review this past quarter, if any,” generated by far the most discussion. The categories of acquiring, organizing, reading, and writing were used to group responses to this question. In general, students’ skills became more sophisti- cated over time, but, contrary to my ex- pectations, no overarching pattern of how students approached the literature review process, such as moving directly from acquiring to reading to writing, emerged. For example, only two participants men- tion all four categories of acquiring, organizing, reading, and writing during one interview period. The most frequently discussed tasks were from the acquiring category; five participants mentioned acquiring tasks every interview time. No participants mentioned reading tasks each interview time, and two participants mentioned organizing and writing every interview time. Contributing reasons to the lack of consistent patterns are likely due to specific departmental or advisor require- ments. In addition, individual person- alities often determined the path these participants took through the literature review process rather than a simplistic formula to addressing the literature re- view process. Following the guidelines and culturally ingrained work habits of their departments and advisors makes sense as graduate students are uniquely grounded within their departments and are learning the disciplinary habits of their scholarly community. For example, humanities scholars often work alone on their research, whereas life scientists are typically involved in a lab setting with several other researchers.28 A student’s individual personality and circumstances will also necessarily impact their style of work and their ability to address various aspects of their research project. However, a student who has repeated problems with time management, for example, should learn strategies for dealing with this issue rather than simply attributing it to individual personality. Acquiring While patterns did not emerge across the main categories, similar themes emerged within the four categories themselves. The quality of acquiring activities changed over time from broad topical searches to more specific searching. Whereas participant 1 stated in the first interview, “I have gotten lots of papers and books,” at the second interview this participant described going to specific databases for specific searches, using table of contents alerts, and citation chaining to find new articles. During the second interview, par- ticipant 3 described searching in Google Scholar and performing broad searches; by the third interview this participant instead talked about doing “clean up searching.” During the first interview, participant 7 broadly talked about find- ing articles, but at the third interview discussed using “targeted searching,” going to specific journals, and even look- ing in course syllabi for citations. In the first interview, participant 8 was getting background information and papers directly from an advisor; at the time of the third interview this participant had learned to search for specific papers and would only search on very specific topics. These results complement the research of Chu and Law, which found graduate students’ search behaviors became more sophisticated with practice.29 Over time these graduate students’ searching needs required them to search other disciplines; and, in several cases. by the last interview, search behavior was quite interdisciplinary. At the last interview, participant 7 stated, “I can read journals from outside the field with overlapping, complementary ideas.” This A Longitudinal Assessment of Graduate Student Research Behavior 539 type of searching shows a deep level of familiarity with one’s own field, such that the scholar knows where the boundaries of the field lie and knows how to navi- gate other disciplines to retrieve needed information. Organizing Organizing habits were divided between those who started on a specific organiza- tional schema immediately and those who started an organizational system only at the very end, if at all. Those who did have an organization system typically used a bibliographic management program, as well as a paper filing system. Those who started an organizing system at the end used a paper filing system, sometimes accompanied by a system for filing PDFs. A computerized bibliographic management option was too big a leap for the latter students at that point in the process. One student still did not have an organizational system by the time of the third interview and admitted to simply having piles of papers on her desk. The most sophisticated organizer tweaked her organizational system each time we talked, continually improving it. Reading Reading is one of the most fruitful ac- tivities students can undertake, as it both helps them learn how scholarly discourse in their field is communicated and pro- vides them with the research background they need to ground their own specific projects. Three out of ten participants mentioned reading in the first interview, four out of eight discussed reading in the second interview, and four out of seven referred to reading in the third interview. During the first interview, participants talked about reading very broadly. For example, participant 9 referred to “scan- ning journals in my field” and reading general topic areas. The reading activities described became more targeted over time; participants mentioned specific article sections they read or sought out as they became more accustomed to read- ing the scholarly literature in their field. During the second and third interview, participants talked about reading specific sections of papers, such as the Methods, Discussion, or Abstracts, to mine particu- lar information. Participant 3 discussed her learned practice of reading “judi- ciously,” beginning with the abstract to evaluate relevancy and focusing more on the Methods section on the last time she read through a paper. Participant 1 found herself rereading the same papers just for the information in the Methods section. Reading to find and compare citations emerged as students approached the end of their thesis work, a sign of trying to recognize whether the literature review work they had done was comprehensive enough. Participant 3 noted, “I find I’m familiar with citations in new papers that I read.” Participant 7 commented that she was “scanning more now, I know what I want to say, I just need the citation. I’m a more utilitarian reader.” The process of re- reading was described during interviews two and three, as participants found they could reread and “discover new things.” The last reading skill students seemed to acquire was reading specifically to learn about theories and reading with their own point of view in mind. Participant 3 referred to this as “critical reading” or “trying to determine how people inter- pret things.” Participant 7 found she was incorporating more theory and “picking up deeper ideas because I have my own position now.” These findings resonate with those documented by Green and Ma- cauley whose participants talked about the literature review as being a process made up of acquiring, rereading, and writing, thereby recognizing the recursive nature of the literature review process.30 Writing Participants who started writing early in the process were driven either by the need to write project proposals or funding applications. These early versions of the literature review could be repurposed in their final thesis, but students recognized 540 College & Research Libraries November 2010 they would need to supplement or refine and update these literature review drafts for their final product. Other participants were hesitant to begin the writing process and expressed anxiety even at the third interview, showing they still lacked an understanding of writing in this schol- arly genre. For example, participant 2 stated, “I don’t know what a literature review should look like,” and participant 6 admitted to simply being “scared” to start writing the paper. In contrast to these students, the most savvy students interviewed noted that the literature re- view was an unfinished document that would require continual revision. Other participants talked about using a system that could help them move from reading to writing (for example, effective note taking). The latter students had devel- oped techniques that would help them move past issues of writer’s block or fear to make progress on their final written product. Problems Working on the Literature Review Another point of discussion that came up during these interviews was the fact that, at various points in their graduate school career, many of these students were not working on the literature review process. Four participants had at least one interview time when they said they were not working on the literature review. The most common reason for this was when students had not yet established a research topic. Not all graduate students begin their graduate career with a clear research focus and may flounder for the first several months trying to find a topic. Three of the participants in this study did not have topics until the second interview. Another reason these graduate stu- dents sometimes experienced difficulty moving forward on their literature review was that they are often dependent on ex- ternal forces to move ahead with phases of the literature review. For instance, students may need to wait for committee approval to move forward. A few partici- pants also expressed a lack of identifica- tion with their scholarly community, not valuing the literature in their field, how findings were expressed, or simply hav- ing difficulty reading scholarly literature. All of these feelings contribute to a lack of motivation to move ahead with the literature review. Finally, several students had problems understanding an appropriate scope for their project and how to view their topic in another light, through a different dis- ciplinary focus, or in a broader context. These students stated, “There isn’t litera- ture on exactly what I’m doing,” or “My research questions aren’t addressed in the literature,” or “My topic doesn’t have literature on my specific geographical area.” Research from Bruce on graduate students’ understandings of the scope of their literature review revealed similar findings.31 Students struggled to define appropriate boundaries for the coverage of their topic and most readily regarded the issue of scope as simply collecting works that were about the topic of their research. In Bruce’s findings, students who were fixated on the topical nature of scope did not take into account general background information or associated topics as possible avenues for exploring their research project. Bruce suggested that a more nuanced way to conceptualize the scope of a literature review is to con- sider relevance on a larger scale, which allows for exploration beyond simply “my topic.” Long-Term Effects of the Literature Review Workshop Participants were asked to describe how attending the literature review workshop affected the way they approached car- rying out their own literature reviews. The most profitable responses to this question came during the first interview period, when the literature review work- shop was still fresh in the participants’ memories. Participants admitted to not knowing about many research tools be- fore the workshop. Participant 6 stated A Longitudinal Assessment of Graduate Student Research Behavior 541 she “didn’t know anything before.” Some participants had never heard of Google Scholar; most were unfamiliar with how to navigate through the library’s Web site and were unsure of how to choose ap- propriate databases. Participants felt they learned how to become more proficient searchers and could be more “effective and efficient.” Participants also men- tioned becoming familiar with a variety of bibliographic management software options and felt more able to choose a tool that worked for them. Another area that was helpful to these participants was hearing about the formal steps of the lit- erature review process and thinking more about issues such as comprehensiveness and how to know when the literature re- view is complete. Finally, these students referred to learning about a variety of Web 2.0 tools such as RSS feeds and Delicious, and several still mentioned using these tools at the last interview. Who Teaches Library Skills? In response to the question, “Who have you learned new library skills (e.g. data- base searching, using EndNote, organiz- ing your searches) from this quarter,” it was found that these graduate students primarily rely on themselves to learn new skills. Advisors would occasionally provide them with research articles, but this did not help students learn to find articles on their own or to evaluate which articles were the most relevant for their field. Some participants also mentioned learning specific tips from friends in their department. Student peers with more experience were seen as desirable sources for information. Surprisingly, peers as a primary conduit for information was not mentioned as often as expected, based on other research that indicates that students typically learn as much from their peers as from faculty.32 Some participants mentioned the com- plexity of their topic or the specificity of their research as reasons they would not consult with a librarian on their research, hinting at the feeling that librarians did not understand their field and were not equipped to assist them in the way that someone from within their own field could. However, several students did mention specific subject librarians they had met with and referred to the lit- erature review workshops as a resource for them. Other researchers have also found that PhD students are hesitant to ask librarians questions about special- ized research, feeling that the librarian’s insights may not be relevant.33 Overall, the students’ expressions of need for assistance diminished over time as they became more confident in their role as independent researchers. Library Research Process Participants were asked, “What have you learned about doing library research this quarter?” Initially, the interviewees mentioned learning about the physical location of items within the library; how to use the libraries’ Web site and data- bases; learning about particular library resources that involved learning library language, such as ILL and “Get this Item” (a link resolver); and learning they did not need to pay for articles they could not immediately access through the library. At later interviews, when these students had become more experienced research- ers, they had settled into a routine of using a particular set of databases and mentioned these databases by name. Participants had improved their search- ing techniques and could keep track of searches, use citation or author chain- ing, and use keywords more effectively. Practice and repetition had made them more comfortable with the whole library research process. These students did not incorporate their concept of research from the earlier question addressing the literature review process, which in- cluded discussion about discovery and evaluation, with their ideas of “library research.” Library research for them was much more tied to a particular building, a set of resources, and the procedures involved in using these resources. 542 College & Research Libraries November 2010 publishing their results. Those who were planning on publishing were at a wide variety of points in this process: several had established some rudimen- tary publishing ground rules with their coauthors, such as work distribution; others had already determined which journals to target; but another student had not even discussed publishing op- tions with an advisor yet. Participants who were not planning on publishing in the scholarly literature had different reasons for not choosing to pursue pub- lishing. Participant 2 felt her research would be best put to use in other ways; she didn’t want to create something that would “just sit on a shelf for 20 years.” Participant 3 felt her data would not stand up to public scrutiny and did not have larger significance for the scientific community. Participant 6 felt she could not write something that would be ap- propriate for her scholarly community and had no desire to contribute to the scholarly academic field. Becoming part of the scholarly com- munity involves not just submitting articles to be published but also under- standing scholarly publishing models. To find out whether information about open access publishing was filtering down to these graduate students, students were also asked at the third interview if they had heard of open access publishing. Five out of seven participants in this interview had not heard of open access publishing. One of the students who had heard of open access publishing was quite fluent on the topic and even knew that she had an open access journal article in her bag. She had heard about open access both from reading papers from open access journals and from a conversation about data and who owned it with a friend in computer science in which they were “using all the same nouns, but found they had different meanings.” Based on this small sam- pling, it appears there is still a long way to go in educating our graduate student community about open access. Problems with Library Research Participants were asked to “describe any problems they encountered with their library research since the previous inter- view.” These graduate students’ prob- lems with conducting library research fell into two categories: frustrations about library resources and frustrations about the process of using the library. Partici- pants were irritated when the library did not have particular items they wanted, although this became less of a concern as they learned about consortial bor- rowing and interlibrary loan. Students also mentioned a wide array of library processes that felt illogical to them, from reshelving schedules to interlibrary loan rules, to using the libraries’ Web interface and link resolvers. Students had a high expectation that they should be able to retrieve whatever they needed from our library, preferably in full text, in a very quick turnaround time. While these types of problems may feel mundane to those of us who work in libraries every day, the inability to quickly access information can feel like a major roadblock for these students. These expec- tations match up with other recent studies on graduate student information-seeking behavior, but greatly contrast a similar longitudinal interview study performed in the late 1970s.34 At the time of the latter study, students were willing to purchase books themselves, travel to other (Euro- pean) countries to study at other libraries, and wait for lengthy interlibrary loans. Students’ current expectations place a high burden on libraries to be nearly flawless in the speed and availability of information provided. Publishing and Open Access To establish how well these students were being incorporated into their scholarly communities by the end of their graduate school career, students were asked during the third interview whether or not they were planning on publishing their results. Four out of seven participants were planning on A Longitudinal Assessment of Graduate Student Research Behavior 543 Conclusions Recommendations The goals of this study were to further un- derstanding about how graduate students approach the literature review, what research processes they establish while working on the literature review, and who they look to for assistance in this process so that librarians can have a better under- standing of graduate students’ needs and can establish logical points of intervention throughout the literature review research process. Based on this study, several rec- ommendations emerged. Tailor Services to Your Population Services must be tailored to meet your population’s needs. In the case of small institutions, this may mean more one- on-one services for graduate students; at larger institutions or in larger de- partments, this could mean targeted workshops that consider the particular stage of research in which graduate students are engaged. For example, students in their first year of study will have different needs from students in their third year of study. Knowing the responsibilities and typical timelines that graduate students follow can help librar- ians appropriately target services to meet those students’ needs. Fleming-May and Yuro showed a lack of understanding of what graduate work entails on the part of many academic librarians.35 To combat this, librarians must become more edu- cated about the graduate student process and what writing and researching for a thesis involves. In this study, workshops have been shown to be a worthwhile means of reach- ing out to graduate students, particularly in regard to focused topics, such as the literature review, that are highly relevant to students. Discovering the appropri- ate “zone of intervention” or “point of need” will make librarian interactions with graduate students more valuable regardless of whether those interactions take place one-on-one or in a workshop or classroom setting.36 Target New Graduate Students During their first year of study, graduate students are in the “beginning” or “antici- patory” stage in which building relation- ships is key.37 While other considerations about graduate students’ research needs must be considered (and are discussed below), the first year is an important time to establish contact with these students and to be available during that crucial relationship building time. This study’s findings indicate that the literature review workshop is most effec- tive early in a graduate student’s tenure, or at least when he or she is new to a particular library. Other research backs up the importance of providing graduate students with an in-depth orientation to the library early on.38 Although students who receive library instruction later in their graduate school career were not part of this study, based on self-assessment feedback from the literature review work- shops, attendees as a whole report their knowledge of new services and search strategies increased as a result of attend- ing the workshop. However, graduate students who are not new to the institu- tion have likely already established their research patterns and may be less inclined to alter them at that point. However, there are caveats to this early targeting strategy, which are explained below. Beware of Disciplinary and Programmatic Differences The findings from this study reveal that graduate students’ progress through the literature review process is largely driven by specific disciplinary forces. Previous research has also indicated that graduate students’ acculturation and approach to scholarly research is discipline depen- dent.39 Because the timing of a student’s progress through his or her degree is closely tied to disciplinary and program- matic requirements, strategizing about appropriate points of need at which to offer library assistance involves under- standing what some of these disciplinary and programmatic differences can be. 544 College & Research Libraries November 2010 One programmatic difference depends on whether the student is working to- ward a nonthesis (or applied) project or a thesis. One place to start a research discussion with a graduate student is to find out what type of final product he or she is working toward. Knowing the final product of the student’s research can help librarians understand the gradu- ate student’s expectations and his or her timeline. While the gap between these two outcomes varies by discipline and within specific programs, a thesis re- quires original research to be performed, while a project is less likely to have this requirement and is generally completed more quickly. In my study, by the second interview there was a clearer separation between students working on a thesis and those working on a nonthesis option. The students working on a thesis were more invested in their project and were more rigorously following sources and gener- ally seemed more motivated to delve into their research questions. Graduate students may come to the library with different initial exploration expectations. If a student arrives with a topic already preselected based on work his or her advisor ’s research group is already working on, he or she will jump directly into more targeted searching. If the student needs to choose his or her own topic, he or she will need to spend a significant amount of time researching various avenues during his or her initial phase of graduate school. In this study, the participants who needed to find a topic on their own began working on the literature review almost a year after beginning graduate school. Receiving information about the literature review process at this later point when they are focused on this aspect of their research might be more useful than at the very beginning of their graduate school career. Not all graduate programs require a project proposal, but students in depart- ments that do require a proposal will need to do more systematic work on fleshing out their topic and delving into the lit- erature from the outset. If a proposal is not required, students may delay starting some of this work until after they have collected data. Again, this has implica- tions for the timing of providing students with information about researching the literature review. Students working on a proposal will need targeted assistance early on, whereas students who do not need to write a formal literature review until closer to the end of their tenure may not remember information that was presented to them at such an early date. Finally, it is important to remember that graduate students are human too. Their energy level for their project may vary widely over the course of their stud- ies. The energy level of the participants in this study seemed to follow a bell-shaped curve. During the first interview, students talked briefly about their research proj- ects; however, by the second interview, students talked at length about their research: describing methodologies, their interest in the topic, and difficulties they were having. By the third interview, stu- dents were less interested in discussing their research projects and were more interested in talking about the logistics of finishing up their projects. Especially when engaging with students in one-on- one consultations, it may be helpful to be respectful of the current level of passion the student has for his or her research project and to be sympathetic if he or she is at a low point in the bell curve. Assist Graduate Students’ Transitions to Becoming Scholars While providing access to information and increasing students’ skills in devel- oping sophisticated search strategies is important, librarians should be in the business of more than just teaching the mechanics of databases and catalogs. Librarians can be involved in stretching graduate students’ thinking to include scholarly communications issues, such as authors’ rights or open access. Librarians can also become more involved in helping students with the issue of scope. Findings A Longitudinal Assessment of Graduate Student Research Behavior 545 in this and other studies have identified this part of the literature review process as a difficult point for many students.40 Students need to learn how to move from a belief that there is one right set of litera- ture to realizing that finding the relevant literature is dependent on their research question and focus. As students develop a relationship with their question and the papers they read, they will be more able to discern what is relevant. As part of this strategy of addressing scope, students must move from trying to acquire and read everything to determining what is most significant. Finally, librarians should work closely with other on-campus student support services to provide holistic support. As part of the university community, we too need to be concerned about the high rate of attrition among graduate students and consider how we can help these stu- dents successfully matriculate.41 When partnerships are created with specific departments or with the writing center, librarians can truly help in the process of shaping new scholars throughout the entire research process from acquiring, to evaluating, to producing knowledge. Summary and Future Directions This study found that a library workshop on the literature review could continue to impact students after the workshop was completed. The literature review work- shop was a valuable resource for starting graduate students on the right path to working on their literature reviews, even though students’ specific approaches to the literature review were observed to be different based on disciplinary and personal characteristics. It should be noted that the students who volunteered for this study may have been more in- clined to enjoy the workshop and the library. Being part of the study also likely made these students more self-reflective about the literature review process than students who were not involved in the study. However, given the necessary limitations of the study, and the fact that the literature review workshop was only one session, it seems that these students clearly benefited from attending. Green and Macauley describe several important components of graduate-level information literacy.42 These components include understanding the literature review process, becoming familiar with graduate student process-specific knowl- edge, and consistently increasing disci- pline-specific knowledge. In addition, Green and Macauley assert that graduate students should make original contribu- tions to professional knowledge and prac- tice and engage in scholarly communica- tion practices. As a group, the participants in this study showed significant growth throughout their graduate student ca- reers, particularly in terms of understand- ing the literature review process, graduate student process-specific knowledge, and discipline-specific knowledge. While this study was not directly measuring these students’ process-specific knowledge and discipline-specific knowledge, these ele- ments were indirectly observed through informal discussions about their comfort level with graduate school and their re- search topics, as well as my observations of their increased engagement with their own projects as the study progressed. Interview questions that examined students’ progress with making original contributions to professional knowledge and their engagement with scholarly com- munications practices showed a mixed level of engagement at this point in their scholarly careers. There was still much room for improvement. However, increased opportunities to participate in their scholarly communities through publishing, presenting, and attending conferences would likely aid them in reaching a higher level of competence in these areas. Several areas of future research emerged from this study. A follow-up study in which graduate students and their advisors are interviewed could determine not only how the student perceives his or her research process 546 College & Research Libraries November 2010 but how faculty perceive their student’s progress through the research process. The addition of faculty insights could help librarians integrate support that matches students’ and faculty’s research expectations. In addition, as librarians become more aware of the research is- sues graduate students face, a follow-up survey would be helpful to assess how librarians’ services for graduate students have changed as a result. Finally, it will be important to measure the suc- cess of library programs that include an increased emphasis on scholarly communications as part of the research process as a whole. Notes 1. Barbara E. Lovitts, Leaving the Ivory Tower: The Causes and Consequences of Departure from Doctoral Study (Lanham, Md.: Rowman & Littlefield Publishers, 2001), 113. 2. Richard McNabb, “Making the Gesture: Graduate Student Submissions and the Expecta- tion of Journal Referees,” Composition Studies/Freshman English News 29, no. 1 (2001): 9–26. 3. Susan K. Gardner, “‘What’s Too Much and What’s Too Little?’: The Process of Becoming an Independent Researcher in Doctoral Education,” The Journal of Higher Education 79, no. 3 (2008): 326–50. 4. Rachel Fleming-May and Lisa Yuro, “From Student to Scholar: The Academic Library and Social Sciences PhD Students’ Transformation,” portal: Libraries and the Academy 9, no. 2 (2009): 199–221. 5. Lovitts, Leaving the Ivory Tower, 73. 6. John Creswell, Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 3rd ed. (Upper Saddle River, N.J.: Pearson/Merrill Prentice Hall, 2008), 89. 7. Chris Hart, Doing a Literature Review: Releasing the Social Science Research Imagination (Lon- don: Sage Publications, 1998), 1. 8. Ann E. Austin, “Preparing the Next Generation of Faculty: Graduate School as Socializa- tion to the Academic Career,” Journal of Higher Education (2002): 94–122. 9. Melissa S. Anderson and Judith P. Swazey, “Reflections on the Graduate Student Experi- ence: An Overview,” New Directions for Higher Education, no. 101 (1998). 10. Lovitts, Leaving the Ivory Tower, 118. 11. Gardner, “What’s Too Much and What’s Too Little?” 12. David N. Boote and Penny Beile, “Scholars before Researchers: On the Centrality of the Dissertation Literature Review in Research Preparation,” Educational Researcher 34, no. 6 (2005): 3–15. 13. Laurene E. Zaporozhetz, “The Dissertation Literature Review: How Faculty Advisors Prepare Their Doctoral Candidates” (PhD dissertation, University of Oregon, 1987). 14. Rosemary Green and Mary Bowser, “Managing Thesis Anxiety: A Faculty-Librarian Partnership to Guide Off-Campus Graduate Education Students through the Thesis Process,” Journal of Library Administration 37, no. 3/4 (2002): 341–54; Rosemary Green and Mary Bowser, “Observations from the Field: Sharing a Literature Review Rubric,” Journal of Library Administra- tion 45, no. 1/2 (2006): 185–202; Hannah Rempel and Jeanne Davidson, “Providing Information Literacy Instruction to Graduate Students through Literature Review Workshops,” Issues in Science and Technology Librarianship 53 (2008), available online at www.istl.org/08-winter/refereed2.html [Accessed 20 September 2010]. 15. Boote and Beile, “Scholars before Researchers.” 16. Fleming-May and Yuro, “From Student to Scholar.” 17. Ibid. 18. Boote and Beile, “Scholars before Researchers.” 19. Christine S. Bruce, “Research Students’ Early Experiences of the Dissertation Literature Review,” Studies in Higher Education 19, No. 2 (June 1994): 217–29. 20. Christine S. Bruce, “Interpreting the Scope of Their Literature Reviews: Significant Differ- ences in Research Students’ Concerns,” New Library World 102 (2001): 158–65. 21. Rosemary Green and Peter Macauley, “Doctoral Students’ Engagement with Information: An American-Australian Perspective,” portal: Libraries and the Academy 7, no. 3 (2007): 317–32. 22. Samuel Kai-Wah Chu and Nancy Law, “Development of Information Search Expertise: Postgraduates’ Knowledge of Searching Skills,” portal: Libraries and the Academy 7, no. 3 (2007): 295–316. 23. Cynthia Corkill, Margaret Mann, and Sue Stone, Doctoral Students in Humanities: A Small- A Longitudinal Assessment of Graduate Student Research Behavior 547 Scale Panel Study of Information Needs and Uses 1976–79, vol. 5, CRUS Occasional Paper 5637 (Shef- field [Yorkshire]: Centre for Research on User Studies University of Sheffield Western Bank, 1981). 24. Julie A. Jackson, “An Examination of Master’s Level Graduate Student Experiences and Attitudes” (Bowling Green, Ohio: Bowling Green State University, 2007), 23. Available online at www.ohiolink.edu/etd/send-pdf.cgi/Jackson%20Julie%20A.pdf?acc_num=bgsu1174940455. [Accessed 20 September 2010]. 25. Rempel and Davidson, “Providing Information Literacy Instruction to Graduate Students.” 26. Oregon University System, “OUS 2008 Fact Book,” 2008. Available online at www.ous. edu/dept/ir/reports/fb2008/index.php#s. [Accessed 20 September 2010]. 27. Creswell, Educational Research. 28. Susan Gardner, ““I Heard It through the Grapevine”: Doctoral Student Socialization in Chemistry and History,” Higher Education 54, no. 5 (Oct. 18, 2007): 723–40. 29. Chu and Law, “Development of Information Search Expertise.” 30. Green and Macauley, “Doctoral Students’ Engagement with Information.” 31. Bruce, “Interpreting the Scope of their Literature Reviews.” 32. Anderson and Swazey, “Reflections on the Graduate Student Experience”; Green and Macauley, “Doctoral Students’ Engagement with Information.” 33. Fleming-May and Yuro, “From Student to Scholar”; Peter Macauley and Rosemary Green, “Can Our Relationships Be Reconceptualized? Librarians, Information Literacy, and Doctoral Learners,” Journal of Education for Library and Information Science 50, no. 2 (2009): 68–78. 34. Carole George et al., “Scholarly Use of Information: Graduate Students’ Information Seek- ing Behavior,” Information Research 11, no. 4 (2006), available online at http://informationr.net/ ir/11-4/paper272.html [Accessed 20 September 2010]; Corkill, Mann, and Stone, Doctoral Students in Humanities, vol. 5. 35. Fleming-May and Yuro, “From Student to Scholar.” 36. Fleming-May and Yuro, “From Student to Scholar”; Rempel and Davidson, “Providing Information Literacy Instruction to Graduate Students.” 37. Vicki B. Sweitzer, “Towards a Theory of Doctoral Student Professional Identity Develop- ment: A Developmental Networks Approach,” Journal of Higher Education 80, no. 1 (2009): 1–33. 38. Evelyn C. Minick, “Millenials in Graduate School: How Do We Support Them?” College and Research Libraries News 70, no. 8 (2009): 449. 39. Gardner, “I Heard It through the Grapevine”; Minick, “Millenials in Graduate School: How Do We Support Them?” 40. Bruce, “Interpreting the Scope of Their Literature Reviews.” 41. Council of Graduate Schools, “Council of Graduate Schools > CGS Research > Graduate Enrollment and Degrees Report ( DNN 3.2.2 ),” June 28, 2009. Available online at www.cgsnet. org/Default.aspx?tabid=168. [Accessed 20 September 2010]. 42. Green and Macauley, “Doctoral Students’ Engagement with Information.”