454 Do They “Get It”? Student Usage of SFX Citation Linking Software Bonnie Imler and Michelle Eichelberger Bonnie Imler is Library Director at Penn State Altoona; e-mail: bbi1@psu.edu. Michelle Eichelberger is Systems and Electronic Services Librarian at Genesee Community College; e-mail: maeichelberger@genesee. edu. © Bonnie Imler and Michelle Eichelberger This study focuses on a quantitative assessment of undergraduate student use of SFX citation linking software. The study seeks to reveal whether or not students are using SFX, and, if so, how they are using it. Study participants were Penn State Altoona students, all of whom had received basic library instruction. Their research sessions were captured with screen capture software and then coded with Studiocode video coding technology. While not the prime focus of the study, the investigators also considered the effect of factors such as database Web design, images, and terminology on student usage of SFX. n 2004, Penn State University began using SFX, the Ex Li- bris citation linking software, which appears to Penn State library users in databases and other col- lections as an icon marked Get It! Penn State. (See figure 1.) Due to its ability to connect to full-text documents across database platforms, SFX soon became an integral part of the library search environ- ment and a core subject taught during bibliographic instruction classes. Despite promotion and instructional efforts, anec- dotal reference desk data suggested that students were not using the SFX features and frequently passed up relevant articles available through the Get It! button for less relevant articles that were readily available in full text within the same da- tabase. The investigators theorized, based on the reference desk interactions, that these students had either not received instruction on how to use Get It! or were not retaining the SFX instruction that they had received. The assumption was made that students were not interested in taking the additional time needed to find and retrieve the best articles available in the library’s databases. The investigators chose to concen- trate their research on student usage of SFX in the ProQuest Research Library. ProQuest is the only general aggrega- tion of databases listed on the Penn State Libraries’ “Try These First” Web page and is a popular research starting point for undergraduates at Penn State’s Altoona campus, where the investigators were based. ProQuest is often taught during Figure 1 Penn State’s SFX icon crl-141 Do They “Get It”? Student Usage of SFX Citation Linking Software 455 bibliographic instruction, regardless of the course subject matter. While the initial research focus was to determine if students would use SFX to locate relevant articles, the investigators were also curious about how far into the Get It! process a student would go to re- trieve the full document. In any database, to fully succeed in obtaining the full text of a document through the Get It! Button, it is necessary to click on two or three links. (See figure 2.) If the full text is not available, links to the library catalog and the interlibrary loan form are provided. To avoid interference in the natural student searching mentality, the investi- gators wanted to find a noninvasive way to capture the screen movements of the research subjects and then statistically analyze that data. In 2007, Penn State obtained a universitywide license for the video coding software Studiocode, and the university community was encour- aged to explore educational applications for this software. Studiocode allows the user to code and analyze video by cre- ating keyword associated buttons and marking sections of the video where the activities being studied occur. While normally used with live action video, the lead investigator correctly determined that the software would also be effective with screen capture video. Literature Review SFX, produced by Ex Libris, “is a con- text-sensitive linking system intended to integrate the databases and other on- line services and resources that a library has to offer, thereby increasing their effectiveness. It is a product based on the OpenURL standard and is produced by exLibris, a library system vendor.”1 SFX has been used in libraries since 2001. In a 2006 study, Livingston found it to be the best product for accuracy, scalability, usage stats, and manage- ment reports, compared to Endeavor ’s LinkFinderPlus and Serials Solutions’ Article Linker.2 Figure 2 SFX results Screen 456 College & Research Libraries September 2011 Libraries spend millions of dollars on database linking software such as SFX, with little data available to prove how ef- fectively it’s being used. Hider noted that early implementations of the technology in the years following its launch in 2001 were seen in a positive light,3 but findings were often more anecdotal than based on hard data.4 He also noted that more work needed to be done to more deeply analyze how the technology was being used by library patrons.5 In 2006, Wakimoto et al. posited that for most end users of SFX, their expectations of the service were higher than their real experience, but librarians reported that it worked most of the time, based on anecdotal evidence.6 For those authors who have intensively studied general information seeking behavior in the library’s databases and catalog, the research methodology has included surveys, interviews, and activity log tracking. In Gong and Loomis’s “An Empirical Study on Follow-up Library In- struction Sessions in the Classroom,”7 the researchers administered a paper survey of students before they received instruc- tion, as well as after instruction, including special follow-up library instruction held in their classroom. Antell and Huang used OPAC transaction logs and user observa- tion interviews to study how students were performing subject searching in the library catalog.8 Researchers Wakimoto et al. and Cummings and Johnson used online surveys, focus group interviews, usage statistics analysis, participant observation, and sample end testing to examine the effectiveness of SFX.9 How- ever, no one has used screen capture technology to track what the students are actually doing in the library’s OPAC and databases, screen by screen. Screen capture technology such as Captivate and Camtasia, and the Mac version, Snapz Pro, is being used in the library field, but mainly for creating online tutorials. Blevins and Elton provide an overview and evaluation of Camtasia’s use as a tutorial-creating software in “An Evaluation of Three Tutorial-creating Soft- ware Programs: Camtasia, Powerpoint, and MediaSite.”10 Other researchers have explored the ways that screen capture technology can enhance Web instruc- tion pages and library tutorials.11 Screen capture technology has been used in Web usability tests that were created to facili- tate the redesign of a library’s Web pages, but it has not been used to track students’ day-to-day use of the library’s databases.12 Hider proposed using screen capture technology to track online information seeking behavior in the library, including usage of SFX; but, like Hargittai, his focus was on the coding scheme that would be used for this research, not the research itself.13 Although it does not capture audio or physical reactions, Hider noted that screen-recording software like Camtasia is a good means to capture information about user-system integration, and to record online information seeking.14 How- ever, the captured data needs to be coded consistently to make it matter.15 Studiocode is a software package that is starting to be used for research in the academic world. It can be used to capture video clips, to code and analyze video footage, create presentations, and to cre- ate data transcripts. So far, it has been used by sports teams to record and mark practices and plays, to help students in deaf interpretation studies improve their presentation and sign language skills, in medical research and simulation, and for many other educational purposes.16 To this date, it has not been used in conjunc- tion with screen capture technology like Camtasia or Snapz Pro, and it has not been used in library research. Methodology To decrease variation in the participants’ exposure to SFX, the investigators chose to limit their participants to students who had been taught to use SFX in a classroom setting. Per their instructor’s request, stu- dents received a standard bibliographic instruction class covering the online catalog and the ProQuest Research Library in the library computer lab. During this hands-on Do They “Get It”? Student Usage of SFX Citation Linking Software 457 session, the SFX link was demonstrated twice, emphasizing ease of use (two or three mouse clicks) and the importance of seeing the process through to article pos- session. The lead investigator taught all of the sessions, which ensured that consistent teaching took place. During the initial class, no mention was made of the research project or that the students would need to recall any of the information at a later date. At least two weeks after the session, the librarian returned to meet with the class and invited students to participate in the research study. Participation was voluntary and in no way influenced stu- dents’ course grades. All 40 participants were undergraduate students over the age of 17, with the majority matriculated in at least one 100 level course. Volunteers were given an appointment card with a time/date to arrive individually at the librarian’s office. Materials An iMac 8.1 computer loaded with Snapz- Pro software, Studiocode software, and the Safari Internet browser was used for the student participation and data analy- sis in this study. At this time, Studiocode is a Mac-only software program. It accepts Quicktime (.mov) file types; therefore, SnapzPro was chosen as the screen cap- ture software because it saves the video directly as a Quicktime file, preventing distortion that might occur by conversion from another file type. The investigators recognized that the Macintosh interface and keyboard could have been unfamiliar to students and were prepared to offer technological instruction if needed. This proved to be unnecessary, and the screen capture videos verified that students had little difficulty navigating the screens needed to complete the assignment. The investigators created an assignment sheet to ensure that every participant would receive the same directions for the research assignment. Each student was given the as- signment sheet at the start of the research session and was encouraged to ask ques- tions if they needed any clarification. Procedure The investigators created a sample research assignment using Penn State University Libraries’ ProQuest Research Library. The search was designed to pro- duce a manageable number of results: not so many that a student would be over- whelmed, but not so few that a student would be led to select certain titles by de- fault. After experimentation, the investi- gators determined that the Boolean search string video games AND boys, limited to scholarly journal results, produced the necessary variety and number of results needed for the study. The ProQuest search resulted in 65 citations listed in reverse chronological order. Each results screen contained 10 citations and provided navi- gational arrows at the bottom of the list to see additional result pages. The lead investigator entered the search terms prior to each student visit to keep the search consistent and to avoid variation and possible spelling errors. The investigators chose to limit the search to scholarly articles because they were less likely to be available in full text within the database and would encourage the participant to use the SFX feature. Students were asked to select and print the full text of the five most relevant articles from the results list that would most help them write a research paper on video games and violence in boys. The investigators purposely stressed finding articles that would “most help” in writing a research paper, in contrast to requesting that students select “any” five articles. If they had requested “any” five articles on the topic, the student participant could have easily chosen the first five articles or picked only articles available in full text within ProQuest and would not have necessarily needed to use the SFX function. Additionally, students were also asked to print the articles they selected in an attempt to ensure that they would complete the process and locate the full text of the article. While students were asked to print the full-text articles they selected, the inves- tigators were concerned that the printing 458 College & Research Libraries September 2011 process could result in an increase in technology-related delays and frustra- tions for the participants. In addition, the investigators did not wish to waste paper on articles that ultimately would not be read. To give the illusion of printing, the computer was set to send the articles to a nonexistent printer and students were instructed to continue their search when they witnessed the printer icon bounc- ing at the bottom of the screen. Since the participants would not receive physical printouts to help them track the number of articles they had printed, a blank box was placed on the assignment sheet and students were encouraged to add a tick mark to the box for each article sent to the printer. Data Collection On his or her scheduled research day, the student arrived at the lead investigator’s office to participate in the study. The participant was given two documents—a copy of the research consent form and the assignment description—and asked to Figure 3 Screenshot of Video and Coding Timeline Do They “Get It”? Student Usage of SFX Citation Linking Software 459 read both carefully. The student was then seated at the computer and presented with the prepared ProQuest search. Students were aware that screen capture software was in use and were asked to click on it to start it at the beginning of the session. Each screen capture video was saved with no identifiers to associate it with the student participant. The lead investigator was not present in the office after the session started but was available for questions. When the student complet- ed the session, he or she was given a $5.00 gift certificate for the campus bookstore. Data Analysis After all of the student research sessions were completed, Studiocode software was used to analyze the screen capture videos. A button coding window was created to capture the elements the investigators wished to collect. The buttons in the win- dow correspond to choices in navigation, document type, articles selected, and the extent to which Get It! was implemented. As the investigator reviewed each video in actual time, she clicked the buttons that coordinated with the action on the screen. The first time a button was se- lected, a separate timeline was created. (See figure 3.) Subsequent button clicks resulted in clips or instances of the video being marked on the timeline. Once all of the videos were coded, databases were created and timelines with similar data from each video were transferred to the corresponding database. Studiocode software includes a feature that allows for the easy creation of matrices from the data within each database. (See figure 4.) Findings Forty students volunteered for this re- search study and 39 were able to complete the project. Due to technical difficulties, ProQuest was not available during one of the assigned research sessions. Four participants experienced delays (Pro- Quest internal errors) but were still able to complete the assignment. Participant Research Characteristics On average, students spent approximate- ly 10 minutes completing the research assignment. One student spent as little as 3 minutes and 45 seconds, and another student invested 16 minutes and 15 sec- onds. The median length of time that a student spent on the assignment was 10 minutes; the mode was 8:15 minutes. As mentioned above, the ProQuest search resulted in 65 citations listed in reverse chronological order, with 10 citations to a page. Only one student chose all five articles from the first result screen without exploring further pages. On average, most students explored all five additional results pages, often mov- ing back and forth through the listing to take a second look at a promising title. The mean number of times that a student used the “next page” button was 5.41 times (median 4 times; mode 3 times; range 26 times). Ten students modified the search at some point during the session. The change observed most often was to limit the search by clicking the Show Only Full Text link. When this was activated, it released the Scholarly Articles limit and opened the search to all types of docu- Figure 4 Sample of Data Matrix 460 College & Research Libraries September 2011 ments. Most students realized they were now receiving a different results list and used the back arrows to retreat to the original search. Two students decided to improve upon the canned search and added “violence” as an additional search term. The investigators were curious about how the students navigated within the ProQuest search result list. Would the students use the Abstract, HTML Text, PDF, or Get It! links under the article title on each citation? (See figure 5.) The inves- tigators created buttons in Studiocode to mark these actions, but there were ex- tremely few occasions when these choices were selected. Students overwhelmingly clicked on the article title link instead of using the additional finding aids under- neath the title. The decision to further investigate an article appeared to be solely based on the words in the title. Use of SFX During their research sessions, 67 percent, or 26 out of 39 participants, explored the Get It! button at least once. Those who used Get It! chose it a mean of 5 times (median 4.5; mode 3, 4, 6). Of the 26 students who found Get It!, 21 clicked on it in the first three minutes of the session. The shortest amount of time that it took a participant to click on the Get It! button for the first time was 15 seconds, and the longest amount of time was nine minutes. (See figure 6.) Both the median and the mode were 1:45 Figure 5 Screenshot of ProQuest Citation List results Screen Figure 6 Of the 26 Students Who use SFX, the Time it Took Before They Clicked on the get it! Button the First Time Do They “Get It”? Student Usage of SFX Citation Linking Software 461 minutes. Five students used Get It! after the three-minute mark: three of them within 4:30 minutes, one at 6:30 minutes, and one at nine minutes. The five students who used Get It! after the 3-minute mark appeared to find it by trial and error after numerous failed attempts at locating the full text. A total of 23 students used Get It! more than once, and the time until they used it for the second time ranged from 30 seconds to 4:30 minutes (mean 1:37 minutes; median 1:30 minutes; mode 1:00 minute). Of the 26 students who clicked on the Get It! button at least once, six of them (23%, or almost one quarter of the group) did not follow through and click on Go to get to the full text of the article. This speaks to the perceived difficulty of using the SFX software. It requires patience and persistence to get to the full text of an ar- ticle. Those 20 students who were able to follow through to the full text clicked on the Go button a mean of 4.3 times (median 4 times; mode 3 times; range 7 times). Of those students who were able to use the Go button to the full text of an article, 15 of them, or three quarters, decided not to use the Go button at least once within a Get It! action, most often because the full text was not available in another database for the article they had selected. Investigators observed that the loca- tion and appearance of the SFX button on the ProQuest screen plays a role in student usage of the feature. By following the movement of the participant’s mouse on the screen, the investigators noted that students tended to begin their search for the full-text article on the left-hand side of the Abstract page, largely ignor- ing the right-hand frame that contained the Get It! button. There appeared to be little recognition that the blue rectangle marked Get it! Penn State was a clickable button. In addition, the adjoining text link reading “Check for SFX availability” did not assist the students in any way since they had not been introduced to the term “SFX” and had only been exposed to the Get It! terminology in both the classroom and the research documentation. (See figure 7.) Students who did not locate the Get It! button initially tended to try a variety of links in hopes of accidentally locating the full text of the article. Often, the students would click on the publication title link, which then produced a list of all the is- sues available for that particular journal. Another commonly attempted link was the ProQuest Document URL. Several stu- dents even copied the URL and pasted it in another browser window with no success. For the investigators, one unexpected bonus of using screen capture video was the ability to interpret some of the partici- pants’ emotional reactions. Having made a conscious decision to not simultaneously videotape participants for fear of making them uneasy, the expectation was that the screen capture videos would be clinical and unemotional in nature. At times, the students’ frustration levels were palpable as the investigators witnessed the mouse slowly slide to the bottom of the screen as if the student had given up and released the mouse. Another move that seemed to signal frustration was a swirling of the mouse in one place after the student fol- lowed several failed links. It was almost as if the student was spinning his wheels. On the other hand, a student often exhibited pure elation when he or she chanced upon the Get It! button and then appeared to “race” back through the results pages to find other promising titles. Ability to Meet Requirements of Research Assignment Sixty-two percent, or 24 of 39 participants, completed the research assignment and printed the full text of five articles. Of Figure 7 Screenshot of Text Surrounding get it! Button 462 College & Research Libraries September 2011 the 15 students who did not print five articles, most printed at least one full-text article, together with one or more printed abstracts. Four of the 39 participants did not print a single full-text article and printed only abstracts. The investigators reviewed each instance where an abstract had been printed to determine if the stu- dent printed the abstract believing it to be the full text, or if they printed the abstract out of frustration at not locating the full text of the article. Thirteen of the total par- ticipants, or 33 percent, appeared to print the abstract believing it to be the full text. While abstracts were briefly mentioned in the instruction session, there seems to be a great deal of confusion among students in regard to the difference between the full text of an article and the abstract. Conclusions and Future Research In this study, 54 percent of student par- ticipants located the Get It! button early in the session and used it often in their research assignment, while the rest of the participants either found the button by trial and error or not at all. There appears to be little in the design of the image that leads students to believe that it is a click- able, linked button. The location in the right-hand frame also works against the Get It! button, as the students tended to move the mouse over the left-hand side of the screen looking for links more often than the right-hand side of the screen. Par- ticipants did not appear to equate the text “Get It! Penn State” with locating the full text of the article. The adjoining text link “Check SFX for Availability” further added to student confusion, since the term “SFX” had not been defined in the classroom instruction or the research documentation. While links marked Full Text and Full Text-PDF appear under some ProQuest citations on the results list, they were overwhelmingly ignored by the par- ticipant group. Student preference was to click on the longer title link listed above the article type choices. Further research will focus on student understanding of the term “Full Text” and recognition of the Adobe Acrobat PDF symbol. It was observed in this study that students who successfully followed through with the Get It! button and ultimately located the citation and abstract in a new database often did not click on the words “Full Text” or the PDF symbol to fully locate the entire article. The number of students who printed the abstract in place of the full article confirmed investigator observations when interacting with the general stu- dent population, who often cannot tell the difference between an abstract and the full text of an article. The investiga- tors went back and reviewed the video clips in which students chose to print an abstract. In most cases, it appeared that the student printing the one-paragraph abstract thought they were printing the full article. There is reason to include a definition of an abstract and a compare/ contrast example of an abstract and a full- text article in future instruction sessions. While librarians can appreciate the tech- nological breakthroughs that have resulted in the development of SFX, some students appear unaware of the “magic” that occurs behind the scenes that makes the article appear in the window in front of them. The students in this study tended to expect an immediate response after hitting the Go button and often closed the window before the search could be completed in the new database or before the article could fully download. This was especially noticeable when SFX would jump to a database with a text-heavy wait interface. Students seemed overwhelmed by the number of words on the screen and did not utilize links marked “manual download.” This study was a preliminary study of the actual student search behaviors based on volunteer participants. Since this was not a true population sample, no attempt was made to gather or stratify by gender, ethnicity, or any other factors. In further study, a true population sample could be sought and the study could be expanded to other campuses within Penn State University or to other college Do They “Get It”? Student Usage of SFX Citation Linking Software 463 libraries that use SFX software. Another area for advanced research could include a comparative study of SFX use among students who have not received standard bibliographic instruction or a comparison between students who received different types of library instruction from different instructors. The results of this study can influence both the design and presentation of the Get It! feature, as well as undergraduate instruction of its use. A closer focus on us- ability testing could determine if SFX us- age would increase or decrease if the size, wording, and/or position of the Get It! button were changed. The use of citation linking software in colleges and universi- ties is an area ripe for further exploration. Notes 1. Philip Hider, “Coding Online Information Seeking,” Australian Library Journal 54, no. 3 (2005). Available online at www.alia.org.au/publishing/alj/54.3/full.text/hilder.html. [Accessed 26 October 2009]. 2. Jill Livingston, Deborah Sanford, and Dave Bretthauer, “A Comparison of Openurl Link Resolvers: The Results of a University of Connecticut Libraries Environmental Scan,” Library Collections, Acquisitions, and Technical Services 30, no. 3–4 (2006): 179–201. 3. J.E. Grogg, “Linking E-Journals to Databases: Full Text Linking Practices,” Serials Librarian 45, no. 2 (2003): 145–51; Nicholas Lewis, “I Want It All and I Want It Now!: Managing Expecta- tions with Metalib and SFX at the University of East Anglia,” Serials 16, no. 1 (2003): 89–95; R. Stubbings, “MetaLib and SFX at Lougborough University Library,” VINE 33, no. 1 (2003): 25–32; Wendy C. Robertson and Paul A. Soderdahl, “Everything You Always Wanted to Know about SFX but Were Afraid to Ask,” Serials Librarian 47, no 1/2 (2004): 129–38. 4. Hider, “Coding Online Information Seeking.” 5. Ibid. 6. Jina Choi Wakimoto, David S. Walker, and Katherine S. Dabbour, “The Myths and Realities of SFX in Academic Libraries,” Journal of Academic Librarianship 32, no. 2 (2006): 127–36. 7. X. Gong and M. Loomis, “An Empirical Study on Follow-up Library Instruction Sessions in the Classroom,” Electronic Journal of Academic and Special Librarianship 10, no. 1 (2009). 8. K. Antell and J. Huang, “Subject Searching Success Transaction Logs, Patron Perceptions, and Implications for Library Instruction,” Reference and User Services Quarterly 48, no. 1 (2008): 68–76. 9. Wakimoto, Walker, and Dabbour, “The Myths and Realities of SFX in Academic Libraries”; Joel Cummings and Ryan Johnson, “The Use and Usability of SFX: Context-Sensitive Reference Linking,” Library Hi Tech 21, no. 1 (2003): 70–84. 10. Amy Blevins and C.W. Elton, “An Evaluation of Three Tutorial-creating Software Programs: Camtasia, PowerPoint, and MediaSite,” Journal of Electronic Resources in Medical Libraries 6, no. 1 (2009): 1–7. 11. Janet Schnall, Terry Ann Jankowski, and Leilani A. St. Anna, “Using Camtasia Studio© to Enhance Web Instruction Pages and Tutorials,” Journal of Hospital Librarianship 5, no. 1 (2005): 77–81; Laurie Charnigo, “Lights! Camera! Action! Producing Library Instruction Video Tutorials Using Camtasia Studio,” Journal of Library and Information Services in Distance Learning 3, no. 1 (2009): 23–30; Kathleen Carlson, “Delivering Information to Students 24/7 with Camtasia,” Information Technology and Libraries 28, no. 3 (2009): 154–56. 12. Susan Goodwin, “Using Screen Capture Software for Web Site Usability and Redesign Buy-In,” Library Hi Tech 23, no. 4 (2005): 610–21. 13. Hider, “Coding Online Information Seeking”; Eszter Hargittai, “Classifying and Coding Online Actions,” Social Science Computer Review 22 (2004): 210–27. 14. Hider, “Coding Online Information Seeking.” 15. Ibid. 16. Studiocode Business Group, “Case Studies.” Available online at www.studiocodegroup. com/Case_Studies.htm. [Accessed 11 May 2010].