Evidence Summary

Usage Volume and Trends Indicate Academic Library Online Learning Objects and Tutorials Are Being Used

A Review of:

Hess, A. N., & Hristova, M. (2016). To search or to browse: How users navigate a new interface for online library tutorials. College & Undergraduate Libraries, 23(2), 168-183. http://dx.doi.org/10.1080/10691316.2014.963274

Reviewed by:

Ruby Warren

User Experience Librarian

University of Manitoba Libraries

Winnipeg, Manitoba, Canada

Email: ruby.warren@umanitoba.ca

Received: 1 Dec. 2016 Accepted: 17 Jan. 2017

2017 Warren. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To discover how users interact with a new online interface for learning objects, user preferences for types of access when given both browsing and searching options, and user needs for tutorial subject matter.

Design – Mixed methods, with quantitative analysis of web traffic and qualitative analysis of recorded search terms through grounded textual theory.

Setting – An academic library in the Western United States of America.

Subjects – Users of the Libraries’ online tutorials and learning objects.

Methods – The researchers collected web traffic statistics and organically occurring searches from the Libraries’ tutorial access interface. They defined the collection period as the 2013/2014 academic year, with collection beginning in September 2013 and ending in April 2014. Web traffic for organic searches, facilitated searches (search results accessed through clicking on particular words in a tag cloud), and categorical browsing was collected via Google Analytics. They categorized other interaction types (accessing featured content, leaving the page, etc.) under an umbrella term of “other.” Their analysis of web traffic was limited to unique page views, with unique page views defined as views registered to different browser sessions. Unique page views were analyzed to determine which types of interface interaction occurred most frequently, both on-campus and off-campus, and whether there were differences in types of interaction preferred over time or by users with different points of origin. Individual organic search keywords and phrases, and the dates and times of those searches, were separately collected and recorded. One of the researchers coded the recorded organic search terms using grounded textual theory analysis, and the researcher formed generalized categories. They sent these categories and a random sample of 10% of the recorded search terms to librarians unaffiliated with the study, and used their categorizations of the search term samples to validate the initial researcher’s textual analysis.

Main Results – After analyzing the 5,638 unique page views recorded, researchers found that categorical browsing was used more frequently than facilitated searching throughout the year, and more frequently than organic searching for 6 of the 8 recorded months. Organic searching was used more frequently than facilitated searching during most months, while both organic and facilitated searching were less likely to be engaged in by users working on Saturday or Sunday. They found that interactions in the “other” category were quite high, and the researchers attributed this to featured videos on the interface homepage being required for a number of classes. The researchers discovered that patterns in interface use were similar between on-campus and off-campus users, and that most traffic to the interface was through referral from other websites (such as the library homepage). Direct traffic (from URLs manually typed in or in documents) was the second most frequent point of access, while users arriving at the interface from a search engine interaction was a distant third. Grounded textual theory analysis of the 14,428 collected organic searches achieved a 92% consensus in coding, and showed a user focus in searching for specific resources, tasks, and knowledge, rather than broader conceptual searches. Additionally, researchers noticed that a significant number of users performed organic searches for videos that were featured on the front page, possibly indicating that certain users engage with search functions before viewing page content.

Conclusions – The researchers concluded that despite the limitations of the study, the usage volume and trends identified indicate that the Libraries’ online learning objects and tutorials are being used. They also concluded that the categorization and labelling of these learning objects has been successful because the categorical browsing function is used more than the other search functionalities. The researchers determined that they should consider the non-user in the future, and examine the barriers that students, faculty, and staff encounter when attempting to use online learning content. They affirm a need to develop, via further studies, a more thorough understanding of the motivations behind user interactions.

Commentary

There is extensive literature available on web interfaces, usability, and online learning. The authors reference literature in each of these areas, much of it informing the way their online learning materials were created or organized. Library and information science and user experience literature also contains a broad knowledge base of the areas of particular concern in this article: user interface usability and user needs. Although studies of particular web interfaces and their relative effectiveness are fairly common, the questions posed in this study could have broad applicability to any library or learning institution providing online, self-serve instructional materials.

Analyzing recorded organic search terms with grounded textual theory analysis was an appropriate method to draw initial conclusions about user preferences and needs regarding learning object and tutorial subject matter. The methods used in this study to evaluate usage statistics provided an overall picture of the ways users choose to interact with the interface, although a fuller understanding could have been obtained with alternate research methods. However, the authors do not justify or explain how usage statistics regarding page access and methods of interaction could reliably indicate the success of an interface design, and their conclusions in these areas do not logically follow from the study results. The fact that features like categorical browsing interfaces are being used does not necessarily prove that these features have been designed well or intuitively. Alternate methods, such as interviews, usability studies, or heuristic evaluations, would better serve to answer these study objectives.

This study scored a 75% overall validity rating using Glynn’s critical appraisal tool for library and information science research (2006), chiefly due to Section C Study Design (rated 25%) and Section D Results (60%). These low rankings were due to aforementioned problems with the chosen methodology matching study conclusions. Additionally, the authors also do not provide detailed information regarding the final categorizations used in their analysis of search keywords and phrases, which limits our understanding of those results.

The stated objectives of this study are met, and readers interested in examining how users at this institution have used the interfaces available to them will find interesting and relevant information, but researchers interested in the conclusions the authors attempt to draw would be better served by work using a different methodology.

References

Glynn, L. (2006). A critical appraisal tool for library and information research. Library Hi Tech, 24(3), 387-399. http://dx.doi.org/10.1108/07378830610692154