Abstract

Objective – To determine how the behaviour of visually impaired persons significantly differs from that of sighted persons in the carrying out of complex search tasks on the internet.

Design – A comparative observational user study, plus semi-structured interviews.

Setting – Not specified.

Subjects – 15 sighted and 15 visually impaired persons, all of them experienced and frequent Internet search engine users, of both sexes and varying in age from early

twenties to mid-fifties.

Methods – The subjects carried out self-selected complex search tasks on their own equipment and in their own familiar environments. The investigators observed this activity to some extent directly, but for the most part via video camera, through use of a screen-sharing facility, or with screen-capture software. They distinguished four stages of search task activity: query formulation, search results exploration, query reformulation, and search results management. The visually impaired participants, of whom 13 were totally blind and two had only marginal vision, were all working with text-to-speech screen readers and depended exclusively for all their observed activity on those applications’ auditory output. For data analysis, the investigators devised a grounded-theory-based coding scheme. They employed a search log format for deriving further quantitative data which they later controlled for statistical significance (two-tailed unpaired t-test; p < 0.05). The interviews allowed them to document, in particular, how the visually impaired subjects themselves subsequently accounted for, interpreted, and vindicated various observed aspects of their searching behaviour.

Main Results – The investigators found significant differences between the sighted participants’ search behaviour and that of the visually impaired searchers. The latter displayed a clearly less “orienteering” (O'Day & Jeffries, 1993) disposition and style, more often starting out with already relatively long and comprehensive combinations of relatively precise search terms; “their queries were more expressive” (p. 386). They submitted fewer follow-up queries, and were considerably less inclined to attempt query reformulation. They were aiming to achieve a satisfactory search outcome in a single step. Nevertheless, they rarely employed advanced operators, and made far less use (in only 4 instances) of their search engine’s query-support features than did the sighted searchers (37 instances). Fewer of them (13%) ventured beyond the first page of the results returned for their query by the search engine than was the case among the sighted searchers (43%). They viewed fewer (a mean of 4.27, as opposed to 13.40) retrieved pages, and they visited fewer external links (6 visits by 4 visually impaired searchers, compared with 34 visits by 11 sighted searchers). The visually impaired participants more frequently engaged in note taking than did the sighted participants.

The visually impaired searchers were in some cases, the investigators discovered, unaware of search engine facilities or searching tactics which might have improved their search outcomes. Yet even when they were aware of these, they very often chose not to employ them because doing so via their screen readers would have cost them more time and effort than they were willing to expend. In general, they were more diffident and less resourceful than the sighted searchers, and had more trust in the innate capacity and reliability of their search engine to return in an efficient manner the best available results.

Conclusion – Despite certain inherent limitations of the present study (the relatively small sample sizes and the non-randomness of the purposive sighted-searcher sample, the possible presence of extraneous variables, the impossibility of entirely ruling out familiarity bias), its findings strongly support the conclusion that working with today’s search engine user interfaces through the intermediation of currently available assistive technologies necessarily imposes severe limits on the degree to which visually impaired persons can efficiently search the web for information relevant to their needs. The findings furthermore suggest that there are various measures that it would be possible to take toward alleviating the situation, in the form of further improvements to retrieval systems, to search interfaces, and to text-to-speech screen readers. Such improvements would include:

more accessible system hints to support a better, and less cognitively intensive, query formulation;
web page layouts which are more suitable to screen-reader intermediation;
a results presentation which more readily facilitates browsing and exploratory behaviour, preferably including auditory previews and overviews;
presentation formats which allow for quicker and more accurate relevance judgments;
mechanisms for (a better) monitoring of search progress.

In any event, further information behaviour studies ought now to be conducted, with the specific aim of more closely informing the development of user interfaces which will offer the kind of support that visually impaired Internet searchers are most in need of. Success in this undertaking will ultimately contribute to the further empowerment of visually disabled persons and thereby facilitate efforts to combat social exclusion.

Commentary

The last 15 years have witnessed the appearance of a very considerable, and now in fact quickly growing, number of publications dealing with the problems and requirements of visually impaired persons in the context of searching for, selecting, and making use of information and resources on the web. The study reviewed here confirms numerous earlier research findings and furthers, in particular, our understanding of how these users interact with search systems, while more fully exploring to what extent, and for what reasons, this interaction is significantly different from that which one observes among searchers who are not visually impaired. It is a useful contribution, as well, because it again explicitly focuses our attention on what could be done to render internet searching less time-consuming and cognitively burdensome, and accordingly more rewarding, for visually impaired individuals.

Aside from the inherent limitations of the research design, as conceded above, the study’s presentation also displays certain shortcomings. Quite remarkably and inexplicably, the researchers tell us nothing at all about the setting in which they conducted their study. Furthermore, they specify neither what the sampling frame was for the random sample of visually impaired searchers, nor exactly what population(s) the samples were meant to represent. Also, the demographic information which they provide is surprisingly limited; we find here no indications, for example, of educational level, of socio-economic status, or of ethnic or cultural identity. It is, therefore, even apart from any lingering uncertainty regarding the internal validity or the reliability of this research, easily imaginable that at least some readers will, justifiably, feel unsure as to just how pertinent the study’s findings actually are within their own specific environments. We should also note that, while the authors do review and cite some of the important earlier research on their topic, they reference none of the relevant non-English-language literature. Even then, remarkable omissions remain. We encounter here, for example, no mention of the very interesting and still highly pertinent study by Theofanos and Redish (2003). Moreover, our present authors’ lengthy “Discussion” section leaves the reader largely in doubt as to which of the conclusions there being drawn are in fact based specifically on their study’s own new findings, and which on the whole accumulated body of research up to and including this study. However that may be, the conclusions themselves, along with the recommendations which accompany them, strike this reviewer not only as justified, but indeed as having clear and compelling implications, possibly even as amounting to a mandate of sorts, for assistive technology designers as well as for search engine interface developers, if not indeed for website developers in general. Mates (2012) has, after all, recently written: “A disconcerting fact is that many websites and applications are becoming less accessible rather than more” (p. 12). And this is in spite of the fact that many of the proposed approaches to usability improvement are already well understood, and would be relatively easy to follow through on. The findings of the present study are of course in themselves neither generalizable nor necessarily transferable, but they are, taken together with those of related research, distinctly indicative of what steps are possible and appropriate, and therefore these findings do have practical evidentiary value.

Nowhere in this article do we find any mention of a possible role for the library or information science and services (LIS) professional, or indeed any suggestion what role human or institutional intermediation of any kind can fulfill in making things easier for visually impaired searchers. The researchers look solely to software enhancements for whatever solutions to the existing problems may be available. Nevertheless, LIS practitioners – public services librarians and library web services developers in particular, as well as anyone involved in accessibility evaluation – would be well advised take notice of, and to take into account, findings such as those emerging from this study, while decidedly also keeping an eye out for any promising fresh developments, such as certainly for example some published too recently to be mentioned in the article here under review: for example Yang, Hwang, and Schenkman’s (2012) experimental “Specialized Search Engine for the Blind”; important new research on the growing accessibility problems associated with dynamically changing webpages (Brown, Jay, Chen, & Harper, 2012); innovative software approaches like behaviour-driven transcoding (Lunn, Harper, & Bechhofer, 2011) or a prototype webpage restructuring system (Guercio, Stirbens, Williams, & Haiber, 2011); and, by no means least of all, Kerkmann and Lewandowski’s (2012) proposed accessibility evaluation framework. LIS professionals and their organizations can pride themselves on a venerable tradition of striving to ensure broad and efficient access to information, literature, and recorded knowledge for all, regardless of disability. Staying abreast of the results produced by this kind of research on accessibility enhancement will ensure that the practitioner remains aware of a valuable pool of potential evidence on which he or she can draw in making decisions which will serve collectively to sustain, perhaps indeed to strengthen, that tradition.

References

Brown, A., Jay, C., Chen, A. Q., & Harper, S. (2012). The uptake of Web 2.0 technologies, and its impact on visually disabled users. Universal Access in the Information Society, 11(2), 185-199. doi: 10.1007/s10209-011-0251-y.

Guercio, A., Stirbens, K. A., Williams, J., & Haiber, C. (2011). Addressing challenges in web accessibility for the blind and visually impaired. International Journal of Distance Education Technologies, 9(4), 1-13. doi: 10.4018/IJDET.2011100101.

Kerkmann, F., & Lewandowski, D. (2012). Accessibility of web search engines: Towards a deeper understanding of barriers for people with disabilities. Library Review, 61(8/9), 608-621. doi: 10.1108/00242531211292105.

Lunn, D., Harper, S., & Bechhofer, S. (2011). Identifying behavioral strategies of visually impaired users to improve access to Web content. ACM Transactions on Accessible Computing, 3(4), Article No. 13. doi: 10.1145/1952388.1952390.

Mates, B. T. (2012). Information power to all patrons. In C. Booth (Ed.), Making libraries accessible: Adaptive design and assistive technology (= Library Technology Reports 48:7) (pp. 7-13). Chicago: American Library Association. doi: 10.5860/ltr.48n7.

O'Day, V. L., & Jeffries, R. (1993). Orienteering in an information landscape: How information seekers get from here to there. In S. Ashlund (Ed.), Bridges between worlds: Conference on Human Factors in Computing Systems: INTERACT '93 and CHI '93: Amsterdam, The Netherlands, 24-29 April 1993 (pp. 438-445). New York: ACM Press. doi: 10.1145/169059.169365.

Theofanos, M. F., & Redish, J. (2003). Guidelines for accessible – and usable – Web sites: Observing users who work with screenreaders. interactions, 10(6), 38-51. doi: 10.1145/947226.947227. Expanded “Authors’ version” retrieved 4 Feb. 2013 from http://www.redish.net/content/papers/interactions.html

Yang, Y.-F., Hwang, S.-L., & Schenkman, B. (2012). An improved Web search engine for visually impaired users. Universal Access in the Information Society, 11(2), 113-124. doi: 10.1007/s10209-011-0250-z.