Filling the Gap in Database Usability: Putting Vendor Accessibility Compliance to the Test ARTICLE Filling the Gap in Database Usability Putting Vendor Accessibility Compliance to the Test Samuel Kent Willis and Faye O'Reilly INFORMATION TECHNOLOGY AND LIBRARIES | DECEMBER 2020 https://doi.org/10.6017/ital.v39i4.11977 Samuel Kent Willis (samuel.willis@wichita.edu) is Assistant Professor and Technology Development Librarian, Wichita State University. Faye O’Reilly (faye.oreilly@wichita.edu) is Assistant Professor and Digital Resources Librarian, Wichita State University. © 2020. ABSTRACT Library database vendors often revamp simpler interfaces of their database platforms with script- enriched interfaces to make them more attractive. Sadly, these enhancements often overlook users who rely on assistive technology, leaving electronic content difficult for this user base despite the potential of electronic materials to be easier for them to access and read than print materials. Even when providers are somewhat aware of this user group's needs there are questions about the effect of their efforts to date and whether accessibility documentation from them can be relied upon. This study examines selected vendors’ VPAT reports (Voluntary Product Accessibility Template) through a manual assessment of their database platforms to determine their overall accessibility. INTRODUCTION Libraries are now providing more access to online databases than ever before. In fact, as Blechner notes, most of the “information patrons seek is located in indexes and databases that are only available digitally. Students and faculty rely heavily on these resources in completing course assignments and conducting research.”1 Vendors frequently revamp simpler interfaces of their database platforms with script-enriched interfaces to make it more attractive to students.2 Sadly, these enhancements often overlook users who rely on assistive technology, leaving electronic content difficult for this user base despite the potential of electronic materials to be easier for them to access and read than print materials. Online databases not only bridge the gap for distance users but can also improve service to users with print disabilities.3 Resources produced digitally or properly digitized for online dissemination more readily allow all users, including patrons with physical or mental impairments, to make use of them than do print materials. These resources allow all patrons to have access to updates and new publications at the same time, and can be presented in multiple formats.4 Key features of electronic access that are helpful to users are zooming in on text and automatic reflow to reduce the need to scroll, improving color contrast or changing colors to make looking at the screen easier on the eyes, and the capability of the text to be read aloud by either a built-in feature or user-provided assistive technology such as a screen reader or refreshable braille display.5 All of this, however, presupposes that the content can be accessed using the platform provided by the vendor to navigate the database, and that the documents be made at least minimally accessible. The question is then, how well do these platforms interact with the assistive technologies employed by the largest minority group in the United States (persons with disabilities), relying on libraries to facilitate “their full participation in society,” and to achieve academic success?6 Many vendors provide accessibility documentation pertaining to their database platforms. Some note considerable limitations in accessibility while others claim to be highly accessible when in mailto:samuel.willis@wichita.edu mailto:faye.oreilly@wichita.edu INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 2 fact they may be no better that the former. Accessibility guidelines like Section 508 of the Rehabilitation Act sets forth are a good starting point, but related literature has emphasized that even conformance to these standards does not guarantee they will be usable for all.7 LITERATURE REVIEW Accessibility in libraries has been examined from a variety of vantage points. Some studies were an inspiration to our work and complementary to it, though our manual and holistic review of library databases from third parties was a unique approach. Dermody and Majekodunmi conducted a usability study of electronic databases, focusing on students unable to fully make use of analog materials.8 They asserted that technology, online databases in particular, can either be a help or a hindrance to users with print disabilities.9 After having visually impaired students use screen readers to test three proprietary databases, the authors concluded that their use of the platforms was disrupted by advanced features designed to engage users. Study participants were frustrated to have to abandon a research article applicable to their topic because it was presented in an unreadable format.10 The authors found that as website design evolves to enhance the user experience, screen reader users and others who relied on assistive technology were often overlooked and unable to make use of the sites due to the construction of the platforms and due to inaccessible PDFs.11 Regarding accessibility assessment, the authors asserted that database providers were unlikely to catch all issues or evaluate their products accurately.12 The legal responsibility for these shortfalls, however, belongs to the subscribing institutions.13 The results of Dermondy and Majekodunmi’s survey demonstrated that the usability of electronic databases was stunted by the limitations of screen readers, the platforms or materials themselves, and by insufficient information literacy training for assistive technology users. In 2015, Blechner wrote about the challenges law students with disabilities face in their education, similar to any undergraduate or graduate program. This study was conducted by a librarian with screen-reading software and an accessibility checklist. Blechner highlighted that using research databases with assistive technology to locate material and complete assignments was a barrier to completing legal education programs or passing the bar.14 In academic institutions, student success is related to library access. As much of a library’s resources are online, inaccessible electronic resources present a massive issue.15 Database design is especially important to users who use assistive technologies to access online resources. Blechner pointed out that an additional barrier to online resource access was an average delay of three years before an accessible version of a requested platform or service was prepared.16 If an undergraduate degree took four years to complete, a freshman living with a disability would be a senior before they have equitable access. Blechner stressed a need for librarians to go beyond addressing the accessibility of their native web platforms and to inspect vendor platforms prior to subscribing to them. Libraries "rarely raise the issue when selecting electronic indexes and databases for procurement from outside vendors.”17 Libraries cannot adequately serve patrons and comply with legal requirements if they are unable to provide meaningful access to information for all library patrons. A significant point from Blechner’s article was that compliance with federal standards does not guarantee a service is easy to use or usable at all. “A product can receive a rubber stamp even when it is not functional or usable despite a company’s good faith efforts to provide an accessible product.”18 Other authors have supported this claim, which, along with our own observations, was an impetus for this research. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 3 In Chapter 8 of Ensuring Digital Accessibility through Process and Policy, Lazar, Goldstein, and Taylor used different web accessibility evaluation methods to verify vendor accessibility information on their platforms. The three methods they examined were (1) Having users with disabilities test the platform or content using assistive technology; (2) Conducting an expert review to ensure compliance with usability standards; and (3) Performing an automated scan of the content using scanning software. Regardless of method chosen for evaluation, the authors stressed the importance of continuous evaluation, as content can easily become inaccessible through changes to the user interface. The authors identified strengths and weaknesses to each of the approaches but recommended that whenever possible Method One be used from early on in the development with a goal of ongoing improvement, and that Method Two be used in conjunction with it. When specifically examining the accessibility review of vendor-supplied database content, the authors noted that a Voluntary Product Accessibility Template (VPAT) is one form of Method Two; however, its findings are only reliable insofar as the template is completed by an accessibility expert, and even then there is room for disagreement. 19 This supported the approach we undertook in this study to examine vendor databases and compare our findings with vendors’ VPATs when available. In our professional experience, some VPAT creators are experts in accessibility, while others are not, and even among experts opinions vary, which led us to the same conclusion as Lazar et al.: “Multiple experts, working independently, can increase the validity of the accessibility inspection.”20 Jennifer Tatomir created a checklist, the Tatomir Accessibility Check-list (TAC), to apply the accessibility guidelines to a usability study.21 At the time the article was written in 2010, the then- current web accessibility standards would have been the WCAG 2.0 (released in 2008) and Section 508 standards, last revised in 1998 to include equitable access to information and data under the protection of the law. WCAG is now in version 2.1, with version 2.2 already in development, and Section 508 requirements were updated in 2017 to include many WCAG principles. The TAC examined (1) documents and webpages; (2) bypass links; (3) page element labels; (4) captions for images and figures; (5) scripts and code that would interfere with assistive technology; (6) duplicate links; (7) transcripts for audiovisual material; (8) site organization; (9) timed responses; and (10) the accessibility of web forms.22 While the testing criteria used in this study differed from ours on several points, Tatomir and Durrance’s work supported our creation of the Accessibility Remediation Guide (ARG), a checklist of which Section 508 standards would be the most important to our libraries (see Appendix A). The ARG will be discussed in more detail later in this article. Finally, DeLancey conducted an assessment of the accuracy of 17 vendors’ VPATs which was similar to one aspect of our research. Her work used automated assessment tools as the primary measure for comparison against VPATs, while this study is a direct comparison of two expert reviews.23 The goal of our research project was to determine the accuracy of vendor-supplied accessibility documentation—VPATs in particular—to inform future communications with those vendors as well as collection development decisions moving forward. The studies used in this paper used sighted librarians, students using screen readers , and native users of screen readers to conduct accessibility testing. Ideal candidates for accessibility testing would of course be users with disabilities. However, this approach can be complemented by a review for basic usability and compliance with Section 508 standards. Librarians are also ideal candidates for accessibility testing since they have access to and expertise in using research databases and are committed to providing access to all.24 Librarians can also provide information INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 4 in advance, in anticipation of need. The findings of such accessibility testing could be beneficial in drafting licensing agreements that would ensure a higher level of service for patrons with disabilities. As Blechner said, it is “critical that libraries independently exercise their power as buying agents to improve the state of electronic resource accessibility.”25 Librarians can be instrumental in the development of database platforms moving forward by continually checking the accessibility of these platforms and sharing opportunities for enhancement with vendors.26 METHODOLOGY This study made use of the ARG (Appendix A) for both VPAT accuracy analysis, and overall testing of database accessibility. The ARG was based on the standards set forth in Section 508 of the Rehabilitation Act and related VPAT creation guidelines. The ARG has 11 criteria and was originally intended for accessibility evaluation of new databases, but two criteria were merged with others to make nine in order for it to be easier for a graduate student to evaluate. Due to the breadth of technologies covered in a VPAT, the authors determined that many of the sections in a VPAT were not relevant to our examination. An example of this is Section 1194.25 which refers to physical accessibility of kiosks and the like and therefore has no bearing on electronic content. The functionalities we chose to test were a restricted subset of the functionalities assessed in a VPAT, but this set was selected for several reasons. Some of the guidelines were selected due to their wide impact on a variety of assistive technologies related to the needs of persons with disabilities including blindness, deafness, limited vision, hearing, or mobility. Following these guidelines would improve the performance of the platforms for use with screen readers and keyboards, eye tracking software, refreshable braille displays, and other assistive technology.27 Other tests were chosen as a result of our preliminary investigation and use of the databases, and resulting evidence that they were areas on concern. Finally, some of these items to be examined were selected because a lack of accessibility in these areas would result in drastic limitations to the usability and therefore utility of the databases overall, even if they rarely applied. The reasons behind this study were threefold. Firstly, 62 percent (48) of our vendors had provided no VPAT. This test would fulfill a similar purpose, allowing us to know how accessible these databases without VPATs were as well as identify particular areas requiring remediation in anticipation of patron needs. Secondly, our library had anecdotal evidence that some of the VPATs that were provided contained inaccuracies but without a thorough examination it was impossible to know the particulars or extent of the issues. Finally, the goal of the project was to identify trends in database accessibility and usability for persons with disabilities, comparing major database providers with smaller vendors. These findings will give insight into what most needs to be addressed based on the size and type of content provider and will likely have some bearing on similar institutions’ collections. These are the criteria we used in testing. Other institutions, if following our example, would likely want to adapt the list to meet their needs and institutional priorities: 1) Keyboard Navigation and Intuitive Forms 2) Presence of Keyboard Traps 3) Platform Optical Character Recognition (OCR) 4) Document OCR 5) Alternative Text 6) Table Data INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 5 7) Skip Navigation 8) Transcripts 9) Closed Captions Note: Criteria 3, 8, and 9 included testing support materials, including video tutorials for Criteria 8 and 9. Once we had determined which sections to include, we hired and trained a graduate student to use the ARG to examine each database to which we subscribed on the nine criteria. We were awarded a grant to fund the student’s work. The student tested each database platform and a minimum of three items in each database, manually checking them using a keyboard and screen reader (NVDA). This testing fulfilled the majority of our priorities but was supplemented by her checking for transcripts and captions for video content. While the findings cannot be comprehensive in a manual test this work is complementary with existing VPATs in enabling us to identify areas in need of development in vendor platform usability. It is noteworthy that by testing the databases manually with a screen reader, certain limitations in the usability of the databases were discovered that would not have been revealed by doing automatic checks as have been done in similar studies. An excellent example of this is poorly designed skip navigation (which was found for nearly half of our databases). Using the data we collected with the assistance of our graduate student, we compiled and compared our findings on the various vendors. Our scoring (based on representative random sampling) gave one point for a database passing a single criterion, half a point for a minor issue, and no points for any criteria that failed our tests. The scores were then added together, ignoring any criteria which did not apply to particular databases, to form a composite score. When analyzing vendors with multiple databases, their overall score was based on the average of the individual database scores. This enabled us to codify a percentage of accessibility for every vendor and compare them. For the purposes of this study, we will refer to any vendor that provided the University Libraries with 15 or more database subscriptions as large vendors (LVs), and the rest as small vendors (SVs). Given that we only subscribe to 15 or more databases from a few vendors, some of the vendors we classified as SVs would likely be considered LVs at other institutions. RESEARCH FINDINGS VPAT Accuracy Assessment As previously stated, one goal of this research was to measure the accuracy of vendor-supplied VPATs. 227 databases assessed had an associated VPAT from the vendor, but the rest did not ( see Appendix B for list of all databases by vendor). We used the ARG (Appendix A), and compared the vendors’ claims on the VPAT to our manual testing of the database functionality. Of the 227 databases, only 10 databases were found to fully match the claims the vendor made on the VPAT for the 11 criteria assessed from the ARG. Databases where the VPAT claims did not match the findings of the testing on one criteria were given a score of “Partial Match.” Of the 227 databases, 138 were considered partial matches (See figure 1 for details). The main incongruity between VPATs and our results were due to the databases not having sufficient skip navigation, meaning they did not have appropriate or functional bypasses. These issues are likely due to outdated VPATs that do not reflect the latest changes to the databases but could also be the result of vendors’ lack of understanding of what it means to be truly usable by persons with disabilities. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 6 For databases that failed two or more of the criteria tested, a score of “Not a Match” was given, with 79 databases of 227 failing (See figure 1 for details). For these databases, skip navigation and alternative text were the main issues. The databases, when presenting essential content in an image, like a photo or chart, did not provide an alternative presentation of that content, which means only sighted users could access the data from that image. The findings of this study are similar to the data from the overall usability study, finding that vendors struggled with skip navigation and alt-text, as we will discuss below. Some of these databases were also found to have keyboard traps that prevented screen reader users from navigating to the entire site and at times may even trap the user’s navigation in a single content area. This number of inconsistencies was even higher than the authors anticipated but reinforced all the more the importance of not taking information in VPATs for granted, especially when the VPAT is several years old and the platform has undergone any changes. Figure 1. VPAT Accuracy Assessment Accessibility Analysis Overall Related to the VPAT accuracy assessment, we conducted manual tests of our databases and database platforms, both those with and without a VPAT provided by the vendor. Of our 351 databases, 124 (35 percent) had no related VPAT, and on a whole, examining all criteria, we found them notably less accessible. That said, there were exceptions where databases with no associated VPAT still had accessibility information giving reasonable detail, and others where the VPAT provided was inaccurate or where it highlighted significant accessibility issues (see tables 1 and 61% 4% 35% Partial Match Match Not a Match INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 7 2). The average composite score of VPAT-linked databases was 74 percent, compared to 67 percent for those with none (see table 3 for comparison). Each criterion was compared and any instance where one category of databases was more than five percent higher than the other was highlighted. Table 1. Summary of Issues for Databases with VPATs (227 total) Good Partial Poor Applicable N/A Download OCR 50 32 78 160 67 Skip Navigation 68 124 35 227 0 Transcripts 42 4 15 61 166 Alt Text 71 39 12 122 105 Tables 17 38 4 59 168 Captions 35 0 8 43 184 Platform OCR 108 41 7 156 71 Keyboard Navigation 202 22 3 227 0 Keyboard Traps 224 0 3 227 0 Average 90.78 33.33 18.33 142.44 84.56 Table 2. Summary of Issues for Databases without VPATs (124 total) Good Partial Poor Applicable N/A Download OCR 61 14 27 102 22 Skip Navigation 47 27 48 122 2 Transcripts 3 0 8 11 113 Alt Text 52 38 26 116 8 Tables 30 8 2 40 84 Captions 6 1 4 11 113 Platform OCR 88 10 16 114 10 Keyboard Navigation 74 35 15 124 0 Keyboard Traps 123 0 1 124 0 Average 53.78 14.78 16.33 84.89 39.11 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 8 Table 3. Comparison of Databases with and without VPATs (351 total) Percent Good of Applicable Databases with VPATs (227) Percent Good of Applicable Databases without VPATs (124) Download OCR 41.25% 66.67% Skip Navigation 57.27% 49.59% Transcripts 72.13% 27.27% Alt Text 74.18% 61.21% Tables 61.02% 85.00% Captions 81.40% 59.09% Platform OCR 82.37% 81.58% Keyboard Navigation 93.83% 73.79% Keyboard Traps 98.68% 99.19% Average 73.57% 67.04% The biggest barriers to accessibility found in this study pertained to downloadable files’ OCR, skip navigation, transcripts, and alternative text (see figure 2 and table 4). The accessibility of downloadable files through OCR or alternative formats (TXT, HTML, etc.) was found to be the most major concern, though it did not apply to all databases. Its overall score for applicable databases was 51 percent, based on the frequency and severity of the issues. Many database platforms had full text available for download only through PDFs that were images of text or that had other issues failing to work with assistive technologies. It was more than twice as frequent for a database to have inaccessible downloadable files as inaccessible full text online. Often HTML or TXT formats were not available for download, but in instances where it was available through the vendor’s platform, another means of accessing the information mitigated this issue. Other times, however, the full text on the platform itself was not accessible. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 9 Figure 2. Accessibility Issues by Database Table 4. Summary of Issues by Database Platform (351 total) Good Partial Poor Applicable N/A Percent Good of Applicable Download OCR 111 46 105 262 89 51.15% Skip Navigation 115 151 83 349 2 54.58% Transcripts 45 4 23 72 279 65.28% Alt Text 123 77 38 238 113 67.86% Tables 47 46 6 99 252 70.71% Captions 41 1 12 54 297 76.85% Platform OCR 196 51 23 270 81 82.04% Keyboard Navigation 276 57 18 351 0 86.75% Keyboard Traps 347 0 4 351 0 98.86% Average 144.56 48.11 34.67 227.33 123.67 72.67% A lack of or poorly executed skip navigation accounted for the second greatest number of issues by vendor. This criterion’s final score was 55 percent. When skip navigation existed, the most common problem was for it to not redirect to the main content. Often times, for example, on the search results page, the link would take the user to the filters in the margin with no easy way to bypass them and get to the actual results. Eighty-three databases were found to have no skip navigation whatsoever, but the majority of issues found were from existing bypass links not working as intended. D O W N L O A D O C R S K I P N A V I G A T I O N T R A N S C R I P T S A L T T E X T T A B L E S C A P T I O N S P L A T F O R M O C R K E Y B O A R D N A V I G A T I O N K E Y B O A R D T R A P S NUMBER OF DATABASES T E S T IN G C R IT E R IA Good Partial Poor INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 10 Databases with audiovisual materials made up a relatively small portion of our databases, but when these types of items existed, problems were not infrequent. Additionally, we examined support videos made available by database providers to test all multimedia content for transcripts and captions. Twenty-seven out of 72 (38 percent) were determined to have inaccurate transcripts or be in need of them. Captions are irrelevant to non-visual materials, so were only applicable to 54 databases. Of these, 13 (24 percent) were found lacking. Therefore, transcripts were the bigger issue. Nearly half of the databases with images had at least minor issues with alternative text, whether in documents or the platforms themselves. In many cases, this issue was not identified by the vendor in any accessibility documentation because alternative text was present, but not properly descriptive. Thirty-eight databases (16 percent of applicable) had major issues where images were important to the performance of the platform or database and no alternative text was provided. In database materials, charts and graphs were often lacking any alternative text, though on occasion we found the information conveyed in the chart was covered in the main text. In these instances, that was not counted as an issue. The results for tables were similar. Both in the platforms and the documents, tables often lacked identifying header and cell information for screen readers to make sense out of the data. A few were entirely unreadable. Fifty-two of 99 databases with tabular data (53 percent) had problems, but most of them were not major, and for this reason, tables were of less concern than alternative text. Finally, keyboard navigation was a rarer issue, but still was found to be a concern in 75 databases (21 percent). This was often related to images or forms not having descriptive text for screen readers, so non-visual users would be unable to know the purpose of the form, etc. On a few occasions database platforms would have keyboard traps that prevented screen reader users from navigating to the entire site, or more often at least buttons or links that could be used only with a mouse. While our testing only included keyboard navigation, it is important to remember that if it is not usable by keyboard, neither is it likely to work with other assistive technology used for navigation. While this area was of least frequent concern of all criteria we tested, it is nevertheless a vital part of making any website or platform truly usable. All these findings were important to our study as they helped us to identify areas of need, especially for databases that had no corresponding VPAT. Whether the databases had a VPAT or not, this research provided us with the details needed to reach out to database providers and request specific improvements. Vendor Comparison by Size The final goal of our research was to compare the relative accessibility of database providers based on the number of databases we subscribed to from each. While at times we may have subscribed to only a small number of databases from a larger content provider, there was a general correlation between what we considered SVs in this study and those vendors that only offer a more limited number of collections. In assessing the percent accessible a provider was for each criterion, we added all good scores, one point for each related database, to the partial scores, one half a point for each, then divided it by the total number of databases in this area. In this way, minor issues were not recorded as negatively as major issues. Overall accessibility of the LV databases was found to be significantly higher than accessibility of individual databases and SVs (see tables 5 and 6 for findings for LVs and SVs respectively). Our INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 11 findings showed our LVs to have an average score of 74 percent accessibility, compared to 69 percent for SVs, both averages being based on the number of applicable databases. There were two tested criteria, however, that LVs scored lower on than SVs: Downloadable files’ OCR, and tables. The details of each criterion will be discussed below. Most LV content is on a consistent platform and we found, similar to an earlier study, that this consistency helped those materials to be more accessible.28 The issues LV databases most often had were related to individual items, rather than to the platform as a whole. For example, LVs were found to have frequent problems with PDF files. Given that our LVs account for 61 percent of our databases (214 of 351) and they are typically larger than the databases of SVs, this has significant impact on ongoing vendor communication and accessibility remediation efforts. Skip navigation issues was the largest problem found for SVs. Interestingly, while no LVs were entirely missing skip navigation, a lack of proper functionality was a major concern for half of them, accounting for 121 databases. Thirteen databases were found where LVs had no skip navigation. In contrast, 70 SV databases (52 percent of SV content for which this criterion applied) had no skip navigation or it failed to function at all. An additional 30 SV databases and 121 LV databases had improperly functioning bypass links. Overall, SVs were more likely to have none at all, and LVs were more likely to have it not properly set up. Full text OCR results varied greatly depending on the type. Platform OCR showed little difference between LVs and SVs, both being found to be 82 percent accessible. As mentioned previously, downloadable files OCR had more accessibility problems than platform OCR, but there was a large difference between LVs and SVs. For this criterion LV content was found to be only accessible about 40 percent of the time, and SV content 70 percent of the time. This may be due to SVs generally having smaller databases so it is less difficult to address accessibility needs for individual items. Whatever the cause, the disparity between LVs and SVs in this area was very significant. Transcripts and captions were far more common for LVs than SVs. Fifty-five databases (26 percent of LV databases) included audiovisual material, including support tutorials, while only 17 (12 percent) of SV content did. LV content was found to be accessible 73 percent of the time for transcripts, and 82 percent for captions. Applicable SVs on the other hand were only 41 percent accessible for transcripts, and 66 percent accessible for captions. This demonstrates the need for development in both these areas, but especially for transcripts, which when synchronized with the videos have the capability to full more user needs than captions can. Closely following transcripts was alternative text for non-textual content like charts, diagrams and other images. It is worth mentioning that some databases have images neither in their platforms nor in their collection materials. If the platform is simple and the database only provides abstracts, for example, there may be no images, in which case this criterion does not apply. Nearly one-third (113) of the databases were found to have no images. Of the 238 databases with images, we found at least some issues with 115 (48 percent of applicable, 33 percent overall), there being no significant difference between SVs and LVs as a whole. Individually, the platforms varied greatly, and regarding major limitations in alternative text there were found to be 21 SV databases, but only 17 LV databases. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 12 While table accessibility applied to only 99 databases, there were significant issues found particularly with one LV. Given the disparity between LVs it is impossible to draw meaningful conclusions comparing LVs and SVs for this criterion. Further study is needed in this area. Finally, the areas of least frequent concern were keyboard navigation and keyboard traps. Seventy-five databases (21 percent) were found to have suboptimal navigation. In this case, LVs did not have as many issues as SVs. Optimization is needed for them, but only one LV had major issues in this area. Forty percent of SV databases (55 of 137) had at least some navigation issues identified, whereas only nine percent of LV databases (20 of 214) had any issues in this area. As for major issues, only four databases were identified in our study as having keyboard traps, two SVs and two LVs. These only seemed to appear for separate platforms and never for large ones, suggesting that our vendors are likely aware of this issue and avoiding it in newly created platforms. The authors hope the remaining databases with this issue will not be neglected in making these improvements. To sum up, LV content was found to be more accessible overall. Their largely consistent platforms more often had skip navigation (29 percent more), transcripts (32 percent more) and captions (16 percent more) for multimedia content, and superior keyboard navigation (18 percent more). SV platforms, however, had a higher score on downloadable files OCR (31 percent more) and on tables (24 percent more). See table 7 for detailed comparison. Table 5. Issues by LV Database (214 total) Good Partial Poor Applicable N/A Download OCR 52 25 86 163 51 Skip Navigation 80 121 13 214 0 Transcripts 38 4 13 55 159 Alt Text 59 41 17 117 97 Tables 14 35 4 53 161 Captions 31 0 7 38 176 Platform OCR 106 39 8 153 61 Keyboard Navigation 194 13 7 214 0 Keyboard Traps 212 0 2 214 0 Average 87.33 30.89 17.44 135.67 78.33 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 13 Table 6. Issues by SV Database (137 total) Good Partial Poor Applicable N/A Download OCR 59 21 19 99 38 Skip Navigation 35 30 70 135 2 Transcripts 7 0 10 17 120 Alt Text 64 36 21 121 16 Tables 33 11 2 46 91 Captions 10 1 5 16 121 Platform OCR 90 12 15 117 20 Keyboard Navigation 82 44 11 137 0 Keyboard Traps 135 0 2 137 0 Average 57.22 17.22 17.22 91.67 45.33 Table 7. Comparison of LV Databases and SV Databases (351 total) Percent Good of Applicable Databases from LVs (214) Percent Good of Applicable Databases from SVs (137) Download OCR 39.57% 70.20% Skip Navigation 65.65% 37.04% Transcripts 72.73% 41.18% Alt Text 67.95% 67.77% Tables 59.43% 83.70% Captions 81.58% 65.63% Platform OCR 82.03% 82.05% Keyboard Navigation 93.69% 75.91% Keyboard Traps 99.07% 98.54% Average 73.52% 69.11% INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 14 CONCLUSION AND LIMITATIONS This investigation was intended to complement existing studies related to library database accessibility. It was unique in that it manually analyzed content from every database subscription in the University Libraries, rather than only major or representative databases or automated tests. Building a comparison between vendor VPATs and our manual assessment was a key value of this research that we hope will be further developed in future inquiry. The comparison of different types of vendors was also important. While the consistency of LV platforms was found to improve the sites overall, the authors expected that LV content would be more compliant with accessibility regulations than they were found to be. From a usability and accessibility perspective, the increased cost of these databases was deemed to be associated with too little improvement of service. It matters little how clean a platform looks to visual users, for example, if it is impossible or very difficult to use by non-visual users. As anticipated, there were few instances of keyboard traps (when a keyboard and screen reader user is caught in a loop or on a single link when attempting to navigate through the website). When these occur, however, it is a major concern, as it renders the site virtually useless for non-mouse users. There was no significant difference between LVs and SVs on three of nine criteria—including keyboard traps— and on two criteria, SVs were superior. Therefore, despite that LVs were found to be 14 percent more accessible on average, the authors urge LVs to work diligently to address the areas where they were found to be deficient. Both aspects of this study concluded that vendors generally misunderstood the execution of sk ip navigation and alternative text, as a usability study of databases proved many databases failed in fulfilling these criteria, while a separate study of their VPATs’ accuracy proved vendors claimed they did comply with the criteria, while the platform was found to not comply fully. This study is limited in that few samples were able to be examined for each content type in every database platform. The authors anticipate that a deeper investigation would bring to light additional accessibility concerns. Another limitation of this research was related to the time involved in testing. Database platforms changed during the course of this work, but the results of this study pertain to only a short period of time, making them in cases outdated even at the time of this writing. Therefore, the manual testing we have performed would work best when used in conjunction with automated tools for testing database content as other studies have done. The authors hope that further study in this area could involve persons with varied impairments to test the platforms directly and assert that there is potential for collaboration between vendors and libraries in this area. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 15 APPENDIX A: ACCESSIBILITY REMEDIATION GUIDE The authors developed the ARG for database testing prior to signing licensing agreements with vendors. While initially create d based on VPAT Version 1 criteria as defined in Section 508 Standards, it was adapted and cross -referenced with VPAT Version 2 criteria following the refresh of Section 508 in January of 2018. The organization of the criteria was altered greatly at tha t time, but VPATs from vendors may use either version, depending on the age of the VPAT. Finally, it was used in this study to create the testing criteria. Testing Criteria VPAT Version 1-1.6 Standards VPAT Version 2-2.3 Standards Notes Section 1194.22 (web-based intranet and internet information and applications) Related standards after Section 508 Refresh 5 (alternative text) A) A text equivalent for every non-text element shall be provided (e.g., via “alt,” “longdesc,” or in element content). E101 (Web, Software), E201 (Application) WCAG: 1.1.1 Non-text Content 8 and 9 (transcripts and closed captions) B) Equivalent alternatives for any multimedia presentations shall be synchronized with the presentation. 500 (Software) WCAG: 1.2.2 Captions (Prerecorded) and 1.2.3 Audio Description FOR STREAMING MEDIA ONLY “Equivalent alternatives” include transcripts. 3, 4 and 6 (platform OCR, document OCR, and table data) D) Documents shall be organized so they are readable without requiring an associated style sheet. E205.2-4 (Electronic Content) WCAG: 1.3.2 Meaningful Sequence “Documents” describes the webpage. Is the webpage well organized so it’s readable without style elements (colors, blocking, font sizes, etc.). 1 and 3 (keyboard navigation and intuitive forms, and platform OCR) L) When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with the functional text that can be read by assistive technology. E205.2-4 (Electronic Content) WCAG: 2.1.1 Keyboard Does the database include interactive content (buttons, check boxes, or other mouse input), news tickers, media players, browser games etc.)? Is this content accurately identified via text for use with screen readers? INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 16 Testing Criteria VPAT Version 1-1.6 Standards VPAT Version 2-2.3 Standards Notes 1 (keyboard navigation and intuitive forms) N) When electronic forms are designed to be completed on- line, the form shall allow people using Assistive Technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues. E205.2-4 (Electronic Content) WCAG: 3.2.1 On Focus Definition of “form” includes search boxes in databases. Is the search box, search boxes’ purpose, and purpose accurate? 7 (skip navigation) O) A method shall be provided that permits users to skip repetitive navigation links. E205.2-4 (Electronic Content) WCAG: 2.4.1 Bypass Blocks and 1.3.1 Info and Relationships Section 1194.24 (Video and Multi-Media Products) Related standards after Section 508 Refresh 8 and 9 (transcripts and closed captions) E) Display or presentation of alternate text presentation or audio descriptions shall be user-selectable unless permanent. 400 (Hardware) WCAG: 1.2.1 and 1.2.3 Audio Description or Media Alternative FOR STREAMING MEDIA ONLY Section 1194.31 (Functional Performance Criteria) Related standards after Section 508 Refresh 3, 4 and 6 (platform OCR, document OCR, and table data) A) At least one mode of operation and information retrieval that does not require user vision shall be provided or support for assistive technology used by people who are blind or visually shall be provided. 302.1 (Vision) WCAG: 1.4.5 Images of Text Do PDFs have optical character recognized (OCR) text, or are they only images of text? If they do have OCR text, is it accurate? Is it missing information in images or figures? INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 17 Testing Criteria VPAT Version 1-1.6 Standards VPAT Version 2-2.3 Standards Notes 8 and 9 (transcripts and closed captions) C) At least one mode of operation and informational retrieval that does not require user hearing shall be provided, or support for assistive technology used by people who are deaf or hard of hearing shall be provided. 303.4 (Hearing) WCAG: 1.2.1 and 1.2.2 1 and 2 (keyboard navigation and intuitive forms, and presence of keyboard traps) F) At least one mode of operation and information retrieval that does not require fine motor control or simultaneous actions and that is operable with limited reach and strength shall be provided. 303.7 (limited manipulation), 303.8 (limited reach) WCAG: 2.1.1 Keyboard Section 1194.41 (Information, Documentation and Support) Related standards after Section 508 Refresh 3, 8 and 9 (platform OCR, transcripts, and closed captions) B) End-users shall have access to a description of the accessibility and compatibility features of products in alternate formats for alternate methods upon request, at no additional charge. 602.2 (Accessibility and Compatibility Features) and 603.2 (Information on Accessibility and Compatibility Features) WCAG: 3.3.5 Help INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 18 APPENDIX B: DATABASE BY VENDOR LIST USED IN VPAT ACCURACY AUDIT Vendor Name AAPG (American Association of Petroleum Geologists) AAPG/Datapages ABC-CLIO ARBAonline ACLS (American Council of Learned Societies) ACLS Humanities E-Book ACM (Association of Computing Machinery) ACM Digital Library ACS (American Chemical Society) SciFinder Adam Matthew Digital African American Communities Migration to New Worlds American Indian Histories and Cultures American West Digital Collection AIAA (American Institute of Aeronautics & Astronautics) AIAA Electronic Library Alexander Street Press Academic Video Online African American Music Reference American Civil War: Letters and Diaries American History in Video Anthropological Field Work Online Anthropology Online Art and Architecture in Video Asian American Drama BBC Video Collection Black Drama Black Studies in Video Border and Migration Studies Online Broadway HD Classical Music in Video Classical Music Library Classical Performance in Video Classical Scores Library Contemporary World Drama Counseling and Psychotherapy Transcripts: Volume I Counseling and Psychotherapy Transcripts: Volume II Counseling and Therapy in Video Dance Online: Dance in Video Dance Online: Dance Studies Collection Diagnosing Mental Disorders: DSM-5 and ICD- 10 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 19 Vendor Name Disability in the Modern World Drama Texts Collection Early Encounters in North America Education in Video Engineering Case Studies Online Environmental Issues Online Ethnographic Sound Archives Online Ethnographic Video Online Food Studies Online Gilded Age Global Issues Library Human Rights Studies Online Illustrated Civil War Newspapers and Magazines Images of America: A History of American Life in Images and Texts International Business Online LGBT Studies in Video LGBT Thought and Culture Music Online: Listening (United States) Music Periodicals of the 19th Century New World Cinema: Independent Features and Shorts (1990-present) North American Immigrant Letters, Diaries and Oral Histories North American Indian Thought and Culture North American Women's Drama North American Women's Letters and Diaries Nursing and Mental Health in Video: A Symptom Media Collection Nursing Education in Video PBS Video Collection Performance Design Archive Psychological Experiments Online Royal Shakespeare Company Collection Silent Film Online Sixties: Primary Document and Personal Narratives 1960– 1974 Social Theory Social Work Online Sony Pictures Classics INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 20 Vendor Name Theatre & Drama Premium Theatre in Context Theatre in Performance Theatre in Video: Volume I Theatre in Video: Volume II Twentieth Century Drama Underground & Independent Comic, Comix and Graphic Novels Women and Social Movements in the United States, 1600- 2000 World History in Video 60 Minutes: 1997–2014 American Institute of Physics Scitation Index SPIN American Mathematical Society MathSciNet APA (American Psychological Association) APA Books E-Collections ASM International ASM Handbooks Online ASME (American Society of Mechanical Engineer) ASME Digital Collection ASTM ASTM Standards & Engineering Digital Library BioOne BioOne Books 24x7 FinancePro ITPro Britannica Encyclopedia Britannica Online Spanish Reference Center Business Expert Press Business Expert Press Cabell's Cabell's Directory - Psychology Set Cabell's Directory - Educational Set Cambridge Crystallographic Data Centre Cambridge Structural Database (WebCSD) WebCSD Cambridge University Press Historical Statistics of the United States (HSUS) Chadwyck Healey Early English Books Online Black Abolitionist Papers Black Studies Center Black Studies Center: History Makers Module Early English Books Online Text Creation Project CLCD (Children's Literature Comprehensive Database) Children's Literature Comprehensive Database (CLCD) CQ Press CQ Researcher INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 21 Vendor Name Credo Reference Masterworks Credo Reference dataZoa dataZoa EBSCO Agricola Alt-HealthWatch America: History & Life (EBSCO) American Antiquarian Society (AAS) Historical Periodicals Collection (Series 1–5) American Doctoral Dissertations 1933–1955 Anthropology Plus Applied Science & Technology Abstracts Art Abstracts Art Full Text Art Index Retrospective ATLA (American Theological Library Association) Historical Monographs Collection: Series I ATLA (American Theological Library Association) Historical Monographs Collection: Series II Auto Repair Reference Center Biography Reference Bank Book Collection: Nonfiction Book Review Digest Plus Business Abstracts with Full Text Business Source Complete CINAHL Complete Communication & Mass Media Complete Computer Source: Consumer Edition Consumer Health Complete Criminal Justice Abstracts with Full Text eBook Collection (formerly NetLibrary) EBSCO Databases EconLit Education Full Text Ergonomics Abstracts ERIC (EBSCO) European Views of the Americas: 1493 to 1750 Fuente Academica General Science Full Text GeoRef INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 22 Vendor Name GeoRef in Process GreenFILE Health Source: Consumer Edition Health Source: Nursing/Academic Edition History Reference Center Humanities Full Text Library Literature & Information Science Full Text Library, Information Science & Technology Abstracts (LISTA) Literary Reference Center MedicLatina MEDLINE (EBSCO) Mental Measurements Yearbook with Tests in Print MLA Directory of Periodicals MLA International Bibliography Music Index Native American Archives Newspaper Source Plus Novelist Plus OmniFile Full Text Mega Philosopher's Index PsycARTICLES Psychology and Behavioral Sciences Collection PsycINFO PsycTESTS Readers' Guide Full Text Regional Business News Religion & Philosophy Collection RILM Abstracts of Music Literature Small Business Reference Center SmartSearch Social Sciences Full Text SPORTDiscus with Full Text Teacher Reference Center TOPICsearch Vocational & Career Collection Women's Studies International Academic Search Complete INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 23 Vendor Name Ei Engineering Village Compendex Elsevier ScienceDirect Clinical Pharmacology Scopus Gale 19th Century U.S. Newspapers 19th Century UK Periodicals Academic OneFile Archives Unbound Artemis Primary Sources British Literary Manuscripts Online Business Insights: Essentials Economist Historical Archive Educator's Reference Complete Eighteenth Century Collections Online (ECCO) Expanded Academic ASAP Gale Databases Gale Digital Collections Gale Virtual Reference Library General OneFile GREENR (Global Reference on the Environment, Energy, and Natural Resources) Health & Wellness Resource Center (with Alternative Health Module) Health Reference Center Academic Indigenous Peoples: North America Informe Academico InfoTrac Newsstand Kansas History, Territorial through Civil War Years, 1854– 1865 LegalTrac Literature Resource Center Making of the Modern World Nineteenth Century Collections Online (NCCO) Opposing Viewpoints In Context Sabin Americana, 1500–1926 Slavery and Anti-Slavery Collection Smithsonian Collections Online: Evolution of Flight 1784– 1991 Testing & Education Reference Center: TERC INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 24 Vendor Name Times (London) Google Google Scholar Guidestar Guidestar HathiTrust HathiTrust HeinOnline HeinOnline: Government, Politics and Law HeinOnline: Slavery in America and the World: History, Culture & Law IBISWorld IBISWorld IEEE IEEE - MIT Press eBooks LIbrary IEEE Xplore Digital Library IEEE-Wiley eBooks Library Infobase Learning Films On Demand Infogroup ReferenceUSA Institute of Physics IOPscience InterDok Directory of Published Proceedings JSTOR JSTOR Kanopy Kanopy Streaming Knovel Knovel LexisNexis LexisNexis Academic Nexis Uni Library of Congress Congress.gov (formerly THOMAS Legislative) Mergent Key Business Ratios Mergent Archives Mergent Intellect Mergent Online National Academies Press National Academies Press Publications National Library of Medicine PubMed (Medline) Naxos Naxos Music Library Naxos Sheet Music Library NCJRS National Criminal Justice Reference Service Abstracts Newsbank Access World News Newsbank OCLC ArchiveGrid ArticleFirst CAMIO: Catalog for Art Images Online Clase and Periodica ECO (Electronic Collections Online) FirstSearch INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 25 Vendor Name OAIster OCLC Electronic Books PapersFirst ProceedingsFirst WorldCat (OCLC) WorldCat Dissertations and Theses WorldCat.org Ovid OvidSP Oxford University Press Oxford Art Online Oxford English Dictionary Oxford History of Western Music Oxford Medicine Online Oxford Music Online Oxford Reference Online: Premium ProjectMUSE Project MUSE ProQuest ABI/INFORM Collection Aerospace Database Agricultural & Environmental Science Database American Periodicals Series (1741–1988) Annual Register (1758–2016) Art and Architecture Archive (1845–2005) Biological Science Database Chicago Defender (1910–1975) (ProQuest Historical Black Newspapers) Cleveland Call & Post (1934–1991) (ProQuest Historical Black Newspapers) ComDisDome Design and Applied Arts Index (DAAI) Digital National Security Archive (DNSA) Dissertations and Theses @ Wichita State University Earth, Atmospheric & Aquatic Science Database EBL- Ebook Library (Now Ebook Central) *EBook Central Ebrary (Now Ebook Central) ERIC (ProQuest) Fold3 Harper's Bazaar Archive HeritageQuest Online Literature Online (LION) INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 26 Vendor Name Los Angeles Sentinel (1934–2005) (ProQuest Historical Black Newspapers) Materials Science & Engineering Database MEDLINE (Proquest) National Criminal Justice Reference Service Abstracts (ProQuest) New York Amsterdam News (1922–1993) (ProQuest Historical Black Newspapers) New York Times (1851–3 years ago) with Index (1851– 1993) (ProQuest Historical Newspapers) New York Tribune/ Herald Tribune (1841–1962) (ProQuest Historical Newspapers) PAIS Index Periodicals Archive Online PILOTS Pittsburg Courier (1911–2002) (ProQuest Historical Black Newspapers) Pittsburg Post- Gazette (1786–2003) (ProQuest Historical Newspapers) ProQuest Civil War Era 1840–1865 ProQuest Congressional Publications (including Hearings) ProQuest Databases ProQuest Digital Microfilm ProQuest Historical Newspapers ProQuest History Vault ProQuest Nursing & Allied Health Source ProQuest Research Library Research Library, ProQuest SciTech Premium Collection Social Services Abstracts Sociological Abstracts Technology Collection The Christian Science Monitor (1908–1994) (ProQuest Historical Newspapers) The Guardian & The Observer (1791–1909) (ProQuest Historical Newspapers) Ulrichsweb.com Vogue Archive Women's Magazine Archive Collection 1: 1883–2005 Women's Magazine Archive Collection 2: 1846–2015 Readex African American Newspapers (1827–1998) INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 27 Vendor Name America's Historical Newspapers (1690–1922) American State Papers, 1789–1838 Early American Imprints Readex AllSearch Territorial Papers of the United States, Series 1 U.S. Congressional Serial Set, 1817–1994 SAGE SAGE Journals Online SAGE Reference Online SAGE Research Methods SAGE Research Methods Cases SAGE Stats Salem Press Salem History Salem Literature SBRnet Sports Market Analysis (formerly SBRnet) Springer SpringerLink State Library of Kansas Mango Languages Cloud Library Digital Books eLending Learning Express Library OneClick Digital Statista Statista Swank Swank Digital Campus Taylor & Francis CRC Press eBooks Europa World Year Book Thomson Reuters Arts & Humanities Citation Index MEDLINE (Web of Science) RIA Checkpoint Science Citation Index Social Sciences Citation Index Web of Science U.S. Department fo Commerce STAT-USA U.S. Census Bureau U.S. Department of Education ERIC U.S. Government Printing Office Catalog of U.S. Government Publications GPO Monthly Catalog Homeland Security Digital Library University of Chicago GSS (General Social Survey) INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 28 Vendor Name University of Michigan ICPSR (Inter-University Consortium for Political and Social Research) UpToDate UpToDate ValueLine ValueLine Investment Survey - Plus Wharton Research Data Services (WRDS) Compustat Eventus Wiley Cochrane Library Wiley Online Library INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 29 ENDNOTES 1 A. J. Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” Legal Reference Services Quarterly, 34, no. 2 (2000): 139, https://doi.org/10.1080/0270319X.2015.1048647. 2 Jennifer Horwath, “Evaluating Opportunities for Expanded Information Access: A Study of the Accessibility of Four Online Databases,” Library Hi Tech 20, no. 2 (2002): 199, https://doi.org/10.1108/07378830210432561. 3 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 140. 4 Horwath, “Evaluating Opportunities for Expanded Information Access: A Study of the Accessibility of Four Online Databases,” 199. 5 Sarah George, Ellie Clement, and Grace Hudson, “Auditing the Accessibility of Electronic Resources,” SCONUL Focus, 62 (2014): 16. 6 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 141. 7 Suzanne L. Byerley and Mary Beth Chambers, “Accessibility and Usability of Web-based Library Databases for Non-Visual Users,” Library Hi Tech, 20, no. 2 (2002) 177, https://doi.org/10.1108/07378831111116976; Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 140. 8 Kelly Dermody and Norda Majekodunmi, “Online Databases and the Research Experience for University Students with Print Disabilities,” Library Hi Tech 20, no. 1 (2011): 150, https://doi.org/10.1108/07378831111116976. 9 Dermody and Majekodunmi, “Online Databases and the Research Experience for University Students with Print Disabilities,” 156. 10 Dermody and Majekodunmi, “Online Databases and the Research Experience for University Students with Print Disabilities,” 156. 11 Dermody and Majekodunmi, “Online Databases and the Research Experience for University Students with Print Disabilities,” 156–7. 12 Dermody and Majekodunmi, “Online Databases and the Research Experience for University Students with Print Disabilities,” 151. 13 Dermody and Majekodunmi, “Online Databases and the Research Experience for University Students with Print Disabilities,” 144. 14 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 142. https://doi.org/10.1080/0270319X.2015.1048647 https://doi.org/10.1108/07378830210432561 https://doi.org/10.1108/07378831111116976 https://doi.org/10.1108/07378831111116976 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 FILLING THE GAP IN DATABASE USABILITY | WILLIS AND O’REILLY 30 15 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 139. 16 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 145. 17 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 138. 18 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 140. 19 Jonathan Lazar, Daniel F. Goldstein, and Anne Taylor, Ensuring Digital Accessibility through Process and Policy (Amsterdam: Morgan Kaufmann/Elsevier, 2015), 150. 20 Lazar, Goldstein, and Taylor, Ensuring Digital Accessibility through Process and Policy, 153. 21 Jennifer Tatomir and Joan C. Durrance, “Overcoming the Information Gap: Measuring the Accessibility of Library Databases to Adaptive Technology Users,” Library Hi Tech 28, no. 4 (2010): 581. 22 Tatomir and Durrance, “Overcoming the Information Gap: Measuring the Accessibility of Library Databases to Adaptive Technology Users,” 581. 23 Laura DeLancey, “Assessing the Accuracy of Vendor-supplied Accessibility Documentation,” Library Hi Tech 33, no. 1 (2015): 104, https://doi.org/10.1108/LHT-08-2014-0077. 24 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 168. 25 Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” 147. 26 Lazar, Goldstein, and Taylor, Ensuring Digital Accessibility through Process and Policy, 155. 27 Nondiscrimination on the Basis of Disability; Accessibility of Web Information and Services of State and Local Government Entities, 81 Fed. Reg. 28,658 (May 9, 2016) (to be codified at 28 CFR pt. 35). 28 Christina Mune and Ann Agee, “Are E-books for Everyone? An Evaluation of Academic E-book Platforms’ Accessibility Features,” Journal of Electronic Resources Librarianship 28, no. 3 (2016): 181, https://doi.org/10.1080/1941126X.2016.1200927. https://doi.org/10.1108/LHT-08-2014-0077 https://doi.org/10.1080/1941126X.2016.1200927 ABSTRACT Introduction Literature Review Methodology Research Findings VPAT Accuracy Assessment Accessibility Analysis Overall Vendor Comparison by Size Conclusion and Limitations Appendix A: Accessibility Remediation Guide Appendix B: Database by vendor list used in VPAT accuracy audit ENDNOTES