Evidence
Based Library and Information Practice
Evidence Summary
PubMed’s Native Interface Remains the Best Tool for Systematic Searching
of its Biomedical Citations
A Review of:
Wildgaard, L. E., & Lund, H. (2016). Advancing PubMed? A comparison
of third-party PubMed/Medline tools. Library Hi Tech, 34 (4), 669-684. http://dx.doi.org/doi: 10.1108/LHT-06-2016-0066
Reviewed by:
Ann Glusker
Research & Data Coordinator
National Network of Libraries of Medicine, Pacific
Northwest Region
University of Washington Health Sciences Library
Seattle, Washington, United States of America
Email: glusker@uw.edu
Received: 1 Mar. 2017 Accepted:
17 Apr. 2017
2017 Glusker.
This is an Open Access article distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Abstract
Objective – To compare the functionality of
third-party PubMed tools for searching biomedical citations in PubMed, in the
specific context of systematic searching.
Design – Comparative analysis of software
functionality.
Setting – Online, freely accessible search
software.
Subjects – Sixteen
third-party tools for searching and managing the full range of PubMed citations
(tools which focused on specific disciplines were not included).
Methods – Tools for
analysis were identified in two ways; those discussed in two published articles
were used, and a supplementary PubMed search was performed. The initial list of
76 possibilities was assessed for study inclusion on 4 criteria: covering the
entire range of PubMed content; being freely available; not limiting to a
particular bio-medical discipline; and incorporating online PubMed/MEDLINE
content. After assessment, 16 tools were chosen for further analysis (the
authors provide a list and description of the tools in their Table I). Each was
examined in relation to 11 crucial operational aspects. Result sets were tested
against a control (a literature search result set on a particular clinical
question which was determined by physicians to yield relevant results, details
of which are provided by the authors in an online appendix).
Main
Results – The 11 identified aspects related to tool functionality were examined
for each tool selected, with results grouped into three sets of factors: 1)
supporting the search (field codes, filters, limits and Boolean operators); 2)
managing the search (output, related articles, links to articles, number of
results, exporting); and 3) documenting the search (saving the search and
search history). In some cases, the tests had to be adjusted to accommodate the
tool's specifications. In Table II the authors present a grid with the results
of the testing, on each of the 11 aspects, for each tool.
The authors found that with many tools it was not straightforward, if
even possible, to filter and limit in order to get more specific result sets.
Few tools were effective at suggesting related articles within the tool itself,
instead linking the user out to PubMed, and only two tools provided the same
number of citation results as the comparison PubMed search. In addition, the
display of results often made it difficult to assess result sets; and only two
tools provided the option to save searches and see search history. Furthermore,
due to unexpected tool limitations, it was not possible to assess the relevance
of citation result sets delivered by the third-party tools, as compared with
the control PubMed search.
Conclusion – Close
analysis of the tools studied indicated that they were not created in order to
support systematic searches. They lack support for filtering/limiting, saving
or exporting searches, which are central functionalities to the work of
performing such searches. While some of the tools studied may still be in the
early phases of development, and while several of them, in enhancing PubMed
searches in particular ways, may suggest additional profitable strategies for
performing a systematic search, not one of them can replace the functionalities
of the native PubMed interface. It remains the best tool for searching and
managing the full range of PubMed citations, for the purposes of performing
systematic searches.
Commentary
This study was an addition to existing literature – specifically
articles the authors consulted by Lu (2011) and Keepanasseril (2014), which merely
listed and described tools – in that the authors analyzed the functionality of
tools using a detailed set of criteria and a validated search as a control.
While it turned out that the third-party tools examined are not suited to use
for systematic searching, they may be useful for other search enhancements. The
authors state that these tools “are beneficial as they give immediate, dynamic
visual assessment of relationships between authors, topics and term
hierarchies, etc. in the bibliographic data, giving a strong starting point in
evaluating and selecting literature to include in a systematic search” (p.
679). For example, in 2010 Kristine Ogden outlined several aspects of the tools
HubMed and Quertle which helped her in clinical searches: a citation finder
which pulls PubMed records for citations in a bibliography; the ability to run
a PubMed search automatically on other sites such as GoogleScholar; the use of
natural language to find relevant citations; and separate tabs for keyword
search results and citations. She also appreciated the clean user interfaces of
these tools.
Furthermore, this paper makes an important contribution toward
supporting medical librarians and others who work with systematic reviewers, in
showing the crucial importance of the systematic searching that underlies such
reviews. It also gives librarians a framework for helping systematic reviewers
assess third-party tools to help with those searches.
For this evidence summary, methodologies were systematically assessed
using Glynn’s critical appraisal checklist (2006). The checklist was designed
to evaluate population-based studies, and so some of its criteria did not apply
to this study, but it does focus on freedom from bias and representativeness of
the subjects studied. In this instance, one question is whether there are more
effective third-party tools for systematic searching which may not be freely
available, but rather exist behind pay walls – could including them have
changed the results? Also, there may be excellent tools that were designed for
specific clinical areas which were excluded from this study. In Table III the
authors list excluded third-party tools, which could prove a resource for
future analysis.
Furthermore, in addition to listing closed projects and dead links,
Table IV lists potentially relevant third-party tools and sites under
construction. The authors are not denigrating the third-party tools they tested
(in fact, they mentioned wanting to re-test them, and entries in Table IV may
be a starting place). Future methodological advances may contribute to the
creation of systematic PubMed search tools. As described in Gonzalez et al.
“computational methods contribute…by bringing knowledge from literature, either
extracted or curated, together with high-throughput data sets, to identify both
known and new relationships” (2016, p. 39). While the context for such
computational methods relates to text and data mining, there is every reason to
expect that they may eventually contribute to systematic analysis of PubMed
citations such as Wildgaard and Lund seek.
References
Glynn, L. (2006). A critical appraisal tool for library and information
research. Library Hi Tech, 24(3), 387-399. http://dx.doi.org/10.1108/07378830610692154
Gonzalez, G. H., Tahsin, T., Goodale, B. C., Greene, A. C., &
Greene, C. S. (2016). Recent advances and emerging applications in text and
data mining for biomedical discovery. Briefings
in Bioinformatics, 17(1), 33-42. https://dx.doi.org/10.1093/bib/bbv087
Keepanasseril, A. (2014). PubMed alternatives to search MEDLINE: An
environmental scan. Indian Journal of Dental Research, 25(4), 527o. http://dx.doi.org/doi:10.4103/0970-9290.142562
Lu, Z. (2011). PubMed and beyond: A survey of web tools for searching
biomedical literature. Database: The Journal of Biological Databases and
Curation, baq036. http://dx.doi.org/doi: 10.1093/database/baq036
Ogden, K. (2010, March 16). Why use a third-party tool to interact with
PubMed? Retrieved from https://nnlm.gov/pnr/dragonfly/2010/03/16/hubmed_quertle