564 Editorial The Signaling Value of Peer Review It is an exciting (and scary) time to be an editor and involved in authorship and pub- lishing. There are so many new developments and initiatives that push the traditional boundaries of knowledge and information. The web is very democratic—Anyone can publish anything. This may take the form of digital scholarship, on one hand—or alternative facts, on the other. That dichotomy highlights the importance of libraries and librarians—that we have both the mission and the expertise to help filter all of the noise from what is published and identify relevant and reputable information for library users. Furthermore, because information is so ubiquitous, there is a growing need to educate and empower users of information to be able to effectively find, evaluate, and use information. Scholars in all disciplines also have a responsibility to help “filter the noise” and evaluate new scholarship for purposes of developing their discipline, contributing to new knowledge, advancing technology and practice, and informing and educating communities. Scholarly communities have traditionally recognized innovative and valued discoveries through peer review—peer reviewed articles, scholarly mono- graphs, competitive grants, etc. However, it is difficult to ignore how the peer review process is receiving more scrutiny and criticism of late. As we are examining peer review throughout the editorials this year, I wanted to offer some thoughts on what it means and the role it plays. This may seem like a non sequitur but it puts me in mind of the trust invested in higher education to adjudicate Title IX complaints. The federal government recognizes that colleges and universities are distinctive communities which individuals self-select to become part of—as such, they have implicitly agreed to adhere to a higher standard of behavior. “In the field of discipline, scholastic and behavioral, an institution may establish any standards reasonably relevant to the lawful missions, processes, and func- tions of the institution” (from the General Order on Judicial Standards of Procedure and Substance in Review of Student Discipline in Tax Supported Institutions of Higher Education.) This statement acknowledges that academic institutions may create their own stan- dards and priorities. In addition, members of this community, students, and faculty, also commit to upholding the education mission and the creation and dissemination of knowledge. Dedicated to critical inquiry and innovation, faculty and researchers also achieve expertise with the scope of their discipline. Reviewers or Gatekeepers? Academic communities—within and across disciplines, and within specific colleges, universities or departments—have long been recognized as setting the standards within their respective fields. It is expected that faculty and researchers are responsible for educating the next generations within a discipline but that they also steward the innovation and creation of new knowledge, contributing to scholarship and practice. Peer reviewers are in a position to be curators of this scholarship in their disciplines— as such, they should be balanced and objective in how they review, applying their doi:10.5860/crl.78.5.564 http://10.5860/crl Editorial 565 expertise for the benefit of the discipline and the advancement of knowledge while also mentoring the next generation of scholars and researchers to sustain the disci- pline into the future. Some of the issue with peer review may be the concerns around “peer”—that those chosen to evaluate may be more concerned with protecting their own reputation or research agenda, or that they may be entrenched in the status quo and thus more likely to dismiss any discovery or research that threatens it. In other words, there is concern that reviewers (and possibly editors, for that matter) are trying to make the discipline into their own image. The nature of filtering or selecting information is one that seems to engender mistrust—hence, the push for a more transparent process. Interestingly (and not wrongly), there is some perception that scholarship is an “old boy network” (which, looking at demographics, may be more true in some disciplines than others). Or that peer review is just a popularity contest. One of the arguments that I have heard from authors and researchers is that the blind is not real and that “everyone” (i.e., the reviewers) has a good idea who the author of a manuscript is. This may be true in some circumstances—recently, I had a reviewer decline to review because she was certain the author was someone she was mentoring and she very graciously let me know. We removed her as the reviewer and invited someone else. Ideally, that is how the system should work—that a reviewer would recuse themselves if the individual author was someone who was likely known. The ethical behavior and respect for the objectivity of blind review has a lot of value in the process. Can it be abused or ignored? Of course, so can any such process. The hope is that the community of scholars and reviewers all value this ethic and the process enough to maintain its integrity. The other argument I hear about peer review is somewhat related to the issue of scholarly traditions being held dearly. In other words, it may appear that reviewers themselves are entrenched in the profession and may perpetuate the traditional values rather than recognizing research efforts that challenge the paradigm or redefine what scholarship is. This may be particularly problematic for interdisciplinary or practical disciplines. In the context of a journal, the editor and editorial board (or the equivalent for other venues) would make efforts to have broad representation and be responsive to how a discipline is evolving. The leadership in ACRL has been having discussions about how to have more in- clusion and representation in its committee service. As editor, this is also something that I looked at when it came time to recommend additional members of the Editorial Board, based on the list of ACRL members who volunteered. In reviewing volunteers, I primarily examined three criteria: 1. Has the individual had any service (as a reviewer) with C&RL of another ACRL publication? This demonstrates some existing commitment to the journal. If not, did they have some evidence of engagement with ACRL or ALA? This might broaden to other communities of practice in an effort to identify some needed expertise or a new perspective. 2. Does the individual have a record of research and publication? This does not necessarily have to be in C&RL but there does need to be some experience, as an author, with the peer review publication process. The number of publications does not need to be extensive but it does need to adequately indicate that the individual has a consistent research record. 3. Does the individual bring an expertise or perspective that is not already repre- sented on the Board? This may include type of institution, area of specialization, engagement in new types of research, or activity in an area that has been largely underserved. 566 College & Research Libraries July 2017 Peer Review, Writ Broadly Some of the concern about peer review may be that it is viewed as a process particular to scholarly journals. Peer review does not have to be limited to the traditional schol- arly journal—the role of recognized experts is critical when evaluating information, no matter what venue or purpose. Context may impact what peer review may look like—particularly for fields that have a more practical aspect, like librarianship. In that case, the appropriate expert may have a grounding in practice or have significant experience. PRIMO provides a good example of what this might look like, as does the Library Support Staff certification. Both make use of peer reviewers or evaluators to assess what quality in performance looks like. Peer review shows up all over higher education. It is being widely used in the class- room (including online or distance education) with calibrated peer review as both a feedback and training tool. The process for reviewing a dissertation (via departmental review committee) is also a type of peer review though not a completely objective process as departments have a stake in the outcome. It is common for dissertations to be rewritten prior to submission to an academic press—many are never even consid- ered for publication. In addition, for purposes of tenure or promotion, the dissertation does not usually stand-alone: departments (and external peer reviewers) look for the objective assessment and outside publication of the research as indicators of quality. The issue of peer review has also come up with regard to open educational resources. Academic freedom dictates that teaching faculty have the latitude to decide how they teach and what resources they choose to use in the classroom. Some faculty may develop their own materials or choose to use a published textbook (which, ironically, are often authored by other faculty in the discipline). Similar to the concerns around scholarly literature, publishers take material created by academics, process and repackage it, and sell it to the students of academics. It is easy to see the issue and be frustrated with the publishing costs. As with open access, I support the idea of open educational resources—it just makes sense—but having an external review or an objective indicator of quality is also critical. We have all seen notable and recent examples of ways in which peer reviewed journals have made missteps that call their standards or their processes into question. These shortcomings are acknowledged in the retractions of the articles: • Wakefield, Andrew J., Simon H. Murch, Andrew Anthony, John Linnell, D. M. Casson, Mohsin Malik, Mark Berelowitz et al. “RETRACTED: Ileal-lymphoid- nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children.” (1998): 637–641. • Fleischmann, Martin, Stanley Pons, Mark W. Anderson, Lian Jun Li, and Marvin Hawkins. “Calorimetry of the palladium-deuterium-heavy water system.” Journal of Electroanalytical Chemistry and Interfacial Electrochemistry 287, no. 2 (1990): 293–348. • LaCour, Michael J., and Donald P. Green. “When contact changes minds: An experiment on transmission of support for gay equality.” Science 346, no. 6215 (2014): 1366–1369. However, it is interesting to note that these instances were pointed out by other scholars—by peers within the field, indicating that the community of scholars com- mendably polices its own. It does not mean that traditional peer review is the end-all, be-all but I believe strongly that communities of scholars, researchers and professionals can work together to help sift through all of the research to indicate what is valuable and impactful and to show us what quality looks like. These communities of experts also have the opportunity to evolve scholarship and knowledge into new venues and media so it contributes in new ways. Wendi Arant Kaspar Texas A&M University