Thoughts on Refereed Articles
Refereed Articles Editor
Issues in Science and Technology Librarianship
bchoinski@istl.org
I have been the Refereed Articles Editor for ISTL since we started a refereed section in 2000. ISTL wouldn't have a refereed articles section if the Editorial Board had not been willing to take the risk. We started discussing the idea of having refereed articles in the middle of 1999. In an e-mail from June 9, 1999, we discussed the agenda for the Board's meeting at ALA in New Orleans. Andrea Duda wrote "Buffy asked about an agenda, and also whether there are any plans to make ISTL a peer reviewed journal." I don't recall being the person to introduce the topic to the board, but it looks like I did prod the discussion along. The Board gave me the support necessary to get the ball rolling and provided much needed advice and encouragement. Over the years, I have learned a few things and have some suggestions for authors and referees.
What I've Learned
I think that most of the submissions to the refereed section come from tenure-track academic librarians who have little experience with research or publishing. I try to keep that in mind when I'm writing up the evaluation of an article submission. I try to get my message across as directly as I can and I try to make my comments and my summary of the referees' comments as constructive as possible. I'm sure that some of my comments have seemed brusque. But I found that if I am not direct, then I will likely be misinterpreted. I have gotten very little negative feedback from authors about my comments, the referees' comments or the review process. I take that as a good sign!
Authors with little experience in publishing tend to be astonished at the copious amount of suggestions and corrections that referees provide. And some are even more astonished when they discover that I'm sending them a second round of revisions. I try to let them know that the object is to produce the best possible article that they can muster, that I'm sticking with it and they should too. Author abandonment in the middle of the revision process is a very rare occurrence.
I have learned that there are really only two fundamental problems that an article can have. The most common is poor writing. An article that has a story to tell but is poorly written can usually be saved. It may take a hugely heroic editorial effort (thank the referees), but it can be done. Sometimes the problems have to do with grammar and word choice. Sometimes the problem is lack of focus so acute that the reader can not figure out what the story is about. It's rare to have an article rejected solely for poor grammar or lack of focus, but it has happened. The other fundamental problem is when an article really doesn't have a story to tell. That is usually fatal. By "story," I mean that the author is not contributing information that we don't already know; for example, if the author has done no original research or is reiterating something that has been done or written about before. I am thankful that most of the submissions to ISTL do have a story to tell. I am extra thankful for those authors who can write really well. The ability to write well can pay off in more than one way. When it came time in 2010 to increase our pool of referees, applications from ISTL authors who had proven excellent writing ability were given top consideration.
When I send an article to two referees for review, I include a set of guidelines that ends with the following:
YOUR RECOMMENDATION:
The article
( ) is acceptable in present form or with minor revisions as indicated.
( ) should be reconsidered after major revisions as indicated.
( ) is unacceptable for the reasons indicated
I haven't checked, but I bet I could count on one hand the number of times that a referee has used the "unacceptable" option. Our referees hate to reject articles. I'm not exactly thrilled with rejecting either. It used to bother me a lot when referees would mark the "should be reconsidered" option when an article clearly was unacceptable. But I understand the impulse to be kind. And I've come to realize that I really have to study the substance of their comments to determine the extent and nature of the referees' suggestions. I try to be mindful of the author's feelings when drafting an e-mail that lets an author down. It is difficult and I've probably botched it a few times. For that, I'm sorry.
I love it when both referees mark "acceptable in present form..." That makes my job easy and the author very happy. When an author replies with "you've made my day," it makes my day! I've made peace with the middle option (reconsider after major revisions). It gives me flexibility in my decisions and helps me to focus on whether the problems with the article are of the fixable, story-telling variety or if something more substantial is the problem.
On rare occasions, I've received two reviews with completely divergent opinions. It makes it seem like the referees were not even reading the same article. When opinions on an article are that different, it is usually because the article is lacking focus or tries to tackle too many questions at once. I struggle with figuring out what to tell the author about what next steps to take.
What I want future authors to know
Almost nothing is nearer and dearer to authors than their figures, charts, and graphs. These usually represent a considerable amount of effort to create. If I delete a figure from an article, it is because I know it is superfluous. Remember the adage that a picture is worth a thousand words? I think a picture (figure, graph, chart) needs to be worth at least 100 words. If the contents of the figure are also given in the text in just a sentence or two, then it is not worth distracting the author from the text. Here's an example from an article submission reporting on the percent of survey respondents who answered a yes/no question. The author wrote in the text that "80% of the respondents said yes and 20% said no (see Figure 1)".
Figure 1
In this case, I deleted the figure because it gave absolutely no more information than what was said in the text. Figures should explain something about the data that cannot easily be visualized or understood from reading the same information. Figures need to add value. If they don't, they're out! So what do you think of my inclusion of this figure to illustrate my point? Is it superfluous? I think it is.
Reporting numerical results is an area in which some authors get carried away with decimal points. For example, an author might report in a bibliometric study that a particular journal is cited on average 13.472 times per year. This implies that the citation rate is known to the nearest thousandth, a level of precision that was not measured and does not make sense. I recommend numerical results to be reported in the same units as the data were collected. It has the added benefit of making results easier to read.
Authors could cut down on the amount of time potentially spent in revision if they would have one or two trusted colleagues read and comment on their work before submitting it. I recommend finding someone whose articles you've enjoyed reading. If the author has produced something that is interesting and well written, then that author may be able to give you some good advice. And to those who have been successful in publishing, please reach out to those who are new to the process. A little mentoring can go a long way in helping them get their articles in shape for submission.
Other advice I can offer is to always read our author guidelines before submitting, be patient and open-minded, and never use the words "utilize" or "methodology". That is a pet peeve of mine and I will make you change them to "use" or "method". If you have any questions at all before submitting, don't be afraid to ask!
The Numbers
ISTL is a low volume journal (thank goodness because I have a day job). These figures show the number of submissions to the refereed articles section and the percent accepted (data for 2011 is through June).
Thank you
Putting together the refereed section of ISTL is a group effort. I owe a great big thank you to the many referees who have served ISTL since 2000. Every referee has shown the utmost in professionalism. Reviews are usually returned to me within the specified time and with a thoughtfulness and dedication that I have come to depend on. I continue to be impressed with how thorough most reviews are and how the referees take pains to make their comments as constructive as possible. Thank you!
I appreciate the continued opportunity to edit the section and I welcome all comments and suggestions for improvement. I take my commitment seriously and I thoroughly enjoy working with authors to make ISTL a highly respected publication. Thank you to the Board, our authors, and all of you who read ISTL.
ISTL's editor Andrea Duda deserves special recognition for her support and her dedication. She is the engine that keeps us running. Thank you Andrea!
