College and Research Libraries A Reputational Study of Academic Publishers Paul Metz and John Stemmer In both selecting individual titles and designing gathering plans, collec- tion d~velopment librarians are strongly influenced by the perceptions they have about publishers. In the near absence of data that might indi- cate the overall perceptions the collection development community has about academic publishers, the authors distributed a reputational as- sessment survey to a national sample of heads of collection develop- ment in academic libraries. The resulting data on perceptions of the quality and academic relevance of selected publishers' monographs are reported and analyzed. lthough the selection of books is only one among the increas- ingly long and varied list of functions that make up collec- tion development, it is still, to a consider- able degree, the defining task of collec- tion development. The image of a bibli- ographer alone in an office cluttered with reviews, approval slips, publisher flyers, catalogs, and bibliographies remains a paradigm of the collection development craft. The importance of book selection is much more than symbolic. Despite the documented increase in the serials com- ponent of research libraries' materials budgets and despite the growth in elec- tronic databases, online services, and video and document-delivery services that compete with books for collections budgets, college and university libraries still spend hundreds of millions of dol- lars annually on books. Each book acquired by an academic library represents the outcome of a deci- sion. Often the decision will be a micro one: this book by this author is the right book for us to acquire in support of our programs. Often, and increasingly, the book will be chosen consequent to a macro decision, such as the addition of a publisher or subject area to an approval plan profile or the initiation of a blanket order or membership. 1 In both the macro and micro decisions that build monographic collections, a few criteria are generally decisive. Leaving aside ancillary criteria such as the book's relationship to the existing collection, the three overriding issues generally are the relevance of the book to the institution's mission and goals; the presumed quality of the book; and, given the desirability of the title on these dimensions, its price and the question of whether the selection Paul Metz is Principal Bibliographer, University Libraries, Virginia Polytechnic Institute and State Uni- versity-Blacksburg; e-mail: pmetz@vt.edu. John Stemmer is Collegiate Librarian for Arts and Sciences, University Libraries, Virginia Polytechnic Institute and State University -Blacksburg; e-mail: jstemmer@vt.edu. 234 A Reputational Study o"f Academic Publishers 235 would represent an efficient expenditure of the library's limited resources. It is important to recognize that in the great majority of cases, it is not the book itself but some surrogate that is being as- sessed against these criteria. If the bibli- ographer is lucky, the surrogate will be a thoughtful review in a trusted source such as the New York Times Book Review at the time of the book's publication. Far more often, the surrogate will be an ·extract of basic cataloging information encountered on a decision slip or in The American Book Publishing Record. Relevance and quality in this case must be assessed in highly constrained and subjective ways. This calls for an on-the- spot exercise of experienced judgment within highly bounded rationality.2 The book's title, the subject scope typically associated with the publisher, and, on occasion, some knowledge about the au- thor have to serve as the basis for rel- evance judgments. There is even less ba- sis for imputing quality, because in this respect the title is rarely helpful. Because even the best bibliographers can be expected to be familiar with only a minority of authors in their fields, and because titles provide limited informa- tion, knowledge of the publisher often furnishes the decisive element in selec- tion decisions. Unfortunately, this consid- eration raises the further nonobvious question of how much bibliographers know about publishers, how they acquire that knowledge, and how much confi- dence they can have in their judgments. It is likely that most bibliographers' im- pressions of most publishers represent an amalgam of conscious conclusions and much more visceral impressions that have been gathered over years of academic training, personal reading, discussions with academic faculty and other librar- ians, inspection of library receipts, and use of book reviews. If bibliographers look to the library lit- erature for guidance on this topic, they will find little beyond two articles, the second a ten-year replication and expan- sion of the first. John Calhoun and James K. Bracken's brief 1983 Research Notes in College & Research Libraries reported the number of titles, number of outstanding book awards, and the ratio of awards to titles for the sixty publishers most often winning Choice Outstanding Academic Book awards (OABs) for the years 1977- 1981. Calhoun and Bracken made their data easily interpretable by normalizing the ratios to a fixed benchmark of 1.0, rep- resenting a ratio of awards to titles equivalent to the proportion of awards won by the "Oxbridge" (Oxford and Cambridge) ~versity presses. Of their sixty publishers, twenty were university presses.3 In 1993, Edward Goedeken published a partial replication of Calhoun and Bracken's study, based on Choice data for the years 1988-1992. He also listed the then prevailing top sixty OAB publish- Because even the best bibliographers can be expected to be familiar with only a minority of authors in their fields, ... knowledge of the pub- lisher often furnishes the decisive element in selection decisions. ers. The university press component had, by 1993, grown to twenty-seven presses, which accounted for 48 percent of awards, up from 33 percent for the ten-year pe- riod of the first study. In his discussion, Goedeken made the unsurprising obser- vation that Calhoun and Bracken's asser- tion that their ratios represented "a mea- surement of publisher quality" had gen- erated controversy based, in part, on the validity of comparisons between the book lists of university presses and trade pub- lishers.4 Although relevance and quality are plainly subjective attributes about which no one would expect to find scientifically validated or conclusive data, the impor- tance to librarians of having some reliable 236 College & Research Libraries basis for publisher judgments contrasts strikingly with the paucity of any sort of information on the topic. The authors con- ceived the present study to address this problem. Methodological Issues This study takes as its premise that both the subject scope that might be associated with a given publisher and the quality that might be attributed to its books rep- resent inherently subjective judgments (which is no more than might be said of book reviews or book awards). It follows that the best way to study the topic is to admit this constraint up front and to at- tempt to assess as directly as possible the perceptions and opinions of informed observers. From a library point of view, a logical set of observers would be heads of col- lection development, and so a reputa- tional survey was sent to these individu- als. To capture the range of higher educa- tion and the variety of collection devel- opment practices, the authors defined the universe of potential respondents as the chief collection development officers at all ARL member institutions, and at the Oberlin group (Obergroup) institutions, which comprise seventy-two liberal arts colleges in the United States. The nominal heads of collection devel- opment at ARL libraries and the directors at Obergroup institutions seemed to be the most likely respondents, but the wide variety of organizational schemes and job titles makes it difficult in many cases to identify this person. Accordingly, respon- dents were given the following instruc- tions so that the questionnaire could be referred at each institution to the person most able to respond: Before completing the survey, please take a minute to make sure you are the appropriate respondent, according to the following definition: The respondent should be the librarian most intimately respon- May 1996 sible for building the library col- lection in all areas. • In ARL libraries, this will of- ten be an associate or assistant di- rector. However, if such an indivi- dual's duties combine collection de- velopment and other major func- tional areas, and if there exists a full- time subordinate responsible for collection development in all areas, the latter should respond. • In the Oberlin group, each di- rector should designate the indi- vidual (potentially her- or himself) most responsible for overall collec- tion development. If you do not match this defini- tion, please forward the question- naire to the appropriate respondent for your library. Several respondents to our first mailing noted that they found it necessary to consult with colleagues more familiar than them- selves with certain publishers or dis- ciplines. This is fine, though we ask that one person attempt to ensure consistent application of the rank- ing scales. The final two sentences of the instruc- tions were included only in the follow- up, but were consistent with advice the authors had given in response to inquir- ies from first-round respondents. After some reflection, and with mixed advice, the authors decided to ask respon- dents to report their perceptions on the relevance of each publisher's output to local collecting programs and on pub- lisher quality, but not price. The rationale for this exclusion rested on several con- siderations. To ask about more param- eters logically would require a reduction in the number of publishers that could be included within a limited imposition on the respondents' time and courtesy. Ac- cepting this constraint, the authors de- cided to focus on the two dimensions for which there are, and can be, no objective data. Price can generally be determined A Reputational Study of Academic Publishers 237 on an individual title basis from the no- tice that generates the original order, and on the macro basis, it can generally be assessed on a per publisher basis from approval vendors' reports and controlled by dollar limitations on automatic book receipt within the parameters of a ven- dor profile. The remaining basic decision, after the respondent universe and the key ques- tions to ask had been determined, was which publishers to include. A broad rep- resentation of the varying types of aca- demically relevant publishers, with inclu- sion of the larger houses, seemed ideal. The actual selection was based on the fol- lowing criteria: • So that the larger houses would be included regardless of academic orienta- tion, the top twenty publishers in book volume as measured by the spring 1994 edition of Yankee Book Peddler's "annual roundup" of publisher data were in- cluded. • To assess all university presses would place undue emphasis on this pub- lishing group, especially because it is of- ten the trade publishers about whom se- lectors have the most questions. But the authors did want to see how, in the per- ception of collection develop- ment officers, the range of uni- ties, or the social sciences. Most of these were well established and well known, but the New Press was included partly out of curiosity about whether collection officers would be able to report their per- ceptions about it. • The authors included a few publish- ers on the basis of their awareness that these publishers were either controversial or the focus of strong opinions within the collection development community. Given the exclusion of price, the three categories about which respondents were asked to indicate their perceptions of each publisher were familiarity, relevance, and quality. The first, familiarity, was of in- terest in that one would like to know which publishers are best known to the collection development community. The question was also asked to make it pos- sible to ensure that the opinions of only those individuals who had some knowl- edge, even if self-reported, were consid- ered in evaluating a publisher. The other two factors, relevance and quality, were the major variables of substantive inter- est in the study. The definitions used in the questionnaire, as defined in the instru- ment itself, were: Familiarity: The degree (little/none, moderate, high) to which you feel famil- TABLEt versity presses and the range of academic publishers would com- pare when superimposed. Ac- cordingly, the authors selected six university presses: Oxford and Cambridge because of their repu- tations, high volume, and previ- ous use as benchmark publishers in the OAB studies; Harvard and Stanford to represent the elite of American university presses; and SUNY and Oklahoma to repre- sent the general body of Ameri- can university presses. Distribution of Respondents' Titles • The authors selected other publishers representing impor- tant houses associated with sci- ence and technology, the humani- 24.4% 14.2 13.4 8.7 7.1 4.7 2.4 1.6 0.8 22.8 Collection Development Librarian Library Director Associate Director for Collection Development Collection Development Coordinator Acquisitions Librarian Acquisitions/Collection Development Librarian Technical Services/Collection Development Librarian Head of Reference & Chair of Collection Development Committee Assistant Head of Public Services for BI & Collection Development Miscellaneous titles 238 College & Research Libraries TABLE2 Distribution of Supervisors' Titles 61.1% 11.1 4.0 Library Director Provost Assistant Director for Technical May 1996 January 15,1995. TheARLresponse rate was about 67 percent and that of the Obergroup was 75 percent. The responding institutions came from all over the United States and Canada. 4.0 Services & Collection Management Associate University Librarian for Public Services & Collection Development The authors gathered a certain amount of institutional information (such as affiliation [public or pri- vate], materials budget, Barron's se- lectivity rating, and whether each in- stitution offered the doctorate in English, electrical engineering, both, 2.4 Associate Dean for Collections & Services Associate Director 2.4 2.4 12.7 Director for Collection Development Miscellaneous titles or neither) from various reports and reference sources. Comparing re- sponding to nonresponding institu- tions on these parameters, the au- iar with a publisher and capable of com- menting on its books. Please skip there- maining two questions for any publisher for which you rate your familiarity as little/none. (In the actual analysis all rel- evance and quality scores for respondents who reported little or no familiarity for a publisher but who disregarded this in- struction were converted to N I A). Relevance: The degree to which a publisher's book titles address topics of interest to the academic community and the extent to which established modes of scholarly or scientific discourse guide the presentation of material. Quality: The overall intellectual and editorial quality of a publisher's mono- graphic offerings, reflecting the expertise of typical authors; the persuasiveness of evidence; the intellectual level of dis- course; the tendency of a publisher's titles to be influential in their fields; and the degree of editorial care. The questionnaire offered respon- dents a five-point Likert scale ranging from "very low" to "very high" for re- sponses on the relevance and quality items. The total response rate to the ques- tionnaire was 70.7 percent, yielding a to- tal of 128 usable responses. The replies were in response to two waves of the sur- vey, the first on November 1,1994, with follow-up to initial nonrespondents on thors found no response bias, even at a criterion of p < .20. The survey devoted a number of ques- tionnaire items to basic background in- formation about the r~spondents them- selves and their supervisors. To determine the experience level of the respondents, they were asked how many years they had been in collection development and in their current positions. Those who an- swered the survey had an average of a little over 16.5 years in collection devel- opment. The average length of time in their current position was 7.2 years. Table 1 reports the most frequent titles of re- spondents. As indicated in table 2, the great majority of respondents reported to library directors or other senior librarians. 96.0% 32.0 17.2 3.1 5.5 13.3 4.7 0.8 1.6 TABLE3 Educational Attainment of Respondents Master's degree in library science Master's in a humanities field Master's in a social sciences field Master's in a sciences field Professional degree Ph.D. in a humanities field Ph.D. in a social sciences field Ph.D. in a sciences field Ed.D. A Reputational Study of Academic Publishers 239 TABLE4 Perce~tions of Academic Publisher Familiaritll Relevance~ and Quality Publisher Famili}uity Publisher Relevance Publisher Quality Cambridge 2.96 Cambridge 4.83 Harvard 4.82 Oxford 2.94 Harvard 4.82 Cambridge 4.80 Gale 2.89 Oxford 4.81 Oxford 4.77 Harvard 2.87 Stanford 4.60 Stanford 4.51 Garland 2.80 National Acad. 4.52 National Acad. 4.44 Routledge 2.76 Elsevier 4.47 Brookings 4.37 Blackwell 2.75 Springer-Verlag 4.46 Springer-Verlag 4.34 Greenwood 2.75 Brookings 4.41 Blackwell 4.25 St. Martins 2.74 CRC 4.38 Smithsonian 4.24 Random House 2.72 Blackwell 4.37 Elsevier 4.15 Knopf 2.70 Kluwer 4.30 Wiley 4.15 Simon & Schuster 2.69 Wiley 4.26 CRC 4.08 Stanford 2.67 SUNY 4.24 Kluwer 4.07 McGraw-Hill 2.67 Academic 4.22 Academic 4.03 Wiley 2.66 . Routledge 4.17 Oklahoma 4.00 Macmillan 2.66 Sage 4.12 Routledge 3.97 Sage 2.65 Greenwood 4.11 Knopf 3.96 Springer-Verlag 2.64 Oklahoma 4.06 Farrar, Straus 3.96 Houghton Mifflin 2.64 Plenum 4.05 Norton 3.90 Prentice Hall 2.64 Gale 4.01 Basic 3.90 Elsevier 2.63 Westview 4.00 New Directions 3.90 HarperCollins 2.63 VCH 3.98 VCH 3.88 Smithsonian 2.63 Garland 3.98 SUNY 3.86 Brookings 2.60 Erlbaum 3.95 Plenum 3.84 Norton 2.60 Humanities 3.94 St. Martins 3.83 Penguin 2.59 Jossey-Bass 3.88 Allen & Unwin 3.81 Praeger 2.58 Basic 3.87 Free Press 3.77 Doubleday 2.57 Praeger 3.86 Sage 3.77 Free Press 2.52 Smithsonian 3.86 Erlbaum 3.76 Haworth 2.52 World Scientific 3.80 Macmillan 3.69 UPA 2.51 M. Dekker 3.79 Praeger 3.68 Farrar, Straus 2.51 St. Martins 3.77 New Press 3.68 Basic 2.50 ME Sharpe 3.76 Chapman & Hall 3.68 Academic 2.50 Allen & Unwin 3.76 Penguin 3.67 Jossey-Bass 2.48 Free Press 3.75 Jossey-Bass 3.65 SUNY 2.48 Chapman & Hall 3.73 Gale 3.65 Viking 2.48 New Directions 3.71 Random House 3.64 Westview 2.46 Transaction 3.67 Humanities 3.62 Oklahoma 2.33 Norton 3.63 Van Nostrand 3.62 Plenum 2.29 Van Nostrand 3.58 Westview 3.62 CRC 2.27 E. Arnold 3.55 M. Dekker 3.61 Kluwer 2.26 Macmillan 3.55 Atlantic Monthly 3.60 Addison-Wesley 2.26 UPA 3.55 Viking 3.59 E. Mellen 2.26 Addison-Wesley 3.54 Addison-Wesley 3.58 N atiomil A cad. 2.21 New Press 3.50 McGraw-Hill 3.56 Van Nostrand 2.21 Lexington 3.48 Transaction 3.56 (Cont. on next page) 240 College & Research Libraries May 1996 TABLE 4 cont. Perce~tions of Academic Publisher Familiarit,L Relevance~ and Quality Publisher Familiarity Publisher Relevance Publisher Quality Humanities 2.19 Guilford 3.48 Houghton Mifflin 3.55 Times 2.16 Ash gate 3.46 E. Arnold 3.51 Atlantic Monthly 2.13 Penguin 3.45 Greenwood 3.50 Allen & Unwin 2.13 Knopf 3.40 ME Sharpe 3.49 Erlbaum 2.10 McGraw-Hill 3.40 Simon & Schuster 3.45 ME Sharpe 2.09 Farrar, Straus 3.38 Prentice Hall 3.45 New Directions 2.09 Prentice Hall 3.37 Allyn & Bacon 3.41 Transaction 1.98 Allyn & Bacon 3.34 HarperCollins 3.40 Chapman & Hall 1.94 Haworth 3.34 Guilford 3.34 Lexington 1.91 Random House 3.19 Lexington 3.28 M. Dekker 1.90 Houghton Mifflin 3.18 World Scientific 3.23 Allyn & Bacon 1.76 Viking 3.18 Ash gate 3.22 E. Arnold 1.65 Atlantic Monthly 3.11 Garland 3.21 Guilford 1.61 Simon & Schuster 3.10 Times 3.10 VCH 1.56 HarperCollins World Scientific 1.55 E. Mellen Ash gate 1.48 Times New Press 1.44 Doubleday The survey also requested the number of advanced degrees held by the respon- dents and the broad fields of those de- grees (science, humanities, social science). Virtually all respondents possessed ali- brary degree. Most had gone on to earn a second advanced degree, most often in humanities fields. A significant percent- Correlations between familiarity and relevance were positive for all publishers. age had gone on to earn a higher termi- nal degree, again the vast majority being in humanities fields. Table 3 reports the respondents' various levels of educa- tional attainment. Major Findings Tables 4 and 5 present the scores for each of the publishers listed in the survey. In interpreting these scores, it is important to remember that they are reputational. They are not based on any quantifiable 3.08 Doubleday 3.00 2.93 UPA 2.91 2.64 Haworth 2.65 2.48 E. Mellen 1.96 data but, rather, on collection develop- ment librarians' collective opinions.5 From both a methodological and a sub- stantive point of view, it was necessary to determine the relationships among the three key measures being assessed in the study. The correlations ~mong observed familiarity, relevance, and quality were almost always positive. Table 6 shows the mean correlation over publishers among the three dependent measures. All corre- lations reported in this study are Pearson product-moment correlations. Correlations between familiarity and relevance were positive for all publish- ers. Correlations of slightly over .50 were obtained for Ashgate, Basic Books, Elsevier, Kluwer, New Directions, and VCH. In other words, for all publishers, perceptions of academic relevance were a positive function of familiarity, more so for the indicated publishers than for oth- ers. Nearly all correlations between famil- iarity and quality were positive, with cor- relations exceeding .50 for Ashgate, Elsevier, National Academy of Science, and New Directions. Three small negative correla- tions were found between famil- iarity and perceived quality, with the -.12 correlation for Mellen being the largest. The correlation between mea- sures of relevance and quality suggests a serious "halo effect." Only in five cases did the corre- lation between these measures for specific publishers not exceed .50. Perhaps it could be argued that publishers with a more scholarly or scientific profile in- vest more care in their books, or can make a profit publishing only their strongest manuscripts, and that therefore the relation- ship between academic rel- evance and quality represents reality on some level. However, it also seems only realistic to con- cede that a halo effect does exist between respondents' percep- tions of these two dimensions. Despite the halo effect between perceptions of relevance and quality, there were a number of publishers for whom perceptions on the two dimensions differed in dramatic and revealing ways. Looking only at publishers whose rank orders on the two dimen- sions differed by fifteen or more places, the authors found two rather different groupings. For Gale, Garland, Greenwood, Sharpe, University Press of America, Westview, and World Scientific, rankings for relevance were at least fifteen places greater than those for quality. To some degree, these publishers target the academic library marketplace, giving their imprints a scope that is relevant almost by definition. The opposite finding, rankings of fifteen or more A Reputational Study of Academic Publishers 241 TABLES Perceptions of Academic Publisher Familiar- it.}:1 Relevance~ and Quality {al~ha order} Publisher Familiarity Relevance Quality Academic 2.50 4.22 4.03 Addison-Wesley 2.26 3.54 3.58 Allen & Unwin 2.13 3.76 3.81 Allyn & Bacon 1.76 3.34 3.41 Ashgate 1.48 3.46 3.22 Atlantic Monthly 2.13 3.11 3.60 Basic 2.50 3.87 3.90 Blackwell 2.75 4.37 4.25 Brookings 2.60 4.41 4.37 Cambridge 2.96 4.83 4.80 Chapman & Hall 1.94 3.73 3.68 CRC 2.27 4.38 4.08 Doubleday 2.57 2.48 3.00 E. Arnold 1.65 3.55 3.51 E. Mellen 2.26 2.93 1.96 Elsevier 2.63 4.47 4.15 Erlbaum 2.10 3.95 3.76 Farrar, Straus 2.51 3.38 3.96 Free Press 2.52 3.75 3.77 Gale 2.89 4.01 3.65 Garland 2.80 3.98 3.21 Greenwood 2.75 4.11 3.50 Guilford 1.61 3.48 3.34 HarperCollins 2.63 3.08 3.40 Harvard 2.87 4.82 4.82 Haworth 2.52 3.34 2.65 Houghton Mifflin 2.64 3.18 3.55 Humanities 2.19 3.94 3.62 Jossey-Bass 2.48 3.88 3.65 Kluwer 2.26 4.30 4.07 Knopf 2.70 3.40 3.96 Lexington 1.91 3.48 3.28 M. Dekker 1.90 3.79 3.61 Macmillan 2.66 3.55 3.69 McGraw-Hill 2.67 3.40 3.56 ME Sharpe 2.09 3.76 3.49 National Acad. 2.21 4.52 4.44 New Directions 2.09 3.71 3.90 New Press 1.44 3.50 3.68 Norton 2.60 3.63 3.90 Oklahoma 2.33 4.06 4.00 Oxford 2.94 4.81 4.77 Penguin 2.59 3.45 3.67 Plenum 2.29 4.05 3.84 (Cont. on next page) 242 College & Research Libraries May 1996 TABLE 5 cont. Perceptions of Academic Publisher Familiarity, < .01) indicated relationships of some interest, but no general pat- tern. Relevance~ and Qualit:r ( al~ha order 1 Most of the statistically signifi- cant relationships between given institutional or individual vari- ables and reported perceptions were unsurprising, and consis- tent with intuition. For example, chief bibliographers at libraries whose parent institutions offer doctorates in electrical engineer- ing are more familiar with Elsevier. Ed win Mellen is more familiar to librarians in institu- tions with larger materials bud- gets. Both the relevance and qual- ity attributed to Allen & Unwin were higher where the doctorate in English is offered. A consis- tently strong finding was that more experienced collection de- velopment heads and those at larger and more selective institu- tions had higher regard for both the relevance and the quality of Publisher Familiarity Relevance Praeger 2.58 3.86 Prentice Hall 2.64 3.37 Random House 2.72 3.19 Routledge 2.76 4.17 Sage 2.65 4.12 Simon & Schuster 2.69 3.10 Smithsonian 2.63 3.86 Springer-Verlag 2.64 4.46 St. Martins 2.74 3.77 Stanford 2.67 4.60 SUNY 2.48 4.24 Times 2.16 2.64 Transaction 1.98 3.67 UPA 2.51 3.55 Van Nostrand 2.21 3.58 VCH 1.56 3.98 Viking 2.48 3.18 Westview 2.46 4.00 Wiley 2.66 4.26 World Scientific 1.55 3.80 places higher for quality than for rel- evance, is found for Atlantic Monthly Press; Farrar, Straus, and Giroux; Alfred Knopf; New Directions; Norton; Penguin; Random House; the Smithsonian Institu- tion; and Viking-publishers that target the "upper-brow" lay reader. These two groups almost might be contrasted by saying that they pose a humble reminder that academic and intellectual are not syn- onymous terms. One of the more interesting findings encountered-perhaps more accurately a "nonfinding" -was the relative weakness of most Quality 3.68 3.45 3.64 3.97 3.77 3.45 4.24 4.34 3.83 4.51 3.86 3.10 3.56 2.91 3.62 3.88 3.59 3.62 4.15 3.23 imprints from the New Press. Re- spondents' experience in collection devel- opment was positively associated with the perceived relevance of books from Ba- sic Books, the Free Press, Guilford, and Knopf. It would be inappropriate to make much of these isolated findings, which were found in a purely post hoc "data dredge" of many possible relationships. Indeed, the major conclusion to be drawn is that the perceptions of chief collection development officers seemed to be drawn from a fairly homogeneous pool. There TABLE6 relationships between ei- ther institutional or indi- vidual variables and as- sessments of familiarity, quality, and relevance. Isolated findings of mod- erately high correlation (.35 or greater, all with p Descriptive Statistics for Within-Publisher Correlation Coefficients, N=64 Coefficient Mean S.D. Maximum Minimum r relevance, quality .65 .09 .87 .44 r relevance, familiarity .32 .12 .54 .02 r familiarity, quality .27 .15 .55 -.12 A Reputational Study of Academic Publishers 243 are no important differences between the perceptions of librarians at large and small institutions, or between those of more or less experienced bibliographers. To test further the apparent homoge- neity of the respondent pool relative to perceptions about publishers, the authors conducted a separate analysis of the per- ceptions of a subset of respondents who might be considered likely to be more knowledgeable than their peers. Rel- evance and quality ratings were studied for respondents who had been in collec- tion development for at least five years, at least two of them as head, and whose materials budgets were at least $1 million. The authors included scores for only those publishers for whom these indi- viduals had reported high familiarity. With only the most trivial exceptions, the relevance and quality ratings from this "informed elite" for each publisher were indistinguishable from the scores re- ported in table 4, again confirming the lack of differentiation among the respon- dent populations. In many respects, the overall results of this study are self-evident and speak clearly from tables 4 and 5. The con- tinuum of perceived relevance ranges from Cambridge down to Doubleday, that · of perceived quality ranges from Harvard down to Mellen, and every reader of this report is equally free to form an opinion about the credibility or relevance of the findings. The authors suspect that these findings, without complex analysis, will be the chief object of interest in this study. Notwithstanding the simplicity of the data arrayed in tables 4 and 5, some ele- ments of these findings merit discussion. It is worth noting that although all famil- iarity scores and the heart of the relevance and quality scores are smoothly distrib- uted, there are some discontinuities. Rel- evance scores have a fairly significant drop after Cambridge, Harvard, and Oxford, and fall off again at the end where Times and Doubleday are seen as significantly less relevant than the publishers above them. Similarly, quality scores fall off after the same three publishers, and then at the low end Haworth is seen as significantly inferior to the publisher above it, while the further drop from Haworth to Mellen is precipitous. In Mellen's case, the quanti- tative data were supplemented by anum- ber of pointedly negative comments writ- ten by respondents. The data also provide an answer to the authors' curiosity about how university presses as a whole would be compared to other publishers. For both relevance and quality, ratings begin with the same four exemplary university presses. No university press is rated lower than eigh- teenth in relevance, and none is lower than twenty-third in quality. Interestingly, of the top ten publishers in both relevance and quality, only four are for-profit, as in One of the most intriguing possibili- ties the current data facilitated was the chance to explore the ways in which publishers group together as they are perceived on the key variables of interest. each case both the National Academy of Sciences and the Brookings Institution, which are neither university presses nor for-profit presses, also appear. One can only speculate about how many univer- sity presses would have headed the list had all such presses been included in the questionnaire. If the dimensions on which publishers vary are considered-the academic or popular, scientific or humanistic scope of their titles, the strength of their reputations, their age and familiarity, their tendency to publish straight monographs as opposed to text books, proceedings, or edited an- thologies-it is clear that publishers are not distributed evenly or randomly across the multidimensional space these attributes describe. It is not uncommon to general- ize about groups of publishers, speaking of "publishers such as X and Y." 244 College & Research Libraries One of the most intriguing possibili- ties the current data facilitated was the chance to explore the ways in which pub- lishers group together as they are per- ceived on the key variables of interest. The relative simplicity of the data made it possible to do this without the strictures of a formal factor analysis. Instead, the authors inspected the correlation matri- ces of scores for relevance and quality to identify pairs of publishers whose percep- tions co-varied, and these results were in turn examined to find larger congrega- tions of publishers whose scores co-var- ied. Once the authors identified such clus- ters, they created new variables represent- ing group scales. They then calculated the correlation between each cluster member and the group scale, discarded cases with weak correlations to their scales, andre- peated the process. Table 7 reports the results of this pro- cedure for relevance scores, giving a pro- visional cluster name, a list of cluster members, and the mean correlation of each member to the scale. It should be noted that because each member contrib- utes to the scale constructed for its group, a certain degree of autocorrelation is present. Generally, this is not considered problematic for larger clusters, but it pre- sents something of a statistical artifact May 1996 that artificially boosts item-to-scale cor- relation for smaller clusters. Even allowing for the effects of autocorrelation, the internal coherence of the reported clusters is very high. Most observers will probably recognize com- monalties among cluster members. It should be noted, however, that the labels the authors provided are purely post hoc and subjective, and ·it is certainly possible that others could furnish names that more accurately capture the defining character- istics of each group. In the construction of any set of scales or clusters, the ideal outcome is relatively high covariation within groups and rela- tive independence between groups. In addition to satisfying the first criterion, the clusters shown in table 7 did relatively well on the second. The mean correlation among scales was .32, with none higher than .43. The results for quality scores were not quite so dramatic as those for rel- evance, reflecting a tendency of chief collection development officers to group publishers more in terms of their scope (which, within the added context of the institution's mission, translates to relevance) than in terms of quality. Nonetheless, interesting clusters emerged from an analysis of the quality scores. TABLE7 High Covariance Clusters for Respondent Perceptions of Publisher Relevance Cluster A: Letters Farrar, Straus, Giroux Knopf Macmillan New Directions New Press Random House . Simon&Schuster Viking Mean r=.76 Cluster B: Cluster C: Intemat'l British Science Letters Elsevier Allen & Unwin K.luwer Edward Arnold Springer Ashgate VCH WorldScien. Mean r=.79 Mean r=.88 Cluster D: Cluster E: Commercial/ Library Textbook Reference McGraw-Hill Gale Prentice Hall Garland Van Nostrand Greenwood Mean r=.84 Mean r=.85 A Reputational Study of Academic Publishers 245 TABLES High Covariance Clusters for Res~ondent Perce~tions of Publisher Quality Cluster A: Cluster B: Trade Trade Popular Academic Doubleday Addison-Wesley Farrar, Straus,Giroux Allen & Unwin Free Press Chapman & Hall HarperCollins Jossey-Bass Houghton Mifflin New Press Knopf Routledge Macmillan Sharpe McGraw-Hill Westview Prentice Hall Random House Simon & Schuster Viking Mean r=.71 Mean r=.70 Table 8 represents the results in a format identical to that used in table 7. Not only were intracluster relation- ships somewhat weaker for the quality clusters than for observed relevance, there was also weaker discrimination (higher intercorrelation) among clusters. The mean correlation between pairs of clus- ters was .38, compared to .32 for relevance clusters. Much of this overall strength of relationship among clusters was attribut- able to a .59 correlation between clusters B and C. These clusters of publishers are somewhat similar in that both represent houses whose books are of a fairly seri- ous academic nature. The difference seems to be that a higher proportion of cluster C publishers are non-U.S. houses emphasizing science. Cluster E, "Library Reference," is identical for both relevance and quality measures, indicating a fairly distinctive identity for the three "letter G" publishers which are prominent in library and reference publishing. One of the final items on the question- naire was an open-ended invitation for respondents to comment on the themes the study addressed. Many responded positively to this invitation or made com- Cluster C: Cluster D: Cluster E: Intn'l Academic Library Science Elite Reference Academic Cambridge Gale E. Arnold Harvard Garland Ash gate Oxford Greenwood Elsevier Humanities Kluwer Plenum Springer-Verlag Mean r=.70 Mean r=.79 Mean r=.83 ments about specific publishers. Several comments confirmed that, though not the most important factor in selecting a book, the reputation of the publisher does play a significant role. This is especially true when more concrete information may not be available, such as the reputation of the author, the existence of a review, or the ability to examine the item directly. Several broad themes recurred through- out the comments. The most significant ones were a discussion of price (even for books that would be both highly relevant and of good quality); the importance in selection decisions of an institution's cur- riculum, level of instruction, and faculty research interests; the subjective nature of the selection process; and the problem of publisher specialties. Many respondents said that the qual- ity and relevance of an item could only be determined by including price in this consideration. Numerous comments criti- cized the exclusion of price as a criterion for review. With hard-pressed materials budgets, these respondents claimed, there is no determining whether an item is of good quality without considering its price. Highly relevant, good-quality items 246 College & Research Libraries can and are legitimately skipped in or- der to stretch thin materials budgets. Related to the consideration of price was the question of an institution's cur- riculum and whether it is supporting a graduate program or active faculty re- search. If an institution supports research by graduate students and faculty, it is more important to spend the resources to acquire and make available high-priced, high-quality research materials. The li- brarian has to consider the price in deter- mining the importance of an acquisition to the overall institutional considerations, not just an item's quality and relevance to a library's collection. Respondents mentioned many means of assessing publishers, including pat- terns in faculty suggestions, reviews, pub- lisher success in winning Choice OAB awards, personal reading and academic preparation, and physical inspection of new receipts. As one respondent noted, "Unfortunately, no tools exist to allow selectors to evaluate systematically the quality and relevance of publishers. One of the reasons that our social science and humanities bibliographers are required to review all new acquisitions physically is that they gain a familiarity with the pub- lishers in their areas." ... the quality of a number of publishers' books varies markedly from one discipline to another. Many respondents believe that al- though relevance and quality are critical issues and do tend to vary by publisher, it is extraordinarily difficult to make valid assessments. Judgments are subjective and vary greatly from one librarian to another. Publishers are so specialized that only those familiar with a field can evaluate their work. Even so, two respondents ar- gued, the quality of a number of publish- ers' books varies markedly from one dis- cipline to another. Chance comments by individual faculty members may in- May 1996 · fluence a bibliographer's thoughts about a publisher for years: Other li- brarians may hold onto perceptions of publishers that were once the best in their fields but are now living off old reputational capital. The remarks that several respondents made about subjectivity and about the variation in quality within the catalogs of individual publishers buttress some of the cautionary notes with which this re- port began. It is critical that readers of this report recognize, first, that quality is a subjective phenomenon; second, that the overall quality associated with a pub- lisher may vary by discipline; and, finally, that even publishers considered to be weak will put out books that are widely admired. It would certainly be an abuse of the results of this study if academic committees concerned with promotion and tenure were to use its findings to as- sess candidates' books without reading them. Quite a number of respondents were very candid about the difficulty of find- ing one person who could evaluate the entire range of publishers. Often they thought that their administrative and budgetary duties had increasingly taken them away from the substance of collec- tion development, perhaps disqualifying them as respondents. A number indicated that they had involved several bibliogra- phers in the rating or had delegated completion of the questionnaire to a bib- liographer with less administrative re- sponsibility, but closer daily involvement in collection-building decisions. Several respondents shared their thoughts on different categories of pub- lishers. One noted that books from uni- versity presses and the standard aca- demic houses are more likely to have un- dergone stringent peer review. Another noted that for financial reasons univer- sity presses have begun to publish more popular material. One respondent, though noting that society publications can be very specialized and bibliographi- A Reputational Study of Academic Publishers 247 cally complex, regretted that societies such as the Institute of Electrical and Electron- ics Engineers (IEEE) had not been in- cluded so that they could have been com- pared to their commercial competitors. Other comments made concerned the publishing industry and its many recent changes. Respondents noted that the in- dustry has been undergoing great change, much like the library field. Sev- eral commentators stated that the lack of concern over the production of the physical book is disappointing, one thinking it may also reflect a less-than- excellent care in intellectual content. A few respondents noted that they try to identify and purchase books on acid- free paper. Conclusion In addition to the main findings about collection development librarians' opin- ions on various publishers, this study con- firms and leads to several other conclu- sions. That university presses are well re- spected is quickly confirmed, with all of them finishing in or near the top third in relevance and quality, and two-thirds of the small sample included in the survey taking the first four slots in both of these categories . Further, tli.e study confirms that librarians tend to think of publishers in groups related around either subject scope (for relevance) or, to a lesser degree, market target (for quality). Collection de- velopment librarians clearly do have well-established mental images for pub- lishers, images that guide both the many micro-level decisions they must make about item selection and the more criti- cal macro-level decisions involved in such matters as the design of approval plans. The present study is apparently the only one of its kind, and suggests anum- ber of possibilities for replication and ex- tension. Other dimensions of publishers' portfolios, including price, audience level, or instructional versus scholarly or scientific emphasis, could be measured. Obviously, other publishers could be included (indeed, the authors regret- fully agree with one respondent who lamented the exclusion of professional societies as publishers), or similar sur- veys could be sent to other kinds of re- . spondents, including academic faculty, book dealers, or publishers them- selves. Au. note: The authors wish to acknowl- edge gratefully Linda Southard's painstak- ing efforts in distributing the survey and entering the results, and the invaluable as- sistance of Bob Frary in instrument de- sign and in the analysis and interpreta- tion of their data. Readers wishing to see the questionnaire used to obtain data for this study may request a copy from the au- thors. Notes 1. Hendrik Edelman, "Selection Methodology in Academic Libraries," Library Resources & Technical Services 23 (winter 1979): 33-38. 2. Charles A. Schwartz, "Book Selection and Bounded Rationality," College & Research Librar- ies 50 (May 1989): 328-43. 3. John Calhoun and James K. Bracken, "An Index of Publisher Quality for the Academic Library," College & Research Libraries 44 (May 1983): 257-59. 4. Edward A. Goedeken, "An Index to Publisher Quality Revisited: A Partial Replication," Library Acquisitions: Practice & Theory 17 (fall1993): 263-68 . 5. Two printing errors should be reported. On the questionnaire sent out in the first mailing, Edwin Mellen was represented as "Edward Mellen" and the "5" score for quality of Stanford University Press was omitted. These errors were corrected for the second mailing. Many respon- dents provided their own "5" for Stanford's quality score, then circled it. The differences in scores for these publishers were not statistically significant between the two versions. In both, Stanford ranked fourth an~ Mellen last on quality. Stanford's quality scores closely co-vary with its relevance scores, for which there was no omission. INFORMATION COMES IN ALL SHAPES AND SIZES OCLC's TECHPRO Service catalogs it all With library resources so stretched, it's challenging to keep your cataloging up- to-date. And it's frustrating, because uncataloged materials are inaccessible to your patrons .. OCLC's TECHPRO Service can help. Since 1985, we've offered solutions for libraries that have cataloging backlogs or need ongoing cataloging support. • Customized cataloging and physical processing to match your specifications • Cataloging for all bibliographic formats and many languages • Quality cataloging at prices that can reduce your overall cataloging costs Contract cataloging for books, serials, scores, non-print items, foreign language materials and more-it's cataloging in all shapes and sizes-from OCLC's TECHPRO Service. 1-800·848-5878, ext. 4386 Bill II