College and Research Libraries Evaluation of the Public Service Functions of Serial File Systems Elizabeth Hanson and Judith Serebnick This study presents a methodology for describing, analyzing, and evaluating the public service functions of serial file syste111:?, with emphasis on evaluation. The system in a university de- partmental library is analyzed~ and a user study is conducted to identify successful and unsuc- cessful search patterns of a group of students who have not used the library previously. The user study employs citations, questionnaires, interviews, and critical analysis of 130 factors related to selected variables to determine the efficiency and effectiveness of the system. The methodology employed is potentially useful for bibliographic instruction. n many library settings, a seri- als known-item search requires a patron to consult a complex network of bibliographic and holdings records reflecting a unique com- pilation of cataloging codes, holdings statement standards, and in-house techni- cal services policies and practices. This record network is organized into a cluster of separate and often single-purpose files that include information on current issue receipts, bound volume holdings, and claims, bindery, and other processing ac- tivities. The patron can access some files directly, while others are consulted by li- brary staff on behalf of the patron. In most libraries, this serial file system includes the union catalog, shelflist, serials check- in files, a variety of in-process records and, in many cases, a separate list of seri- als holdings. The investigators who undertook the project described in this paper were inter- ested in exploring the overall question of how the public service functions of such systems can be defined, described, and evaluated. This process begins with a li- brary delineating what patron-oriented functions it has assigned its serial file sys- tem, specifically or by default, and then proceeds to a description of the system and, finally, to an evaluation of how well the system fulfills its public service re- sponsibilities. The current study focuses most specifically on the evaluative facet of this process. The intent is to suggest a use- ful approach toward the evaluation of cur- rent systems, either to improve their effi- Elizabeth Hanson is associate librarian, Indiana University Libraries, and Judith Serebnick is associate profes- sor, School of Library and Information Science, Indiana University, Bloomington, Indiana 47405. The authors want to thank the Indiana University Office of Research and Graduate Development, the School of Library and Information Science, and the University Libraries for their encouragement and financial support of this research. This study would not have been possible without substantial assistance from Lois Heiser, head, Geology Library, Indiana University Libraries. Marie Conlin, a recent M. L. S. graduate, worked closely with the authors on all major phases of the study, and her contributions were invaluable. Julie Bobay, Gary Charbonneau, Stephen Harter, ]on Masland, and Patricia Steele also provided much appreciated advice and assistance. 575 576 College & Research Libraries ciency or to redesign them in anticipation of an online serials control system . The utility of this exercise seems clearly indicated by previous studies, which show that library users are often thwarted in their search for serials and mono- graphs, and that accessibility problems frequently outnumber acquisitions prob- lems. u3'4 Also, there is considerable evi- dence that many library patrons will not seek assistance from library staff when they run into trouble-in fact, many do not even realize that they are in diffi- culty. 5'6'7 Thus, previous research provides a strong impetus for the examination and evaluation of serial file systems and indi- cates that one key requirement of an effec- tive system is that patrons find it easy to use . W. F. Lancaster goes a step further to assert that: "Evidence suggests that per- ceived ease of use may be the major crite- rion considered in selecting an informa- tion source and the overriding factor influencing whether or not a particular in- formation service is used. " 8 However, ease of use has seldom · been treated as a variable in systematic descriptions and evaluations of serial systems. RELATED RESEARCH Beyond providing a rationale for the study, previous research was of limited usefulness . The investigators conducted an extensive computerized and manual literature search focusing on patron use of serial systems and, in general, the applica- tion of systems analysis techniques to li- braries and library use surveys. Through Dialog, the ERIC, LISA, and Soc Sci Search databases were searched. ABI In- form was considered, but a preliminary search indicated that a full search was not warranted. Libra7 Literature and bibliog- raphies by Tuttle, Pan, 10 and Pitkin11 were analyzed manually. Only a small body of literature seemed even peripherally rele- vant. Works by Adalian, Rockman, and Rodie, 12 Golden, Golden, and Lenzini, 13 Melin, 14 Murfin, 15 Penzelik, 16 and Whitlatch and Kieffer17 were most useful in suggesting areas of investigation and methodological considerations . Majorie Murfin's study "The Myth of November 1986 Accessibility: Failure in Retrieving Period- icals" 18 served as a major departure point . Her study was designed to determine ''how many patrons failed to locate peri- odical issues for titles to which the library subscribed" and "what part of the failure was solely the library's responsibility ... and what part was due to user errors." 19 Thirty-one undergraduate students in a basic reference class were each given five citations drawn from 252 Readers' Guide to Periodical Literature citations. The students had one hour to retrieve the articles, all of which were in journal titles held by the li- brary. Subsequently, they reported the results of their searches and filled out a questionnaire that inquired about their search strategy, their understanding of li- brary terminology, and so forth. A check was then made of all citations to deter- mine if the failures were the responsibility of the library or of the user. Murfin's results showed that 55 percent of the 155 citations were retrieved by stu- dents. Of the 45 percent not retrieved, 15 percent were judged to be due to user er- ror, and 30 percent were attributed to li- brary problems (broadly defined). Con- tent and use of a directory of periodicals, library operations failures, and physical separation of the microform and bound volume collections contributed to a high proportion of the user errors. PARAMETERS OF THE STUDY As the current study progressed, the in- vestigators asked questions similar to Murfin's. How many patrons fail to locate materials that are in the library, whether on the shelf or in a temporary location such as a carrel? How many fail to gain in- formation about the status of materials . that are not currently in the collection, e.g., on claims or at the bindery? When a patron finds neither material nor informa- tion, are these failures due to user errors, or are they the library's responsibility? Additional questions came to the fore. If patrons are successful in locating materi- als or information, what are the reasons for their success? What search strategies are used by successful and unsuccessful searchers? How do patrons. approach a search for serials in a library they have not Evaluation of the Public Service Functions 577 used previously? How does the organiza- tion of the serials file system relate to the patrons' search strategies? By designing a study to describe and evaluate the public service functions of se- rial file systems, the investigators sought to test a potentially useful evaluative methodology in a small-scale context that would allow for a carefully controlled study. They opted to focus on access to se- rials in the Geology Library of Indiana University Libraries, Bloomington. This medium-sized branch library occupies part of the sixth floor of the Geology Build- ing and functions as a unit of the central- ized university library system. It houses a collection of seventy-five thousand mono- graphs and bound serial volumes. Through subscriptions and an extensive exchange program established by the In- diana Geological Survey, it maintains a se- rials list of 1,365 currently received serial titles. All cataloging services and acquisi- tions activities are provided by the central- ized technical services units of the Main Library. A high proportion of its clientele consists of Geology Department students and faculty and survey staff members, but students and faculty from other depart- ments also use the library. The staff con- sists of one professional librarian, two f_ull-time support staff members, and two FTE part-time student assistants (usually eight students each academic term). The Geology Library was chosen as the site for a number of reasons. It has an ex- tensive serials collection and a well- developed and efficient technical services operation that is large enough to sustain an extensive analysis. Also, the librarian has a strong commitment to high-quality public service and was willing to lend sup- port to the study. She provided invaluable assistance and advice throughout the project. It is important to emphasize that the in- vestigators wanted to evaluate the serials file system independent of its present use. This approach has distinct advantages, as W. F. Lancaster and Deanne McCutcheon point out: in terms of assessing the value of the library as a service to the entire community, concentration upon the present users gives, in many ways, a distorted picture of the success of the library .... If we always evaluate library ser- vice in terms of the expressed needs of the present users, then as a result of our evaluation, when we attempt to improve the service, we tend to move the library service toward the de- mands of the present users, and away, per- haps, from the unexpressed needs of the users and the needs of those who are not presently using the library. 20 DESCRIPTION OF THE SYSTEM The serial file system of the Geology Li- brary needed to be carefully analyzed and described before being evaluated. The in- vestigators first constructed a map of the library that graphically depicted the entire system and identified its individual com- ponent files within the context of the over- all physical layout of the library. The map is reproduced at the end of the article as a reference for the following discussion. The library is generally well laid out. The circulation desk and card catalog are visible upon entering the room, and the stacks area is well arranged with clear signs at the end of the ranges. However, the current issues section is not visible from either the entrance, the circulation desk, or the card catalog. Also, although the computer printout of serial holdings is located on a stand directly across from the circulation desk, there are no signs direct- ing patrons to its location. Secondly, a description of each compo- nent file was drawn up, with emphasis on content and function. The files are as fol- lows: Card Catalog-A dictionary catalog of all cataloged materials in the Geology Li- brary including records of serial bound- volume holdings. These are reflected in several ways including the ticking off on date cards of new volumes received or the penciling in of updated holdings on unit cards. Some additions such as extra cross-references have been made by staff in recognition of the special needs of their clientele. Shelflist-A standard shelflist of Geology Library holdings that reflects serial bound volume holdings, some biblio- graphic information regarding title changes, etc., and certain detailed loca- 578 College & Research Libraries tion information, e.g., titles shelved in the storage area, etc. Computer Printout-A listing of all serials received by the Geology Library via the Main Library's Serials Department. Ti- tles currently received but not yet cata- loged are noted as NEW. Newly as- signed call numbers are written on the printout by Geology Library staff pend- ingreceipt of an updated copy. This is the only public-access record of newly received serial titles. Kardex-Contains receipt and claims rec- ords of all serials (journals, mono- graphic series, conference proceedings, annuals, etc.) received on an ongoing basis in the Geology Library. New serial titles, including those yet to be cata- loged, are entered therein. Order File-Includes records for serial backfiles being purchased. Circulation File-Includes records of all serial volumes sent to bindery via the Main Library's Preservation Depart- ment, checked out for use in a carrel or to a faculty member, or on reserve for a course. GEOLOGY LIBRARIAN'S VIEW The geology librarian was asked to de- scribe in detail how she believed patrons located serials in her library. She postu- lated that the search pattern of new pa- trons was: 1. Patrons usually go to the circulation desk first and normally ask for either the card catalog or ''periodical file'' (meaning a card catalog just for serials). The desk at- tendants send patrons to the card catalog or to the printout of the periodical file. 2. The patrons check the card catalog or the printout. 3. When they have found the titles in ei- ther of the above, they return to the desk and ask where the periodicals are. Experi- enced library users ask where bound vol- umes or current issues are if they believe they can tell in which section their titles are located. 4. The desk attendant is expected to tell the patrons that ''bound volumes are in the stacks and current issues are over there" (gesturing toward the current- issues section). November 1986 5. If the patrons do not find the mate- rial, approximately 75 percent go back to the desk, where attendants will attempt to determine if the materials are bound vol- umes and will check the circulation file un- der the call numbers to see if the volumes are checked out to the bindery, to a faculty member, or for use in a carrel. If in a carrel, patrons are directed there. If it is unclear whether the material is bound, the attendant asks if the patron looked in the current issues section. If yes, the attendant checks the kardex to see if the material has been received and if so, searches the library tables, etc., to try to lo- cate the items. METHODOLOGY As indicated above, the investigators wanted to determine the overall rate of success and failure of library users' known-item searches for serial literature. They also sought to identify patron and library-related factors that contributed to this success or failure. They planned to re- construct the search strategies employed by both successful and unsuccessful par- ticipants and to analyze these in relation to the operation of the library's file system. How well the organization of the file sys- tem relates to the strategies employed, which facets of the system work well, and which need to be improved were key questions. Early in the study, the decision was made to conduct a user survey broadly based on the one in Murfin's study. The student participants, both undergradu- ates and graduates, would be given cita- tions to serial articles and asked to locate the cited articles in the Geology Library. The students would not have used this li- brary previously, and the desk attendants would not be informed of the study. The geology librarian searched the GEOREF database and identified one hundred randomly selected citations to ar- ticles in a wide variety of serials covering an extensive time period. These were largely English-language serials to which the library subscribed. Each citation was searched to determine the status of the · volume or issue that included the article and the ease or difficulty of locating it. For volumes and issues not on the shelves, the Evaluation of the Public Service Functions 579 searcher verified that the circulation desk or kardex files showed that the items were at the bindery, on reserve, in a carrel, or on claim. The approximate time required for the search was also recorded. Subsequently, each search was rated as easy, medium, or hard using the criteria shown in table 1. A written questionnaire was developed to elicit information about the students, their search strategies, and their use of the serial file system. Some variables were chosen from previous research, e.g., class rank, frequency of library use, and amount of instruction in library use. 21 Though previous findings are not clear- cut, it was expected that higher class rank, more frequent use of libraries, and more intensive bibliographic instruction might relate positively to success in finding cita- tions. THE PRETEST The next step in the investigation was to conduct a pretest using eight students, four undergraduates and four graduates, who were identified as articulate individ- uals with good analytical skills. The pur- poses of the pretest were (1) to test proce- dures developed for conducting the user survey, (2) to test the questionnaire, (3) to determine if participants could be ob- served unobtrusively, in order to chart their search strategies, and ( 4) to gather in- formation helpful in determining the opti- mum size of the citation list given to each participant, the number of participants, and the length of a search session. Two versions of the questionnaire were used: one included all the questions in a written format; the other presented two- thirds of the questions in a written format and the remaining third in a structured in- terview. The investigators wanted to de- termine if an oral interview would elicit in- formation not easily included in the written form. Each student was given ten citations- four easy, four medium, and two hard- and was told to use a "normal" search strategy-one the student ordinarily used to locate serials-and to feel free to ask for assistance from the desk attendants. The format of the citations was not explained in detail, but it was described as including the author and title of ah article and the ti- tle of the serial that contained the article. Students were allowed forty-five minutes to locate the ten citations and approxi- mately twenty minutes to complete the written questionnaire and interview. Each student was asked questions relating to the effectiveness of the pretest methodol- ogy. The pretest provided invaluable infor- mation. The most important finding was the individuality of each student's search strategy and response to questions con- cerning ease of use of the file system. The validity of the citations and questionnaires and the usefulness of a structured inter- view were established. However, it TABLE 1 Level Easy Medium Hard CRITERIA FOR RATING SEARCHES Criteria Title was located quickly in the card catalog . Bound volume containing the article was found easily in the stacks area or Article was in a 1984 unbound issue-this was the most recent year for the citations in the study . Search was estimated to take less than five minutes . Serial was published in a foreign language (but author and title were in English in the citation) or Serial title began with an initialism or Article was in a 1982 issue that was still shelved with current unbound serials. Issue or volume was in a carrel, sent to the bindery, on reserve, or being claimed, thus necessitating that the patron get information from the desk attendant or Article was in an uncataloged serial, thus necessitating use of the computer printout. 580 College & Research Libraries proved impossible to observe the partici- pants unobtrusively without alerting the desk attendants to the investigation, and the decision was made to forego such ob- servation. The most unexpected finding was that students had difficulty locating unbound current issues. This may be explained by the fact that many students had previ- ously used only the Main Library, where unbound periodical issues are located be- low street level. There are no bold signs di- recting patrons to this room and unless they are somehow informed, patrons may not realize that the Main Library separates unbound issues from bound volumes. Probably the most important result of the pretest was that it encouraged a sharp- ening of the survey's focus. The research- ers had envisioned that the user study would involve as many as thirty students, each of whom would be given ten cita- tions. However, because the pretest high- lighted the individuality of each student's response to the study, in-depth analysis of a smaller number of students seemed more appropriate than the shorter, more superficial studying and interviewing of a larger group. THE FINAL STUDY The final study was conducted in March and April 1985. Fifteen students were se- lected to participate, ten undergraduates and five graduates, none of whom had previous! y used the Geology Library or worked in a library. The selected students were among those who responded to ad- vertisements on bulletin boards in dormi- tories and to investigators' visits to classes in folklore, business, and computer sci- ence. They represented a wide variety of majors, and each was paid five dollars an hour for participating. The ten undergrad- uates included six freshmen, one sopho- more, two juniors, and one senior. It was decided that each student partici- pant would conduct six searches. Five ci- tation packets were compiled: each in- cluded two hard, two medium, and two easy searches, and each was used by three students to allow for comparisons. The ci- tations included articles in bound vol- ---m:nes, current unbound issues, items be- November 1986 ing claimed, uncataloged titles, and materials in carrels and at the bindery- regrettably, the investigators neglected to include items on reserve. Each student had thirty-five minutes to search for the ci- tations and twenty-five minutes or longer to complete the questionnaire and the in- terview. The investigators decided to use the long form of the written questionnaire and to supplement it with an interview that would seek additional information on search strategy; use of serial files; under- standing of library terminology and prac- tice, e.g., current periodicals, bindery; in- dividual reactions to ease of use of the file system; and effectiveness of circulation desk assistance. A structured interview schedule including twelve questions was designed. Immediately before each stu- dent was given a citation list, an investiga- tor checked the library to ensure that all the appropriate volumes and issues were properly shelved, and so forth. This was considered an important control factor be- cause it ensured that searches would not become snagged on factors unrelated to the study. RESULTS In evaluating the results, a successful search was defined as one in which the student either retrieved the appropriate volume or issue or determined that the is- sue or volume containing the article was at the bindery or being claimed. Items charged to carrels were retrieved. All other searches were considered unsuc- cessful. As noted above, each student had 6 citations. A perfect score was labeled 6/6, and at the opposite extreme, a score of 0/6 indicated that no citations were found. Of the fifteen students, only one achieved a score of 6/6, one achieved 5/6, and five achieved 4/6. These seven students (47 percent) were labeled ''successful partici- pants." The remaining eight (53 percent) were labeled "unsuccessful partici- pants"; of these, one student scored 3/6, two scored 2/6, one scored 1/6, and four scored 0/6. The fifteen students conducted ninety searches, of which 39, or 43 per- cent, were successful. As noted above, in Murfin's study, 55 percent of the 155 cita- Evaluation of the Public Service Functions 581 ~ tions were retrieved successfully by stu- dents. Were the unsuccessful searches due to patron errors, or were they the library sys- tem's responsibility? The answer to this question is not clear-cut. For example, it was not always possible to determine to what extent the library's public service staff contributed to user failures. Student testimony indicated that the desk atten- dants gave incorrect or unclear informa- tion on only seven searches, but several students did not receive the amount of service they requested, and other stu- dents could have been helped if the staff had noticed and responded to their per- plexity and inefficient search patterns. In some cases, the library and the patron shared almost equal responsibility for search failures. For example, in several in- stances both desk attendant and student failed to retrieve 1982 issues located in the current-issues section. Rather than assign failure responsibilities arbitrarily, the in- vestigators critically analyzed a number of variables related to the patron and to the library system. These included desk at- tendant, card catalog, computer printout, stacks, current issues, retrieval of in- process or temporarily located materials, and physical layout, including signage. For each variable, success and failure factors were identified from an analysis of the participants' questionnaires and inter- views, and each was labeled _a patron fac- tor or a library system factor. For example, for the desk attendant variable, the suc- cess factors for the library system included ''desk attendant conducted question ne- gotiation" and "desk attendant in- structed patron regarding the card cata- log.'' On the patron's part, success factors relating to the desk attendant variable in- cluded "patron asked for assistance con- cerning a serial'' and ''patron asked for as- sistance in a knowledgeable fashion.'' The total spreadsheet included 130 factors re- lating to the seven variables. The variables were analyzed for the suc- cessful and the unsuccessful participants, and the resulting spreadsheets were used as the basis for characterizing the searches and evaluating the serial file system. In supplying data for the spreadsheets, the investigators included only data that came directly from the participants' question- naires and interviews. No attempt was made to second-guess a situation or its contributing factors. The successful participants included the five graduate students, one senior, and one sophomore; the unsuccessful partici- pants included two juniors and six fresh- men. As expected, the successful partici- pants were more frequent library users. Also, although all the students in both groups had used the Main Library, the successful participants were heavier users of the departmental libraries. In addition, six of the seven successful participants had received some bibliographic instruc- tion at Indiana University, while only three of the eight unsuccessful partici- pants had such instruction. Three success- ful participants had actively sought op- portunities for bibliographic instruction, while no unsuccessful participant had done this. Thus, higher class rank, more frequent use of libraries, and more ·inten- sive bibliographic instruction distin- guished the successful from the less suc- cessful group. With the small samples involved, tests for statistical significances were omitted, but it is interesting that the differences were those that were pre- dicted. Analysis of the desk attendant variable showed that each of the seven successful participants asked for assistance (100 per- cent), while seven of the eight unsuccess- ful participants did so (88 percent). How- ever, while a similar number asked for assistance, the two groups differed widely in the type of aid sought, in how they asked for help, and in their interpretation and use of the information they received. For example, at least half of the unsuccess- ful participants asked for assistance in a vague fashion-several students told the desk assistant that they did not know what to look for, and others asked for help in finding books without understanding that they were searching for serials. The successful participants specified that they were looking for serials and posed their questions in a knowledgeable fashion. For example, if particular issues were not in ~he library, they asked the desk attendants 582 College & Research Libraries if the serials were at the bindery, etc. It is worthwhile noting that during his interview, the student who achieved a perfect score of 6/6 commented that he thought the best preparation for finding information in a library was to ''ask for help if needed." However, he added, "If you go up with a vague question, they'll say' Go to the card catalog.' You won't get much help if you are vague." His com- ments were most appropriate for this sur- yey. The desk assistant who ably an- swered his clear questions was the same individual who earlier had summarily told a vague student to "go to the card cata- log." For the card catalog variable, substantial differences between the two groups of students were apparent. The unsuccessful participants were hurt the most by the card catalog's exclusion of serials received but not yet cataloged. Only the computer printout records new serials received but uncataloged in the Geology Library. The failure of the library system to provide ad- equate links between the card catalog and other components of the serial file system contributed to failures for more than half of the unsuccessful participants and two of the seven successful participants. In general, the card catalog was more diffi- cult for the unsuccessful participants to use. Also, half of the unsuccessful partici- pants searched for at least one citation un- der the author and/or title of the article rather than the title of the serial. As noted above, there is no sign in the library alerting patrons to the existence and location of the computer printout, even though this lists most of the serials, including uncataloged items, and gives call numbers for cataloged titles. The desk attendant told three students in each group about the printout. However, one of the unsuccessful participants still did not consult the printout, and another failed to understand its use. The stacks in the library are well ar- · ranged and they have clear call-number signs at the ends of the ranges. Most of the successful participants commented on the signs, and four unsuccessful participants commented on them in a manner that in- November 1986 dicated these markers may have been a factor in their successful searching. A major failure factor for the unsuccess- ful participants related to the current is- sues variable. Six of the eight unsuccessful participants were unfamiliar with sepa- rate current-issues sections, and because each student had at least one and often two citations for current unbound issues, this lack of knowledge was crucial. Only one unsuccessful participant was directed to that section by the desk attendant, and she was given inadequate information about the purpose and use of the materi- als. The majority of successful participants were familiar with current unbound is- sues and either were told about the li- brary's current-issues section by the desk attendant or observed the area when they walked around the library as part of their search strategy. In general, students in both groups were uncertain about the age of serials in this section. Some correctly in- terpreted the bound volume information in the card catalog records and assumed later volumes or issues were unbound or · at the bindery. However, all students who had citations for 1982 unbound issues were surprised to learn that serials more than two years old could be in the current issues section. None of the unsuccessful participants was successful in retrieving serials that were in process or in temporary locations, i.e., in a carrel or at the bindery. As noted above, citations for these serials were rated ''hard,'' and a successful search re- quired receiving correct information from the desk attendant. Most of the successful participants obtained the needed informa- tion after realizing that the items were not located in the stacks or the current-issues section. Also, the successful participants were more likely than the unsuccessful to be familiar with the bindery, carrels, claims, etc. Few students commented on the li- brary's physical layout. However, two successful participants did take self- directed tours of the library as part of their search procedures. Four students com- mented on the lack of directional signs, and two wanted a clearly visible map of Evaluation of the Public Service Functions 583 the library (a map is located on a bulletin board near the entrance to the library, but no student noticed it). Three successful participants commented favorably on the library's compact, open floor plan. guided tour; he "likes to wander," and it is his normal pattern to take a tour each time he uses a new library. He noted the current issues section as well as the stack area and the card catalog. Next, Fred went to the desk attendant and asked for ''a list of periodicals in the library.'' When inter- viewed, he said that he would not start a search for periodicals at the card catalog; previous experience at Northwestern Uni- versity and at other libraries had taught him that librarians often have a complete list of their periodicals . The desk atten- dant showed him the computer printout, w hich he had not noticed on his self-tour. SEARCH STRATEGIES The students' questionnaires and inter- views provided extensive information on their search strategies. The data were ana- lyzed to determine in what order each stu- dent progressed through the library's se- rial file system, at what point desk assistance was sought, etc . Table 2 indi- cates the search strategy of each student. As shown in table 2, and figure 1, the search strategy of the most successful stu- dent, named Fred, is unique. Upon enter· ing the library, Fred took a brief, self- After he obtained the call numbers from the printout, Fred went to the stacks for the older items and to the current-issues section for newer ones . Last, he asked the TABLE 2 OUTLINE OF SEARCH STRATEGIES Successful Participants Unsuccessful Participants (N = 7) (N = 8) Order of Search 6/6 5/6 4/6 4/6 4/6 4/6 4/6 3/6 2/6 2/6 1/6 0/6 0/6 0/6 0/6 1 SeT cc cc cc CI SeT cc cc cc cc cc cc cc DA cc 2 DA ST ST ST DA cc ST DA DA ST ST ST DA cc ST 3 PR DA CI DA PR ST cc cc PR DA DA DA ST DA 4 ST PR DA CI ST DA DA ST CI cc 5 CI CI cc CI cc ST ST 6 DA Lege nd : CC = Card Catalog; Cl = Curre nt-Issu es Sec tion ; DA = Desk Attend ant; PR = Computer Pri ntout; SeT = Self-G uid ed To ur; ST = Stacks Map no t drawn to scale fNIJANA UNIVER51TY G£aXY liBRARY~ ---------- ~-------- _J ,--------------------, I I §g I I C'l I I (iS I I I L---~-----------------J ._______.I c=:J c=J c=:J c:=J ~ ~------- - -----------: ~: ~ : I ~ I I I I_---------------- - - - - _J ,---- - ------------ - - - FIGURE 1. INDIANA UNIVERSITY GEOLOGY LIBRARY 584 College & Research Libraries desk attendant for locations of remaining items. He was familiar with the bindery, carrels, and in-process procedures and easily interpreted the information he re- ceived. Fred's search strategy proved to be the most efficient and effective for this partic- ular library and for the particular assign- ment he was given. On his tour, he found the current-issues section, which was missed by most students. The computer printout was easier to use for locating call numbers for serials than the card catalog, and because it included information on uncataloged serials-and Fred had a cita- tion for one uncataloged item-it contrib- uted to his success. No doubt his knowl- edgeable way of asking for assistance was also a major success factor. In analyzing the search strategies used by the other fourteen students, it is impor- tant to remember that each needed to use the card catalog or computer printout for call numbers, the stacks, the current- issues section, and the desk attendant. Nine students (60 percent) were given one citation to an uncataloged item, the search for which required using the computer printout to obtain locations. Table 1 indi- cates that six of the seven successful par- ticipants (86 percent) used the card catalog and/or computer printout, stacks, current-issues section, and desk attend- ant, though the order in which these were used varied. Of the unsuccessful partici- pants, only one (13 percent) used both the current-issues section and the computer printout. Often, however, desk attend- ants did not point out either of these; also, many students did not return to the desk after trying-and failing-to locate items in the stacks and current-issues section. Some ran out of time, and others incor- rectly assumed that the library did not own the items they were seeking. In many respects, the participants' search patterns approximated the pattern anticipated by the geology librarian, but there were nota- ble exceptions. Discussion As stated above, the main focus of this article is the presentation of a methodol- ogy that the investigators believe many li- November 1986 braries can use profitably. Though the number of participants was relatively small and participants were not chosen randomly, a number of interesting find- ings still warrant discussion. The investigators were struck by the number of participants who articulated an implicit faith in the card catalog as a guide to all materials in the library and were un- aware that a number of important catego- ries of materials (uncataloged titles, cur- rent periodical issues, etc.) were not reflected there. Also, many did not know how to interpret the information given about bound serial volumes. While the computer printout available from the Main Library's Serials Depart- ment was not designed to be a public ser- vice tool, the Geology Library-in com- mon with many of the other branch libraries-dearly felt it filled a need and made it available to patrons. The staff in- creased its usefulness by adding call num- bers. But while prominently displayed, the printout does not bear a label, nor does a sign point to its location, so that unless their attention is specifically called to it, patrons are unlikely to discover its exis- tence and usefulness. These first two observations relate to the fact that the ease of using a file system de- pends not only on the accuracy and user- friendly orientation of the individual com- ponents per se, but also on how easily patrons come to understand which file they must consult for any given type of in- formation they want-in other words, they need a basic understanding of the file system as a whole to be able to use it effec- tively. What any given file does not con- tain is at times as important to define as are its contents. Few files are so self- explanatory and their contents so stan- dard across libraries that students unso- phisticated in library use can be expected to be able to utilize more than a single sys- tem. This factor is also closely related to the physical layout of the library. For ex- ample, if related files are adjacent, their use is facilitated. The quality of assistance provided by the circulation desk attendants became an important factor in the search for serial lit- erature because the librarian assigns them Evaluation of the Public Service Functions 585 a major role in patrons' searches that in- cludes consulting certain files to which pa- trons do not have direct access. The qual- ity of this assistance was uneven. Perhaps this is to be expected in most university departmental libraries that rely heavily on part-time student assistants. However, Carolyn Snyder and Stella Bentley argue · that the ''key public service role of student employees requires that the~ be trained and prepared for such work.' ' 22 Assistants could be instructed to ask probing ques- tions and to encourage patrons to return to the desk if they do not find what they are seeking. This study's methodology al- lows for detailed evaluation of the public service given by the desk assistants, and such an evaluation seems important. As noted above, many participants had considerable difficulty with current-issue materials: students in both groups were uncertain about the age of serials in the current-issues section. This problem also surfaced in Murfin's survey; she reported that 40 percent of users failed to under- stand what was meant by "current" and "back" issues. 23 The extent of this particu- lar problem suggests that a library needs to consider extraordinary measures for alerting patrons to the existence and ar- rangement of current issues in their partic- ular setting . In designing signage systems in gen- eral, librarians should be sensitive to how confusing the layout of a library can be for new users. Signage systems often over- look the obvious. Perhaps self-guided tours could also be used to improve patron awareness. Conclusions The methodology for evaluating the public service functions of serial file sys- tems discussed in this study can be sum- marized as: 1. developing a description of the com- ponents, contents, and layout of the serial file system under study; 2. interviewing the library's adminis- trator to obtain his or her analysis of how patrons locate serials in the library; 3. conducting a user survey employing potential (not habitual) patrons, carefully selected serial citations, a pretest, and un- obtrusive measures; 4. analyzing data from the user survey in terms of successful and unsuccessful patron and system variables; 5. identifying specific aspects of these- rial file system that relate to patron success and failure and formulating suggestions for modifying the system to increase the potential for higher success rates. An important strength of the suggested method is that it includes a number of con- trols to ensure that the data collected are trustworthy and support sound conclu- sions. Examples of this approach are care- ful consideration of many factors in con- structing the citation packet and the verification of cited items immediately be- fore participants conduct their searches. Also, the process involves the evaluation of a system independent of its current use, an approach that takes into account needs that present users may not express as well as individuals who are currently nonus- ers, possibly because the system does not serve them well. Ideally, this methodol- ogy should be supplemented by studies of present users doing actual searches. An additional use of this methodology can be postulated in the area of biblio- graphic instruCtion. Though the students in the study were at times frustrated, all said they had enjoyed the search experi- ence and had learned valuable informa- tion about the library and searching. They were eager to discuss their failures as well · as their successes and responded well to questions relating to past experiences with searching for information. Advan- tages of the methodology are that if the ci- tations are carefully chosen, a small num- ber of searches can introduce the student to a large number of potential problems in using the library and that the student re- ceives immediate feedback on how to solve those problems. This study's findings sugges~ that in bibliographic instruction, librarians should never underestimate a patron's ig- norance of how to use libraries. Specifi- cally, librarians could: 1. strongly encourage patrons to ask for assistance and supply all the information they have; 586 College & Research Librarie s 2. stress the limitations of the card cata- log; 3. emphasize the need to look up jour- nal titles, not article titles and authors; 4. define current versus back issues. It is hoped that this discussion of the need to evaluate serial file systems and the presentation of a useful methodology will facilitate further work in this important November 1986 area of library operations. Serial literature remains one of the most important forms of publication but, a~ the same time, one of the most difficult to control and make available. The careful review of a library's serial file system in light of how patrons locate needed materials can make a large contribution to that institution's efforts to provide effective public service. REFERENCES 1. Daniel Gore, "Let Them Eat Cake While Reading Catalog Cards: An Essay on the Availability Problem," Library Journal100:93-98 (Jan. 15, 1957). 2. Marjorie E. Murfin, "The Myth of Accessibility: Frustration & Failure in Retrieving Periodicals," Journal of Academic Librarianship 6:16-19 (Mar. 1980). 3. T. Saracevic, W. M. Shaw, Jr., and P . B. Kantor, "Causes and Dynamics of User Frustration in an Academic Library," College & Research Libraries 38:7- 18 (Jan . 1977). 4. Jo Bell Whitlatch and Karen Kieffer, "Service at San Jose University: Survey of Document Avail- ability," Journal of Academic Librarianship 4:196-99 (Sept. 1978). 5. Roger Horn, " Why They Don't Ask Questions," RQ 13:225-33 (Spring 1974). 6. Mary Jane Swope and Jeffrey Katzer, "The Silent Majority: Why Don't They Ask Questions?," RQ 12:161-66 (Winter 1972). 7. Whitlatch and Kieffer, p.196- 99. 8. Wilfred F. Lancaster, The Measurement and Evaluation of Library Services (Washington, D.C.: Infor- mation Resources Pr., 1977), p .312-13. 9. Marcia Tuttle, Introduction to Serials Management (Greenwich, Conn.: JAI Pr., 1983). 10. Elizabeth Pan, Library Serials Control Systems: A Literature Review and Bibliography (Washington, D .C.: ERIC Clearinghouse on Library and Information Sciences, 1970). 11 . Gary Pitkin, Serials Automation in the United States: A Bibliographic History (Metuchen, N.J. : Scare- crow, 1976). 12. Paul T. Adalian, Jr., Ilene F. Rockman, and Ernest Rodie, "Student Success in Using Microfiche To Find Periodicals," College & Research Libraries 46 :48-54 (Jan. 1985). 13. Gary Golden, Susan Golden, and Rebecca Lenzini, "Patron Approaches to Serials: A User Study," College & Research Libraries 43:22-30 (Jan . 1982). 14. Nancy Jean Melin, "The Public Service Functions of Serials," Serials Review 6:39-44 (Jan ./Mar. 1980). 15. Murfin, p.16- 19 . 16. Barbara Pinzelik, "The Serials Maze: Providing Public Service for a Large Serials Collection," Jour- 11al of Academic Librarianship 8:89- 94 (May 1982). 17. Whitlatch and Kieffer, p.196-99. 18 . Murfin, p.16-19. 19 . Ibid, p.17. 20 . Wilfrid F. Lancaster and Deanne McCutcheon, ''Some Achievements and Limitations of Quanti- tative Procedures Applied to the Evaluation of Library Services," in Quantitative Measurement and Dynamic Library Service, ed . Ching-Chih Chen (Phoenix, Ariz . : Oryx, 1978), p.17. 21 . Golden, Golden, and Lenzini, p .25-26, 28 . 22 . Carolyn A. Snyder and Stella Bentley, "Staff Utilization in Branch Libraries : A Research Report," in Energies for Transition: Proceedings of the Fourth National Conference of the Association of College and Research Libraries, ed. Danuta A. Nitecki (Chicago : Assn. of College and Research Libraries, 1986), p.148. 23. Murfin, p.18-19.