cottrell.p65 334 College & Research Libraries July 2001 Applying an Information Problem- solving Model to Academic Reference Work: Findings and Implications Janet R. Cottrell and Michael B. Eisenberg The usefulness of the Eisenberg–Berkowitz Information Problem-Solv­ ing model as a useful categorization for academic reference encounters is examined. Major trends in the data include a high proportion of ques­ tions about location and access of sources, a lack of synthesis or pro­ duction activities, and a consistent presence of system problems (such as hardware problems) that impede the information-seeking process. The implications of these trends for the reference process, librarian train­ ing, and campus organization are discussed. he rapid and prolific introduc­ tion of electronic resources poses new challenges for aca­ demic reference desks. To help meet these challenges, new ways of look­ ing at the reference process may help. Traditionally, reference encounters have been considered in terms of the type of question or the type of patron involved. Recently, however, the reference encoun­ ter has been considered more fully as a process. After reviewing both models, this study examines the usefulness of one specific model in the reference process— the Eisenberg–Berkowitz Information Problem-Solving (IPS) model. Characteristics of Reference Questions Much of the existing literature on char­ acterizing reference encounters focuses on categorizing type of patron or type of question. Although clientele in the aca­ demic reference environment are not al­ together homogeneous, most patrons are affiliated with the university in one way or another. However, the variety in ques­ tions is almost endless; in response, re­ searchers have devised ways to charac­ terize them. Richard L. Derr categorized questions based on their conceptual pre­ supposition:1 • Existence (Does X exist?) • Identity (What is X? Who is X?) • Properties (What are the features of X?) • Relation (How is X related to Y?) • Number (How many of X are there?) • Location (Where is X?) • Time (When is X?) • Action (What is X doing?) Although this categorization offers an illuminating look at types of factual ques­ tions, it does not apply well to other types of questions encountered at an aca- Janet R. Cottrell is Director of Information Access, Library and Information Services, at Kenyon College; e-mail: cottrellj@kenyon.edu. Michael B. Eisenberg is a Director and Professor in the Information School at the University of Washington; e-mail: mbe@u.washington.edu. 334 mailto:mbe@u.washington.edu mailto:cottrellj@kenyon.edu Applying an Information Problem-solving Model to Academic Reference Work 335 demic reference desk. Richard E. Bopp described several types of reference ser­ vices:2 • Ready reference: Ready reference questions are answered quickly by con­ sulting one or two standard tools. • Bibliographic verification: Facts about publications are used to find or re­ trieve the publications. • Interlibrary loans and document de­ livery: This involves the verification of bibliographic information preparatory to interlibrary lending. • Information and referral services: These services involve the identification of community resources and agencies that can respond to a query. • Research questions: Research ques­ tions reflect broader user needs and are more complex than ready reference ques­ tions. They require more effort and care­ ful review and may take place over time. • Selective dissemination of informa­ tion: This service involves ongoing noti­ fication and keeps researchers abreast of current developments in their fields. • Database searches: This is mediated searching, often by appointment, away from the desk. F. Wilfrid Lancaster and Amy J. Warner used a somewhat similar scheme to char­ acterize reference encounters, categoriz­ ing information needs by purpose and scope.3 They proposed two types of infor­ mation need: the need to locate and ob­ tain a copy of a particular document for which the author or title is known (“known-item need”); and the need to locate documents that are on a particular subject or are capable of answering a par­ ticular question (“document delivery ca­ pability”). In addition, they identified three types of retrospective search: the need for a single item of factual data (ready reference); the need for one or more documents on a subject (but less than the total literature); and the need for a comprehensive search in which as much of the literature as possible on a subject is retrieved during a period of time. Other researchers have focused on the complexity or difficulty of questions ac­ tually recorded at reference desks, as well as the type of question or answer. Diane M. Brown analyzed telephone reference questions to characterize them by Dewey Decimal subject, answer format, and level of complexity as indicated by the specificity of the sources consulted.4 Tho­ mas Childers, Cynthia Lopata, and Brian Stafford studied reference questions looking for a measure of different levels of difficulty and indicative measures to stand for the concept of difficulty.5 Characteristics of the Reference Process Studies such as those mentioned above are interesting from a theoretical and sta­ tistical point of view, but in the midst of a demanding desk shift, they lose some of their appeal. More important, they place their emphasis on the question itself; by doing so, they draw attention away from another important aspect of the reference encounter—the process. Particularly in academic reference work, the process as­ sumes great importance because it fre­ quently occurs as part of an undergradu­ ate or graduate student’s educational experience. In these encounters, it is not necessarily the patron, the question, or the answer that matters; rather, it is the pro­ cess by which student and librarian solve an information problem and find infor­ mation appropriate to the task at hand. Every basic reference textbook con­ tains information on conducting refer­ ence interviews, and many emphasize looking beyond the originally presented question. Ellen D. Sutton and Leslie E. Holt noted that, in fact, the entire inter­ view process can be seen as problem cen­ tered rather than question centered.6 The emphasis on process rather than question was the basis for work by Carol C. Kuhlthau, who developed a model of the library search process incorporating both cognitive and affective aspects.7–9 Based on her studies of students per­ forming library research, Kuhlthau’s model included six steps: 1. Initiation: Student receives assign­ ment. 336 College & Research Libraries July 2001 2. Selection: Student selects general topic area. 3. Exploration: Student explores ma­ terial on general topic. 4. Formulation: Student focuses on more specific topic. 5. Collection: Student collects materi­ als on focused topic. 6. Presentation: Student prepares and presents finished assignment. Kuhlthau emphasized three main findings of major importance to librar­ ians: • Library searching is a process over time and not a single event. • Library searching is a holistic event rather than a simple activity. • Library searching commonly ini­ tially increases rather than decreases un­ certainty. Kuhlthau recognized that her model had implications for reference practice: reference librarians who have become aware of the stages in the information search process model “listen for an indi­ cation of the stage in the process of the user and particularly note when some­ one is ‘in the dip’ [of uncertainty] and needs some extra help to formulate a re­ search focus.”10 Both the head of the reference department and the researcher/ librarian agreed from the beginning that because the research relies on the reference process, it must be almost totally unintrusive. Lynn Kennedy, Charles Cole, and Su­ san Carter applied Kuhlthau’s model specifically to academic reference work, using a simplified version of it to assess and explicitly label undergraduates’ in­ formation needs as expressed at the ref­ erence desk and then designing appro­ priate search strategies for each phase.11 They first reduced Kuhlthau’s model to three stages: the Pre-Focus stage, in which the student selects a topic and lo­ cates general information (Kuhlthau’s stages 1, 2, and 3); the Semi-Focus stage, in which the student chooses a specific focus from the background information (Kuhlthau’s stage 4); and the Post-Focus stage, where the student gathers specific information to prove or disprove the the­ sis and presents the findings (Kuhlthau’s stages 5 and 6). Reasoning that explic­ itly labeling these stages would help identify the actual information need, Kennedy, Cole, and Carter next deter­ mined appropriate search strategies for each stage: high-recall searches for the Pre-Focus stage, high-recall/high-preci­ sion searches for the Semi-Focus stage, and high-precision searches for the Post- Focus stage. This groundbreaking re­ search illustrates the utility that informa­ tion-processing models can have in ref­ erence work. Eisenberg-Berkowitz Model Michael B. Eisenberg and Robert E. Berkowitz describe a six-stage model of Information Problem-Solving (IPS).12 At its most general level, the IPS model sim­ ply asserts that whenever students are faced with an information problem (or with making any decision that is based on information), they can use a system­ atic, problem-solving process. At its more detailed levels, the model includes six stages in the information problem-solv­ ing process, each having two specific components: 1. Task Definition 1.1 Define the problem. 1.2 Identify the information require­ ments of the problem. 2. Information-seeking Strategies 2.1 Determine the range of possible sources (brainstorm). 2.2 Evaluate the different possible sources to determine priorities. 3. Location and Access 3.1 Locate sources (intellectually and physically). 3.2 Find information within sources. 4. Use of Information 4.1 Engage (read, see, hear) the infor­ mation in a source. 4.2 Extract relevant information from a source. 5. Synthesis http:phase.11 Applying an Information Problem-solving Model to Academic Reference Work 337 5.1 Organize information from mul­ tiple sources. 5.2 Present information. 6. Evaluation 6.1 Judge the product (effectiveness). 6.2 Judge the information problem- solving process (efficiency). Although its stages often occur in the order listed, the model allows jumping between stages or looping back to previ­ ous steps as needed. Colloquially called the Big Six because of its six steps, this model has been widely adopted in education circles; the K–12 information literacy standards for many states are based on it. As more and more colleges and universities develop information literacy competencies, aca­ demic librarians also are becoming famil­ iar with its potential (see Mary Warner for an example13). Considering the Eisenberg–Berkowitz model in the academic reference setting suggests the following research ques­ tions: • Can academic reference encoun­ ters be characterized using the Eisenberg-Berkowitz IPS model? Do they fit gracefully? • Can the IPS model tell anything about how to approach the encounters, in terms of both answering the question and helping the client? If so, what impli­ cations does this have? The remainder of this paper describes an attempt to study these questions. Methods Collection and Coding of Data Sets Two sets of data were collected and used in this study. The first set consists of brief descriptions of 170 reference encounters at a midsize state university’s main li­ brary. The encounters took place during a weekly 2.5- to 3-hour midday, midweek desk shift over the course of one semes­ ter. The reference librarian jotted down very brief notes during the shift, then ex­ panded them as soon as possible after the shift ended. The first fifty descriptions were examined to see whether they could be characterized in terms of the compo­ nents of the Eisenberg–Berkowitz model. Consultation with the model’s codeveloper helped clarify questions about how to code certain types of en­ counters.14 The remaining descriptions were then coded, and the entire set of 170 encounters was treated as one prelimi­ nary data set. The second data set consists of brief descriptions of another 123 reference en­ counters from the following semester. These encounters were recorded and coded according to the methodologies developed with the first set. The second set of data was collected in an attempt to refine and verify the ear­ lier set. However, the constraints oper­ ating during the first phase were still op­ erating during the second phase, so the methodology did not change substan­ tially. In addition, some characteristics of the reference area (including database as­ sortment and printing procedur es) changed substantially during the inter­ val between the two collection periods. The data from the two phases were there­ fore analyzed and reported separately. Constraints on Data Collection Because the data result from reference encounters in a university library, great care was taken during data collection. Both the head of the reference department and the researcher/librarian agreed from the beginning that because the research relies on the reference process, it must be almost totally unintrusive. In other words, the reference process itself must always take precedence over the needs of the researcher. This placed severe con­ straints on the research process, especially during data collection. Constraints on the data collection in­ clude: • cannot elicit certain types of data (patron status, age, affiliation, other de­ mographics; too invasive); • cannot have an external observer or machine recording the encounters (un­ ethical to tape without consent; impos­ sible to observe session closely without being in the way); http:counters.14 338 College & Research Libraries July 2001 • cannot take extensive notes during the encounters or between encounters (too intrusive during encounter; no time between encounters); • cannot have a checklist visible to the patron (too intrusive/disruptive to patron). The compromise was that the re­ searcher/librarian carried a small pad and jotted down a few keywords during or right after encounter. Often the re­ searcher/librarian would do this any­ way (for example, to jot down search terms while working on a search with a patron), so this activity was considered part of the ordinary reference process. Clearly, this meant that the re­ searcher/librarian must reconstruct the encounters from memory, using the brief notes jotted down during the encounter. The goal was to do this as soon as pos­ sible after the end of the shift; occasion­ ally, this was not possible and some time elapsed between the encounter and the summarization. One additional requirement was al­ ways enforced on data collection: must record all encounters during the shift, not just the memorable ones. This require­ ment is essential to compiling a fully rep­ resentative data set, even though it de­ mands a strong commitment on the part of the librarian and may limit the num­ ber of data collectors in a full-blown study. Clarification of Coding Guidelines were needed to promote con­ sistent coding of the encounters. The fol­ lowing list, developed as a preliminary coding aid, includes brief definitions of each component from Eisenberg and Berkowitz, then identifies behaviors spe­ cific to the academic reference setting.15 1. Task Definition 1.1 Define problem. Determining what is required for an assignment, outlining tasks, determining order. Behaviors specific to the academic reference environment might include: • consulting the assignment; • focusing/clarifying the topic (of paper, etc.); • relaying information about the task to the librarian. 1.2 Identify information requirements of problem. Realizing specific types of information required for a project. Behaviors specific to the academic reference environment might include: • discussing assignment or task in terms of: —number of sources needed; —type of sources needed (book, jour­ nal articles, scholarly, popular, etc.); —type of information needed (focus, level of detail, information not needed). 2. Information-seeking Strategies 2.1 Determine range of sources. Brainstorming possible sources of in­ formation to meet the requirements. Be­ haviors specific to the academic reference environment might include: • discussion about what has been tried already; • discussion about likely fields (hu­ manities, social sciences, etc.—starting points on main menu); • discussion about likely indexes to use; • discussion about likely keywords; • discussion about potentially use­ ful print sources; • discussion of appropriate sources (print, Web, etc.). 2.2 Evaluate possible sources to deter­ mine priorities. Deciding which of sources are likely to be most useful. In the academic refer­ ence environment, this stage is often rolled in with 2.1; specific behaviors might include: • picking which database to try first; • looking through a list of citations to select useful items. 3. Location and Access 3.1 Locate sources intellectually and physically. Finding sources of information, in­ cluding specific books, articles, full-text sources, etc. In the academic reference setting, this might include: • questions about location of specific call numbers, reference books, special collections, etc.; http:setting.15 Applying an Information Problem-solving Model to Academic Reference Work 339 • ILL (suggesting ILL, accepting ILL requests for processing). 3.2 Find information within sources. Behaviors specific to the academic ref­ erence environment might include: • using the index or table of contents in print sources; • skimming full-text sources online; • skimming print sources. 4. Use of Information 4.1 Engage. Using a source to gain information. Behaviors specific to the academic refer­ ence environment might include: • reading full text or Web page online; • reading print source (atlas, fact book, reference book, etc.); • obtaining nondirectional facts from reference librarian. 4.2 Extract Taking notes or recording information from a source. In an academic reference environment, this might include: • taking notes from full-text or print sources; • printing full-text sources; • printing Web sites. 5. Synthesis 5.1 Organize information from mul­ tiple sources. Integrating information from a range of sources. In an academic reference en­ vironment, this might include: • discussion of how information from different sources will be used; • writing a bibliography after com­ paring formats from more than one style guide. 5.2 Present information. Creating a product such as a paper, presentation, illustration, final project, etc. This stage is seldom observed in the academic reference setting. 6. Evaluation 6.1 Judge the product (effectiveness). Determining whether the information need has actually been met effectively, as determined by the final product or project. This step is seldom observed in the academic reference setting because the product has not yet been created. IPS stages 1.1 � 6.2 plus System FIGURE 1 Percentage of Encounters In Which IPS Stage or System Problem Was Observed (Set 1) 340 College & Research Libraries July 2001 FIGURE 2 Percentage of Encounters in Which Each IPS Main Stage Was Observed (Set 1) TD ISS L&A IU S E 6.2 Judge the IPS process (efficiency). Determining how well the information problem-solving process worked. In the aca­ demic reference setting, this might include: • periodically evaluating whether useful hits are being found; • periodically evaluating whether correct database is being used; • reference librarian asking patron whether useful information is being found. One additional coding category was developed to deal with problems en­ countered during reference that were outside the scope of the IPS process. These problems, tentatively labeled “Sys­ tem problems” were often (though not always) hardware problems. They were not stages in the information process but, rather, directly impeded the IPS process, often by preventing access to an infor­ mation source. Each encounter was evaluated for any indication of any of the twelve IPS sub­ components plus any indication of sys­ tem problems. Any given encounter could exhibit any number of the possible cod­ ing components. Results For each of the two data sets, the stages observed in each encounter were tallied, along with any system problems. Results were compiled for the number of en­ counters in which each of the twelve IPS substages (and system problems) were observed and for the total number of stages observed in each encounter. The data sets from the first and second phases are presented separately. Phase One Data Set PreqsencnfeffIaSfStetes Figure 1 shows the percentage of encoun­ ters in the first data set (a total of 170 en­ counters) in which each of the model’s twelve substages were observed. The percentage of encounters in which sys­ tem problems were observed is noted at the far right of figure 1. Despite the coding guidelines, there were still some instances in which the coder was unsure how to classify an ob­ servation. Often this uncertainty had to do with which substage the observation reflected within the six major steps; in other words, the coder might recognize Applying an Information Problem-solving Model to Academic Reference Work 341 FIGURE 3 Number of Stages per Encounter (Set 1) an observation as task definition but be unsure whether to code it as 1.1 or 1.2. Because the uncertainty was at the sub­ stage level and not the main stage level, the data were then collapsed across sub­ stages. Figure 2 illustrates the percent­ age of encounters in which each IPS main stage was observed, regardless of sub­ stage coding. In both figures 1 and 2, the most no­ table features are the high frequency of location and access observations (step 3 in the model), and the low number of synthesis observations (step 5 in the model). Nearly 90 percent of all encounters in­ cluded location and access, primarily characterized by physically or intellec­ tually locating sources (3.1); in fact, 44 percent of all encounters included only stage 3.1 activities. Generally, the stage 3.1 activities included either giving di­ rections for locating call numbers in the stacks, departments of the library, and so on (physical location) or helping patrons find likely sources of information by us­ ing indexes, databases, and so on (intel­ lectual location). Even the two cases in which synthe­ sis was observed were not clear-cut: they represent encounters in which students discussed how they planned to combine information from diverse fields of study in their final product. As figure 3 shows, more than half the reference encounters in the first phase included just one of the twelve informa­ tion problem-solving subcomponents. Most of these single-stage encounters in­ volved location and access: 44 percent of all encounters included only stage 3.1 ac­ tivities. In some encounters (the “zero” cat­ egory in figure 3), no IPS components were observed. These cases represent en­ counters in which only system problems (outside the IPS process) were observed. System Problems As indicated in figure 1, about 16 per­ cent of the encounters in the first phase included system problems, defined dur­ ing the coding as problems encountered during reference that were outside the scope of the IPS process, often (though not always) hardware problems. System problems were not steps in the IPS pro­ cess; instead, they impeded the IPS pro­ cess, typically by preventing access to an information source or extraction of information from a source. Examples of system problems include microcom­ puter “freeze-ups,” server crashes, printer malfunctions, access problems Zero One Two Three Four Five Six Seven Eight Nine 342 College & Research Libraries July 2001 FIGURE 4 Percentage of Encounters in Which IPS Stage or a System Problem Was Observed (set 2) on databases requiring university ID (bar code access), slow or no response from remote databases, and so on. En­ counters in which a badly designed user interface impeded the IPS process also were designated system problems. In one case, for example, even the printed instructions for using a specific database provided no clue on how to print the full-text findings. When the system problem clearly im­ peded a specific step in the IPS process, both the system problem and the specific IPS stage were coded. For example, the case just described (badly designed in­ terface) was coded as involving a system problem during information extraction (4.2). Phase Two Data Set Preqsency of IaS Stetes Figure 4 shows the percentage of encoun­ ters (out of the total of 123 in the second data set) in which each of the model’s twelve substages were observed. The percentage of encounters in which sys­ tem problems were observed is noted at the far right of the graph. The data then were collapsed across substages in case of any uncertainty at the substage level. Figure 5 shows the percentage of encounters in which each IPS main stage was observed, regardless of substage coding. In figures 4 and 5, as in figures 1 and 2, the greatest number of observations are in the location and access stage of the model, and the least number of observations are in the synthesis stage. Nearly 80 percent of the encounters in this data set include location and access activities, primarily the physical or intellectual location of sources (3.1). In fact, 46 percent of all encounters included only stage 3.1 activities of physi­ cal or intellectual location. Number of Stages per Encounter As figure 6 indicates, over 60 percent of the reference encounters included just one of the twelve IPS components. Most of these (46% of the total set) involved only stage 3.1 location and access activities. System Problems As figure 4 indicates, about 15 percent of the encounters in the second data set in­ Applying an Information Problem-solving Model to Academic Reference Work 343 FIGURE 5 Percentage of Encounters in Which IPS Main Stage Was Observed TD ISS L&A IU S E cluded system problems. Because the printing process changed substantially between phases one and two, fewer printing problems occurred. Many of the system problems in this data set con­ cerned IPS-related activities that a patron might reasonably expect to accomplish in the workstation area, which were ei­ ther not available at all or impeded by lack of a clear interface and instructions. For example, several system problems concerned using campus e-mail software (which is unavailable on these worksta- Librarians can use the model to ascertain where the student is in the research process, advise accordingly, and verify that all relevant steps are covered. tions) or registering online for credit courses (which is available but may be blocked by passwords, locks, or other features of the system that may require intervention by other offices such as the registrar or accounting). As in phase one, when the system problem clearly impeded a specific step in the IPS process, both the system prob­ lem and the specific IPS stage were coded. However, if the IPS stage was not clear or relevant (e.g., registering for courses), only the system problem was recorded. Discussion Dcncuslsiindings The constraints on data collection did af­ fect the research process. Without the use of an external recorder, detailed notes, or checklists during the actual reference encounter, the description of each en­ counter was necessarily brief. Even though these brief descriptions were an­ notated and coded as soon as possible afterward, inevitable delays and incom­ plete memories affected the quality of the data. Thus, it would be a mistake to overinterpret the current data. Despite the data collection and cod­ ing challenges, the encounters did seem to fit gracefully into the model. Three trends stand out: (1) location and access activities were observed in many en­ counters; (2) synthesis activities were ob­ served in very few encounters; and (3) 344 College & Research Libraries July 2001 in many encounters, only one stage of the model was observed. These trends are consistent across both data sets. These trends are also consistent with what might be expected intuitively. Many of the reference encounters in­ cluded questions about location and ac­ cess, and about half the encounters were “one-shot” questions that concerned only one step of the IPS process, prima­ rily location and access. These findings will not surprise many reference librar­ ians, who already feel they spend much of their time giving location directions. The lack of stage 5 synthesis activities also is not surprising because the work­ stations in the reference area do not sup­ port application software such as word processing, database management, spreadsheet, or graphics packages. As Eisenberg and Doug Johnson pointed out, students often use such tools dur­ ing synthesis; without the presence of the appropriate tools, the stage is unlikely to be observed.16 Librarians in wired academic librar­ ies will not be startled to hear that slightly under 20 percent of the questions revealed “system problems,” problems that were outside the scope of the IPS process (usually hardware problems, not information problems) but directly im­ peded it by preventing access to an in­ formation source or extraction of infor­ mation from a source. Implications Although the results of categorizing aca­ demic reference events into the IPS model may be unsurprising, they still can provide insight into the academic refer­ ence process and how best to prepare li­ brarians to meet it. The Eisenberg–Berkowitz IPS model also provides a useful checklist as refer­ ence librarians address specific patron requests. Librarians can use the model to ascertain where the student is in the research process, advise accordingly, and verify that all relevant steps are covered. This is particularly useful for novice ref­ erence librarians who may feel over­ whelmed by the collection itself and the vast range of questions confronting them: it provides an easily accessible “handle” or entry point. Using the IPS model also can prompt librarians to think about their own natu­ ral tendencies in approaching reference work. If the researcher ’s experiences are any indication, using the model to ana­ lyze reference encounters may raise FIGURE 6 Percentage of Encounters with Number of Stages Covered (Set 2) Zero One Two Three Four Five Six Seven Eight Nine http:observed.16 Applying an Information Problem-solving Model to Academic Reference Work 345 awareness of which stages come natu­ rally and which ones may need more at­ tention, not necessarily in specific en­ counters but, rather, in general. Awareness of the IPS model can help librarians keep the reference process on track. For example, when a system or hardware problem impedes a patron, the natural tendency is to try to solve it. Sometimes, however, it is useful (indeed necessary) to step back and look at the problem in IPS terms. For example, if a printer is malfunctioning, the patron can­ not extract information (step 4.2). If the printer cannot be fixed quickly, the real solution from the patron’s point of view is not to fuss with the printer but, rather, to redirect the output to a working printer or, if that is not possible, to reen­ ter the search on a properly functioning workstation and print the results there. Analyzing the problem in IPS terms helps refocus the effort onto the patron’s actual need. One unanticipated implication of this research applies to an aspect of campus planning. As colleges and universities begin to accept information problem- solving models such as the Eisenberg– Berkowitz model, they are taking a closer look at how to promote core information competencies on their campuses. But the current research indicates that one of the areas normally thought to promote these competencies—the library—is, in fact, not providing resources for major stages of the model: the stages having to do with creation and evaluation of the prod­ uct. Synthesis and product evaluation will not occur in the library reference work­ station area until “synthesis tools” (word processing, spreadsheet, database, graphics, or citation management soft­ ware, laser quality printers, and so on) are available there in addition to the more typical library databases. Adding tools such as these to the reference area toolbox is controversial because they add greatly to the support burden. However, if schools are going to com­ bine their library and computing center, as many are, this is a good reason to do so: to create one place on campus where students can work through all the infor­ mation problem-solving steps including brainstorming, locating, using, evaluat­ ing, and synthesizing information. In­ stead of a reference area that focuses only on database searches, schools who wish to emphasize all aspects of information literacy may choose to create an infor­ mation commons that includes tools to create printed or electronic information products. Such a facility would need to be staffed by people who are familiar with all the IPS steps and can support not just database searches, but also other aspects of product synthesis, including applica­ tions software, multimedia production, and effective writing. This may mean pulling together services that ordinarily may reside in different locations such as the library reference area, the writing center, the computer center, and so on.17 Administrative arguments for com­ bining libraries, computing centers, or other campus services often are based on budgetary or logistical considerations: consolidation saves money and space. In practice, however, these savings may be difficult to realize. The current study, al­ though preliminary, indicates that there may be a stronger rationale for collabo­ ration or consolidation, a rationale based on providing better services to students. Future Research The findings of this study suggest sev­ eral promising avenues for future re­ search. One obvious follow-up would be to compare these findings with those of other academic environments, including different kinds of schools. Moreover, these findings could be en­ hanced by combining the observational techniques used here with interviews or other substantive discussion. Follow-up interviews with patrons would provide qualitative data allowing a more in- depth understanding of their perceptions of the reference encounters. For example, it would be particularly interesting to de­ 346 College & Research Libraries July 2001 termine whether the patrons were aware of going through a series of stages dur­ ing their work. In schools with active information lit­ eracy programs, the techniques de­ scribed here could be used as part of a pre- and posttest to help assess the effec­ tiveness of the programs. Conversely, in­ terview techniques could be used to de­ termine whether patrons already had had any information literacy instruction. Finally, given the implications for campus planning discussed above, com­ parable data from different service points on the same campus would provide in­ sight into whether and how students re­ quest and receive aid throughout the IPS process. In schools where computing support is separate from library support, computing help desks probably field many questions related to the synthesis stage, as students rely on application software to produce papers, slides, and so on, just as campus writing centers may focus on task definition, synthesis, and evaluation as they help students tackle major writing assignments. In schools where computing and library support have been combined, the research tech­ niques described here may help evalu­ ate the effectiveness of the combined ser­ vice points in addressing the full spec­ trum of information problem-solving tasks. Notes 1. Richard L. Derr, “Questions: Definitions, Structure, and Classification,” RQ 24 (winter 1984): 186–190. 2. Richard E. Bopp, “History and Varieties of Reference Services,” in Reference and Informa­ tion Services: An Introduction, ed. Richard E. Bopp and Linda C. Smith (Englewood CO: Librar­ ies Unlimited, Inc., 1995). 3. F. Wilfrid Lancaster and Amy J. Warner, “Some Basics of Information Retrieval,” chap. 1 in Information Retrieval Today (Arlington, VA: Information Resources Pr., 1993). 4. Diane M. Brown, “Telephone Reference Questions: A Characterization by Subject, An­ swer Format, and Level of Complexity,” RQ 24 (spring 1985): 290–303. 5. Thomas Childers, Cynthia Lopata, and Brian Stafford, “Measuring the Difficulty of Ref­ erence Questions,” RQ 31 (winter 1991): 237–43. 6. Ellen D. Sutton and Leslie E. Holt, “The Reference Interview,” in Reference and Informa­ tion Services: An introduction, ed. Richard E. Bopp and Linda C. Smith (Englewood CO.: Librar­ ies Unlimited, Inc., 1995). 7. Carol Collier Kuhlthau, “Developing a Model of the Library Search Process: Cognitive and Affective Aspects,” RQ 28 (winter 1988): 232–42. 8. ———, Seeking Meaning: A Process Approach to Library and Information Services (Norwood, N.J.: Ablex, 1993). 9. ———, “Impact of the Information Search Process Model on Library Services,” RQ 34 (fall 1994): 21–26. 10. Ibid., 23. 11. Lynn Kennedy, Charles Cole, and Susan Carter, “Connecting Online Search Strategies and Information Needs: A User-centered, Focus-labeling Approach,” RQ 36 (summer 1997): 562–68, and “The Optimization of Online Searches through the Labeling of a Dynamic, Situa­ tion-dependent Information Need: The Reference Interview and Online Searching for Under­ graduates Doing a Social-Science Assignment,” Information Processing and Management 32 (Nov. 1996): 709–17. 12. Michael B. Eisenberg and Robert E. Berkowitz, Information Problem-solving: The Big Six Skills Approach to Library & Information Skills Instruction (Greenwich, CT.: Ablex, 1990). 13. Mary Warner, “Western Carolina University’s Model of Integrating Information Lit­ eracy: Partnering the First-Year Composition Instructor, Students, and a Personal Librarian,” paper presented at the annual meeting of the Conference on College Composition and Com­ munication, Chicago, Apr. 1–4, 1998. 14. Eisenberg, personal communication, Apr. 20, 1998. 15. Eisenberg and Berkowitz, Information Problem-solving. 16. Michael B. Eisenberg and Doug Johnson, Computer Skills for Information Problem-solving: Learning and Teaching Technology in Context (Syracuse, N.Y.: ERIC Clearinghouse on Informa­ tion and Technology, 1996). ERIC ED 392463. Applying an Information Problem-solving Model to Academic Reference Work 347 17. For a more extensive discussion of integrating research and technology support in li­ braries, see Chris Ferguson, “‘Shaking the Conceptual Foundations,’ Too: Integrating Research and Technology Support for the Next Generation of Information Service,” College & Research Libraries 61 (July 2000): 300–311.