shoaf.p65 124 College & Research Libraries March 2003 124 Using a Professional Moderator in Library Focus Group Research Eric C. Shoaf Eric C. Shoaf leads the Materials Care and Delivery Service Group at Brown University Library; email: ecs@brown.edu. Brown University Library hired a professional marketing and opinion re- search firm to conduct focus group meetings with library users and to provide data analysis. The discussion includes a review of the library literature on focus group use, practical aspects of focus group method- ology, and the benefits of employing professionals where librarian ex- pertise is low. Logistics of focus group preparation, meetings, and re- port are discussed. Findings and lessons learned are presented along with outcomes for the library. sing focus groups to find out about customer needs and ex- pectations has been customary for many years in the adver- tising business, politics, and broadcasting. A focus group is a small set of interested consumers who meet with a moderator/ facilitator to discuss a product or topic in depth. Focus group use in the library set- ting has been steadily documented over the years but has not achieved wide- spread use. This is most likely because costs in dollars and time prevent such a detailed survey when print or online tools can provide similar, though not as de- tailed, results. Yet, borrowing the focus group tool from marketing and advertis- ing professions and applying it in librar- ies can lead to a higher level of expertise in user satisfaction and improvements in library services and programs. In the library setting, focus group use is especially helpful after a print or online survey has been administered because the results can guide the scope of the focus group meetings. Thus, focus group use becomes an additional means of data gathering at a much more in-depth scale, with the potential to provide information about user needs, habits, research, and overall satisfaction with the library. Literature Review Focus group use in libraries, though well documented in library literature, does not appear to be common. The standard guide for moderating focus groups by Thomas L. Greenbaum is useful to those librarians who desire to organize and fa- cilitate their own focus groups.1 It is well written and can be a resource for those who prefer to contract out their focus group survey. Some of the other more useful publications in library literature since 1990 include a how-to guide for administrators by Deborah Leather that focused on possible uses of focus group surveys for developing library building projects and a report of actual focus group use from Purdue University that outlined the library�s experience with focus group research and its practical applications.2,3 Using a Professional Moderator in Library Focus Group Research 125 Notably, the latter were able to accom- plish their focus group research with no monetary outlay and garnered useful re- sults. Given the precarious nature of li- brary funding, this is helpful for those who are contemplating focus group re- search with little funding. Similarly, Victoria Young and Barbara Valentine applied focus group research at their respective institutions to develop bases of information about library use, as did Heather Morrison and Virginia Massey-Burzio.4�7 The information is used for a variety of purposes in these librar- ies, including to document user satisfac- tion and testing program outcomes. The reports show that successful applications of focus group research can be adminis- tered in-house. Morrison is of particular interest because a complete moderator �s script is appended to the report and is a useful example. Elaina Norlin docu- mented a combination of approaches to gathering data on user satisfaction that included focus group use, while Lynn Connaway provided an in-depth look at the structure and use of focus group stud- ies and noted that they can provide a use- ful public relations element.8�10 In particu- lar, the conversational approach of focus group meetings can show users that the library is interested and listening to their needs. Of particular note, in all these pub- lished reports of focus group use in librar- ies, library staff act as moderators/facili- tators for the focus group meetings. Skill or training does not appear to be a quali- fication, only an interest or desire to work with users. Of the literature surveyed, only Elizabeth Parang reported on using a professional consulting firm to moder- ate focus group studies at the University of Nebraska Library.11 However, the uni- versity chose to conduct the data analy- sis in-house instead of having the consult- ing firm do it. Likewise, Nancy Young and Marilyn Seggern used a �neutral ex- perienced moderator� for their library focus group who was a faculty member at the institution, but they also did the data analysis themselves.12 None of those reporting in library lit- erature hired marketing and opinion re- search professionals to both conduct fo- cus group research and provide analysis of the results. For libraries desiring to do focus group research, but for whom lim- ited resources mean the work must be performed in-house, Beryl Glitz�s manual is essential and covers all aspects of the topic in exceptional detail.13 It was refer- enced throughout the focus group plan- ning at Brown University Library, even though an outside firm was chosen even- tually to conduct the focus group re- search. Similarly, Mary Wagner and Suzanne Mahmoodi offered a guide for librarians, but theirs was much more suc- cinct.14 In it, the authors provided a cap- sule approach to planning and imple- menting focus group research, a sort of field guide, and included planning help, lists of possible questions, report forms, debriefing, and evaluation guidelines. Nearly all those reporting in the library literature noted that conducting focus group surveys could be costly, particu- larly when a professional moderator is used. Besides the moderator �s fee, there are location costs, refreshments, and honoraria for participants. Given that there are usually several focus group meetings, the amounts can quickly esca- late to many thousands of dollars. These costs lead some institutions to select a moderator from their staff or from within the institution in order to reduce costs. Sometimes the selected person receives training; other times, he or she relies on a manual or guidebook. However, nearly all those reporting in the literature clearly stated that useful results depend on the skill and experience of the moderator. Group interactions are highly complex and involve different aspects of person- ality, social development, and verbal skills. The most valid and useful results are obtained with a skilled and experi- enced moderator. Methods The Brown University Library formed a User Needs Team (LUNT) in 1997 to ob- 126 College & Research Libraries March 2003 tain and disseminate information about library users� needs and preferences in order to assist library staff and units/ teams in setting priorities, establishing policies, making decisions, developing programs, and providing services. Mem- bership is drawn from a variety of depart- ments and throughout the hierarchy of responsibility. The eight LUNT members demonstrate skills and interest in data gathering and oversee the planning, implementation, data analysis, and re- porting of all types of surveys. Generally, work of the LUNT is split among sub- groups and the larger group acts to orga- nize and oversee the work. LUNT is led by a team leader, and some members have formal training in data-gathering and analysis techniques. However, although some LUNT members were exposed to focus group surveys, none had any direct experience with plan- ning or moderating a focus group. It was this lack of expertise that ultimately led the group to investigate using an outside resource to conduct the focus group sur- veys. At the time that planning for focus group surveys began, LUNT had admin- istered and analyzed the results of print surveys for faculty and graduate students. An undergraduate print survey was planned as well. The decision to investi- gate focus group surveys was informed by a strategic planning report that posi- tioned the library to become more user centered. A need for specific user input in decision making and for evaluation of services, as well as an understanding that print surveys did not contain detailed information about user needs, further drove the notion of developing focus group surveys. Finally, a desire to clarify certain findings in the print surveys to guide future resource allocation decisions pointed to focus group research. A sub- group of LUNT was formed to look into using the focus group tool at the library. The three main parts of managing fo- cus groups are preparation, group meet- ings, and a final analysis report.15 LUNT�s focus group subgroup looked at the lit- erature on focus group applications and quickly decided that expertise to conduct such research was beyond the skills set of library staff, nor could such expertise be developed quickly. A professional li- brarian cannot simply follow the book on focus group surveys and expect good results. The primary concern about focus group research is that the moderator has extraordinary influence over the results, either from conscious or unconscious in- tervention. Also, group dynamics can af- fect and influence the responses of par- ticipants. A skilled hand is needed to bal- ance these forces and maintain an envi- ronment conducive to useful feedback. These skills are only developed after years of experience. The decision to use a pro- fessional moderator to conduct the focus group research was reached easily. Queries were made about possible ex- pertise in focus group applications among faculty members at the university, but no one was available for assistance. Finally, market research firms in the community were analyzed and several were con- tacted. From this group, one firm stood out because its proprietor was a gradu- ate of Brown University and he had a keen interest in the library and using the focus group technique in the library set- ting. A meeting between LUNT and this marketing researcher (MR) proved fruit- ful, and the decision was made to con- tract focus group surveys to his firm. The MR had some thirty-five years� experi- ence in marketing and opinion research, survey analysis, and focus groups. A con- tract was prepared and signed after nor- mal review by and approval from the university�s legal office. The decision to use a contractor for fo- cus group research was not taken lightly. This was not an inexpensive endeavor: The cost was several thousand dollars. How- ever, focus group research must be ap- proached with experience and wisdom and with an understanding of group dy- namics. The focus group moderator must be skilled in drawing critical information from a group of strangers, careful in the Using a Professional Moderator in Library Focus Group Research 127 analysis to differentiate complaining from objective criticism, and savvy enough to present negative observations as targets for change. Lacking these skills and attributes internally, and wanting to do the job right, the LUNT decision to use a professional opinion researcher was the correct one. Preparation LUNT shared the questionnaires and re- sults of the print surveys with the MR. In addition, a series of meetings and tours was set up to familiarize the MR with the library and its services, physical ameni- ties, and collections. During this process, the MR educated LUNT members on the specifics of focus group meetings. As a result, two focus group meetings were planned for faculty and three for gradu- ate students. Logistics were discussed, and it was decided that LUNT members would organize the meeting space, re- freshments, and recruiting of focus group participants. A meeting room was secured. For ease of planning and logistics, each of the five focus group meetings was held in the same library meeting room over a three-week period. Refreshments were ordered from an on-campus vendor and also supplied by LUNT members. There was no cost for the meeting room, and refreshment costs were minimal. Finally, a recruiting plan was developed. The MR provided insight into focus group composition, which helped determine the projected size of the groups. By their nature, focus group meet- ings must be small. Some institutions, by either chance or design, end up with larger groups and find that the size inhibits in- teraction and full participation. Brown�s target was ten to twelve participants, with a minimum of five required for the meet- ing to take place. Recruitment of focus group participants was one of the more difficult parts of the project. As part of the print surveys already completed, respondents were asked if they would be willing to participate in follow- up. The names of those responding in the affirmative were collated into lists. This formed a sizable pool of potential focus group participants. Although the focus group meetings were planned for early in the semester, before midterm break and many exams, it was still not easy to recruit participants who could be counted on to attend. An honorarium in the form of a $25 gift certificate at a local bookstore was of- fered as an incentive, payable only upon attendance. However, the honorarium was not mentioned until after the participant had agreed to take part in order to limit inclusion to those truly interested in tak- ing part. Two recruitment approaches were used, and in each case a recruitment script was developed with the assistance of the MR. Faculty members were telephoned directly and asked to participate. At- tempts were made to get broad represen- tation among various disciplines, but eventually efforts were directed toward securing attendance of a sufficient num- ber of participants. Those accepting the invitation to participate received a follow- up e-mail with complete details. Finally, a reminder telephone call was placed on the day of the focus group meeting. Graduate student participants were contacted first via e-mail and asked to participate. Those expressing interest re- ceived a follow-up e-mail and then were telephoned to confirm attendance. A re- minder telephone call then was made on the day of the meeting. The MR drafted a moderator �s guide based on survey data, orientation tours, and educational meetings. This printed report covered the topics to be discussed in the focus group meetings and outlined talking points for participants. The guide was reviewed and discussed by LUNT, and changes were suggested that were incorporated into the final guide. As part of this process, the decision was made that LUNT members would not attend or take part in the focus group discussions in any w a y. Focus Group Meetings The first focus group meetings were held in the early evening hours, but this was changed to an afternoon time for the fi- 128 College & Research Libraries March 2003 nal meetings. Participants were greeted by a LUNT member who remained on duty, outside the meeting room, for the duration. Each meeting was tape-re- corded by the MR with approval of the participants. Though a room with a two- way mirror is often used for observation in focus group meetings, none was avail- able so there was no observation beyond the MR. No time limit was set for the du- ration of the meetings, but most seemed to naturally wind down after seventy or eighty minutes. The refreshments ap- peared to be less of a draw to faculty fo- cus group meetings, more so for gradu- ate students. Meeting time was found to be much more important to both groups and was adjusted after the first two meet- ings. Despite persistent recruitment endeav- ors, yields for the faculty focus group meetings were somewhat disappointing. Each of the two faculty focus group meet- ings had only five participants, the bare minimum required. Some participants phoned or e-mailed regrets; others sim- ply failed to show up. Nonetheless, the MR reported lively and useful focus group meetings with these small groups. Yields were much better for the gradu- ate student focus group meetings, with nine in the first, nine in the second, and ten in the third. As previously pointed out, moving the meeting time from early evening to afternoon appeared to play a major role in greater participation. Final Analysis Report Two months after the focus group meet- ings, the MR delivered a written report. The MR�s final report is considered the complete set of results for all the focus group meetings and is filled with infor- mation. The report included an outline of the purpose and strategy behind the fo- cus group research, descriptions of the participating groups, an introduction to the findings, specific findings from par- ticipant discussions, as well as commen- tary on the focus group meetings and their validity and on the findings and how they may be applied. In the final report, the MR grouped the participants� comments and observations into themes that included the library�s physical environment, services and poli- cies, support services, and holdings. This organization facilitated better under- standing of the document. In addition, more than a hundred direct quotes from the focus group participants helped paint a picture of user perceptions of the library. Many of the quotes were passionate and highly charged with negativity. But they also represented input well beyond what any sort of print survey could provide because they were made in the context of discussions on issues and concerns about library services. The final report was reviewed by LUNT members, and a series of meetings was arranged with the MR to discuss and elaborate on the findings. This is an im- portant step because, by their nature, fo- cus group meetings engender some nega- tive comments and observations. In sub- sequent meetings, the MR provided back- ground and context for understanding the comments and observations and for trans- lating them into action proposals. After much discussion about concerns with negative aspects of the document, the fi- nal written report was distributed to all library staff and a summary of the find- ings was mounted on the library Web site. Results The five focus group meetings provided clear results to LUNT and the Brown li- brary at large. High levels of dissatisfac- tion with the library�s collections and its physical environment were mentioned consistently and with passion by many of the focus group participants. To a lesser extent, concerns about services and sup- port also were apparent. However, many on the library staff were reluctant to ac- cept the focus group findings because they were so much more negative than those reported by other surveys that had been conducted over the years. Results of the focus group surveys pro- vide a radically different view of the Brown library compared with earlier Using a Professional Moderator in Library Focus Group Research 129 print surveys. For example, in a 1999 print survey of graduate students, 82 percent of the respondents reported that they were satisfied or very satisfied with the library. A faculty survey from the same year showed that 85 percent were either satisfied or very satisfied with the library, whereas an undergraduate print survey conducted in 2001 had a surprising 93 percent of respondents satisfied or very satisfied with the library. Taken on face value, one might speculate that overall satisfaction with the library was quite high. In fact, the opposite was the case. In some ways, the findings of the fo- cus group meetings were not unlike those of any other library survey. Participants spoke of the need for more books and materials, difficulty finding books on shelves, problems with photocopiers, and hours of operation. But after probing by the MR who pointedly asked about spe- cifics, clear patterns of deep concern be- gan to emerge. Dissatisfaction about many issues can be traced to the physical environment or support services, areas over which the library asserts minimal control. However, nearly all focus group participants voiced concern about the library�s holdings. Several library policies were singled out as contributing to dis- satisfaction. Negative comments far out- weighed positive in the MR�s final report, and the picture painted was one of key library users with their needs unmet. However, the report was not entirely negative. Praise also was given to many library functions. But more interestingly, it was the MR�s sense that faculty spend far less time in the library than graduate students and depend on the library much less for research than the student do. This can have implications in planning future library services. One conclusion to be drawn in com- paring the negative focus group results with the mostly positive print survey re- sults is that the focus group participants were dominated by outspoken complain- ers. As such, the results would not be valid for the wider user population. How- ever, this is incorrect. In spring 2002, the Brown library participated in the LibQUAL survey, joining 162 participat- ing libraries in administering this instru- ment to a large sample of faculty, gradu- ate students, and undergraduates. Brown received just over 1,100 completed sur- veys, which represents a large percentage of the user population. LibQUAL is a tool for measuring library users� perceptions of service quality that identifies gaps between minimum and desired expectations of service and per- ceived levels of service. The results ob- tained were rather different from any pre- vious survey the library had performed on its own. But staff at the Brown libraries were disappointed with the LibQUAL re- sults because perceived levels of quality were low in many areas and the gaps be- tween desired expectations and perceived levels of service were high. However, it was clear that results from the LibQUAL survey underscore those reported in the MR�s report on focus group meetings. Comparing the results of Brown�s fo- cus group meetings with those obtained at other institutions is difficult because each focus group meeting is unique. However, several themes emerged from the Brown experience that compare well with those from institutions reported on in the literature review section. 1. Focus group meetings usually cover a wider range of issues and con- cerns because even though the MR has a script to follow, discussion is driven by participants and deviations from the script are allowed and can lead to more input by participants. 2. Correct use of the focus group as a survey tool is important. Focus groups cannot be used to gather qualitative data. Care must be taken when generalizing results to a larger population without cor- roborating data. 3. The results are directly tied to the MR�s experience and to a carefully man- aged discussion. 4. The focus group meetings can serve as a public relations tool. Participants get to provide targeted feedback and appre- ciate the opportunity to do so. 130 College & Research Libraries March 2003 5. Focus group use works well as an adjunct to other, more general surveys as a way to pinpoint issues and concerns. The MR�s skill is paramount to valid results. An unfortunate byproduct of group dynamics is that participants often defer to the leader on issues under dis- cussion. It is not the nature of most people to be negative, but it is in their nature to want to be liked. This influences the re- sponses of focus group participants, who may respond positively when speaking to a librarian about library issues, but who also may harbor deep concerns about those same issues. It may be courtesy or friendship that prevents the concerns from surfacing, or it could be that a group dynamic creates �warm and fuzzy� feel- ings that color responses. The danger to valid results is that responses can be skewed by the social interaction between MR and group and by the group dynam- ics of the participants. Brown�s MR specifically announced at each focus group meeting that he was a market researcher and had no affiliation with the library beyond that fact. As part of his script, he asked the participants to provide open and honest feedback on the library�s programs and services. He also asked that they not concern themselves with his opinion or even that of others in the discussion. In short, every effort was made to avoid instances of influence or misunderstanding that could lead to false results. In the two years following the focus group meetings, the library received a major increase to the base acquisitions budget, funding for cosmetic enhance- ments, and acquisition of a building for library storage. Although none of these results is the direct result of focus group meetings, there is no question that the information obtained by LUNT in print and focus group surveys supplied data that helped the library build a case with university administrators to fund specific projects for improvements in the library�s collections and physical environment. Internally, the library staff has exam- ined several services and some policies in light of observations from focus group participants. New shelving practices cut the time to re-shelve a book after use. Tracking newly acquired materials through the acquisitions/cataloging/pro- cessing function is enhanced. Support services are improved. More important, in planning and discussions about the library�s future direction, the focus group results are nearly always referenced. They continue to have a profound influence in framing user needs and desires. In light of recent LibQUAL results, the library has confirmed many of the issues of concern raised in focus group meetings. Such was the success of the focus group project that the library expects to use focus group meetings to survey undergraduate stu- dents in the coming year. Lessons Learned You Get What You Pay For Contracting out the focus group survey to a professional firm was worth the costs. The library and LUNT could not have achieved its results with staff trying to become focus group moderators nor could the library have analyzed the data from the focus group meetings in such a way to produce useful results. A market- ing and opinion research professional is the best qualified for such work. These individuals have the skills and abilities to elicit responses without influencing participants. Do not be afraid to negoti- ate the fee or to try and undertake parts of the project that you know you can ac- complish. Brown was able to minimize expenses somewhat by managing the lo- gistics of the focus group meetings: the recruiting of participants, booking a meet- ing room, and preparing refreshments. But do not try to do what you are not trained to do; instead, hire a professional. Divide the Work The existence of LUNT to organize and guide the process cannot be overstated. All eight team members contributed to the focus group work and made the project a success. In addition, group dis- cussion guided the process and informed Using a Professional Moderator in Library Focus Group Research 131 many of the decisions that were made. No single library representative could man- age this sort of project and achieve posi- tive results. Planning is Everything The busiest and most intensive period is the period leading up to the actual focus group meetings. Careful planning en- sures the best outcome. But learn from your mistakes. When it became clear that the evening meeting time negatively im- pacted focus group attendance, Brown quickly changed to an afternoon meeting time. The Focus Group Moderator is Key The person leading the focus group meet- ings must not only be skilled and experi- enced (see above) but also must possess a knowledge of the subject under discus- sion and the ability to react quickly and change direction as dictated by the col- lective mood of the group. Consider care- fully whether you want to risk obtaining biased results from having a librarian act as focus group moderator. Help to Frame the Moderator�s Guide The moderator �s guide is produced be- fore the focus group meetings and serves as a script for the moderator. It is the best opportunity for the library staff to influ- ence the content and direction of the fo- cus group meeting and to be sure that is- sues needing clarification are discussed. Although focus group meetings often deviate from the script and should do so when properly moderated, all points in the guide will generally be covered. It is important to make sure the completed guide covers all areas of concern that need clarification. Recruiting is Difficult Recruiting is hard work, period. Library staff can do it, but it means lots of remind- ers and follow-up, lots of telephone calls and e-mails. Although this approach worked for the Brown library, other in- stitutions have reported success in sim- ply asking people arriving at the library if they would mind talking about their library experiences. Consider the options. Criticism is Painful It is part of the nature of focus groups to draw a lot of criticism from participants, particularly when the participants feel at ease with the MR and speak freely about their thoughts and concerns. It is espe- cially painful to those on the staff who read the final report and feel singled out. However, careful elucidation of the ob- jects of concern by a wise moderator can minimize the pain and turn the criticism into areas for improvement. Satisfaction is a Job Well Done Results may be the best measure of suc- cess. Focus group data provide the library with a set of actionable proposals for im- provement and also help document clear needs to university administration. Notes 1. Thomas L. Greenbaum, Moderating Focus Groups: A Practical Guide for Group Facilitation, revised ed. (Thousand Oaks, Calif.: Sage, 2000). 2. Deborah Leather, �How the Focus Group Technique Can Strengthen the Development of a Building Program,� Library Administration & Management 4 (spring 1990): 92�95. 3. Richard Widdows, Tia Hensler, and Marlaya Wynott, �The Focus Group Interview: A Method for Assessing Users� Evaluation of Library Services,� College & Research Libraries 52 (July 1991): 353�59. 4. Victoria Young, �Focus on Focus Groups,� College & Research Libraries News 7 (July/Aug. 1993): 391�94. 5. Barbara Valentine, �Undergraduate Research Behavior: Using Focus Groups to Generate Theory,� Journal of Academic Librarianship 19 (Nov. 1993): 300�304. 6. Heather Morrison, �Information Literacy Skills: An Exploratory Focus Group Study of Student Perceptions,� Research Strategies 15 (winter 1997): 4�17. 7. Virginia Massey-Burzio, �From the Other Side of the Reference Desk: A Focus Group Study,� Journal of Academic Librarianship 24 (May 1998): 208�15. 132 College & Research Libraries March 2003 8. Elaina Norlin, �Reference Evaluation: A Three-Step Approach,� College & Research Librar- ies 61 (Nov. 2000): 546�53. 9. Lynn Connaway, �Focus Group Interviews,� Library Administration & Management 10 (fall 1996): 231�39. 10. Lynn Connaway, Debra Johnson, and Susan Searing, �Online Catalogs from the Users� Perspective,� College & Research Libraries 58 (Sept. 1997): 403�20. 11. Elizabeth Parang, �Using Focus Groups to Match User Expectations with Library Con- straints,� Serials Librarian 31 (1997): 335�39. 12. Nancy Young and Marilyn Seggern, �General Information Seeking in Changing Times: A Focus Group Study,� Reference and User Services Quarterly 41 (winter 2001): 159�68. 13. Beryl Glitz, Focus Groups for Libraries and Librarians (New York: Forbes Custom Publica- tions, 1998). 14. Mary Wagner and Suzanne Mahmoodi, A Focus Group Interview Manual (Chicago: ALA, 1994). 15. Greenbaum, Moderating Focus Groups.