Kayongo.indd 493 Identifying Student and Faculty Needs through LibQUAL+™: An Analysis of Qualitative Survey Comments Sherri Jones and Jessica Kayongo Sherri Jones is Head of Information, Research & Instructional Services, University Libraries of Notre Dame; e-mail: sedwards@nd.edu; and Jessica Kayongo is Reference Librarian, Hesburgh Library, University of Notre Dame; e-mail: jkayongo@nd.edu. The LibQUAL+™ survey allows libraries to collect quantitative as well as qualitative data concerning their users. Comments that survey-takers provide contain a wealth of information that shed light on the quantita- tive responses they give to questions. This article reports on a study conducted by the authors to analyze the written comments provided by faculty and students participating in the 2006 LibQUAL+™ survey at the University of Notre Dame to determine, in a systematic way, what issues are most important to users and to see if there are differences among the three primary user groups. The authors were interested in whether the analysis of the comments would reveal the same issues that the quantitative data had. ibrary users participating in the LibQUAL+™ 2006 survey at the University of Notre Dame indicated that issues of Information Control were most important to them and that this was also the area where the library was not doing so well in meeting their needs. The authors exam- ined the respondents’ comments to deter- mine the differences between user groups, finding that undergraduates, graduates, and faculty differed in their desires and in the importance they placed on various library services. The one common area for all three groups was in desiring and placing great importance on having “print and/or electronic journal collections I re- quire for my work.” Evaluating both the quantitative and qualitative results of the LibQUAL+ survey has resulted in a bet- ter understanding of users’ needs and a clearer picture of where a�ention should be focused for service improvements. Background The University of Notre Dame is a com- prehensive research university located in South Bend, Indiana. The university has four undergraduate colleges (Arts and Let- ters, Engineering, Science, and Business), the Law School, the School of Architecture, the Graduate School, ten major research institutes, and over forty centers and special programs. The Graduate School offers forty-three master’s and twenty-two doctoral degree programs in twenty-eight university departments and institutes. Demographic data in 2006 showed 11,417 students and 1,586 faculty.1 The University Libraries of Notre Dame serves the entire 494 College & Research Libraries November 2008 campus, with the exception of the Law School, and is a member of the Association of Research Libraries, an organization of 123 research libraries from comprehen- sive, research-extensive institutions in the United States and Canada.2 The Notre Dame Libraries have used a number of informal assessment tools such as focus groups and usability studies over the years to understand the needs of users and to assess how those needs are being met. Understanding the needs of library users can be challenging and is complicated by the differences in needs and expectations among different user groups. Faced with this challenge, the Libraries recognized the need for a more formal, sys- tematic assessment tool, and LibQUAL+™ provided such a tool. LibQUAL+™ is a tool administered by the Association of TABLE 1 LibQual+™ Survey Structure Affect of Service (AS) Information Control (IC) Library as Place (LP) Employees who instill confidence in users(AS-1) Making electronic resources accessible from my home or office (IC-1) Library space that inspires study and learning (LP-1) Giving users individual attention (AS-2) A library Web site enabling me to locate information on my own (IC-2) Quiet space for individual activities (LP-2) Employees who are consistently courteous (AS-3) Printed library materials I need for my work (IC-3) A comfortable and inviting location (LP-3) Readiness to respond to users’ questions (AS-4) The electronic information resources I need (IC-4) A getaway for study, learning, or research (LP-4) Employees who have the knowledge to answer user questions (AS-5) Modern equipment that lets me easily access needed information (IC-5) Community space for group learning and group study (LP-5) Employees who deal with users in a caring fashion (AS-6) Easy-to-use access tools that allow me to find things on my own (IC-6) Employees who understand the needs of their users (AS-7) Making information easily accessible for independent use (IC-7) Willingness to help users (AS-8) Print and/or electronic journal collections I require for my work (IC-8) Dependability in handling users’ service problems (AS-9) University of Notre Dame Local Questions from LibQUAL+™ Pool of 100+ Library Orientation/Instruction Sessions Making me aware of library resources and services Accuracy in catalog, borrowing, and overdue records Timely document delivery/interlibrary loan Adequate Hours of Service An Analysis of Qualitative Survey Comments 495 Research Libraries to measure library users’ perception of library service quality and to help libraries identify service areas needing improvement.3 The LibQUAL+™ survey is part of ARL’s New Measures Initiative, which seeks to explore innovative ways for libraries to measure their value instead of using the more traditional values such as the size of library collections or number of patrons served.4 The University of Notre Dame Librar- ies first participated in LibQUAL+™ in 2002 and then again in 2006. Both the 2002 and 2006 survey targeted the three major user groups—teaching and research faculty, graduate students, and undergraduates. About the Survey In April 2006, all University of Notre Dame faculty, graduate students, and undergraduates received an e-mail invit- ing them to participate in LibQUAL+™ by completing a Web-based survey. Over the next three weeks, follow-up e-mails were sent each week reminding them to take the survey. As an incentive to participate, anyone completing the survey could en- ter into a random drawing for one of six video iPods.5 The LibQUAL+™ survey gathers brief demographic information, including sta- tus, discipline, sex, and age group. The core of the survey contains twenty-two questions that relate to three dimensions of library service quality: Affect of Ser- vice, Library as a Place, and Information Control (table 1). The eight questions in the column on Information Control (IC) measure how users would like to interact with the modern library and include ease of navigation, modern equipment, scope, timeliness and convenience, and self-reli- ance. Affect of Service (AS) contains nine questions meant to assess responsiveness, assurance, reliability, and empathy of library employees; while Library as Place (LP) includes five questions that measure the symbolic value of the library, useful- ness of space, and the library as a refuge for work or study.6 Libraries participating in the survey were also given the option to add five “local” questions selected by local survey administrators from a pool of more than 100 additional questions pro- vided by LibQUAL+™ administrators.7 For each question, respondents were asked to use a scale of 1–9 to indicate their minimum acceptable service level, their desired service level, and their perception of actual service provided by the library. The survey also asked three questions relating to general satisfaction and three questions regarding library usage, in ad- dition to the twenty-two core questions and five local questions. Last, respondents were invited to add wri�en comments at the end of the survey. Summary Survey Results A total of 2,737 people completed the survey, including 1,850 undergraduates, 553 graduate students, and 229 faculty. Over 1,000 participants provided wri�en comments at the end of the survey.8 The response rate was 22.5 percent, and the results were representative (that is, re- spondent percentages by user group very closely mirrored campus demographics). A representative result is more important than a large response rate in that the re- sponses likely are an accurate reflection of a particular population.9 Mean scores for the minimum, desired, and perceived levels of service were cal- culated for each of the twenty-two core questions, the three service dimensions, and the five local questions. LibQUAL+™ methodology offers the ability to measure the gaps or differences between a user’s de- sired or minimum service expectations and their actual perceived level of service. Service Adequacy Gap A Service Adequacy Gap score is calcu- lated by subtracting the minimum score from the perceived score on each ques- tion. The Service Adequacy Gap is an indication of the extent to which libraries are meeting the minimum expectations of their users. A negative service adequacy gap score indicates that the user’s per- 496 College & Research Libraries November 2008 ceived level of service quality is below his or her minimum level of service and can be used by Libraries to identify areas of service needing improvement.10 In the aggregate, Notre Dame Librar- ies met patron expectations for library services in most areas.11 The top three areas where the quality of service was perceived as best overall by Notre Dame faculty and students, based on service adequacy gap scores (perceived minus minimum), were: 1. Library orientation/instruction ses- sions (optional local question) 2. Employees who are consistently courteous (AS) 3. Giving users individual a�ention (AS) Even though most participants were generally satisfied with the service being provided, negative service adequacy gap scores indicated that there was room for improvement in several areas.12 The top three areas needing improvement, based on service adequacy gap scores, for each user group were in the area of “Informa- tion Control” and are shown in table 2. Desired Service Level LibQUAL+™ also allows for identifica- tion of services most important to library users. The “desired” service level can be thought of as an indicator of the level of importance users a�ach to the various dimensions of service measured through LibQUAL™.13 In table 3, the questions are in ranked order according to the average scores for the “desired” level of service for all users. Below are the five most important services for each of the three major user groups, based on mean average scores for the “desired” level of service. Clearly, issues relating to information control are important to all users and are also the areas that cause them the most dissatisfaction. Undergraduate students 1. Making electronic resources acces- sible from my home or office (IC-1) TABLE 2 University of Notre Dame 2006 LibQUAL+™ Survey: Top Service Areas Needing Improvement Based on Service Adequacy Gap Scores User Group Area Needing Improvement Service Dimension Undergraduate Modern equipment that lets me easily access needed information (IC-5) Information Control Print and/or electronic journal collections I require for my work (IC-8) Information Control Easy to use access tools that allow me to find things on my own (IC-6) Information Control Graduate Students Print and/or electronic journal collections I require for my work (IC-8) Information Control The electronic information resources I need (IC-4) Information Control A library web site enabling me to locate information on my own (IC-2) Information Control Faculty Print and/or electronic journal collections I require for my work (IC-8) Information Control Printed library material I need for my work (IC-3) Information Control A library web site enabling me to locate information on my own. (IC-2) Information Control An Analysis of Qualitative Survey Comments 497 2. Modern equipment that lets me easily access needed information (IC-5) 3. Adequate hours of service (Local question) 4. A library Web site enabling me to locate information on my own (IC-2) 5. Print and/or electronic journal col- lections I require for my work (IC-8) Graduate students 1. Print and/or electronic journal col- lections I require for my work (IC-8) 2. A library Web site enabling me to locate information on my own (IC-2) 3. The electronic information resourc- es I need (IC-4) 4. Making electronic resources acces- sible from my home or office (IC-1) 5. Easy-to-use access tools that allow me to find things on my own (IC-6) Faculty 1. Print and/or electronic journal col- lections I require for my work (IC-8) 2. A library Web site enabling me to locate information on my own (IC-2) 3. Timely interlibrary loan/document delivery (Local question) 4. The electronic information resourc- es I need (IC-4) 5. Easy-to-use access tools that allow me to find things on my own (IC-6) Analysis of Survey Comments All survey participants are invited to add wri�en comments at the end of the LibQUAL+™ survey. The comments are collected by ARL and delivered to each participating institution as text files, pro- viding institutions with another valuable source of information in addition to the quantitative survey data. The analysis of qualitative data informs the understanding of quantitative data. As Jankowska et al. pointed out in their study of graduate student comments, TABLE 3 University of Notre Dame 2006 LibQUAL+™ Survey: Top 10 Services Based on Desired Service Level Ratings Dimension Question Text Rating Information Control Making electronic resources accessible from my home or office (IC-1) 8.29 Information Control Print and/or electronic journal collections I require for my work (IC-8) 8.28 Information Control A library web site enabling me to locate information on my own (IC-2) 8.25 Information Control Modern equipment that lets me easily access needed information (IC-5) 8.22 Information Control Easy-to-use access tools that allow me to find things on my own (IC-6) 8.22 Information Control The electronic information resources I need (IC-4) 8.21 Information Control Making information easily accessible for independent use (IC-7) 8.13 Local Question Adequate hours of service (Local question) 8.12 Local Question Accuracy in the catalog, borrowing, and overdue records (Local question) 8.05 Information Control The printed library materials I need for my work (IC-3) 8.01 Mean Rating = 1 (lowest) – 9 (highest) n = 2737 498 College & Research Libraries November 2008 “Qualitative data … drawn from survey comments … provide richness and con- text that add life to the numbers and meat to the bones of the quantitative data.” Further, survey administrators should feel an obligation to carefully consider user comments individually, as well as in context, because the effort put forth in providing wri�en comments “indicates a certain amount of enthusiasm or frustra- tion” on the user’s part.20 Over 1,000 Notre Dame faculty and stu- dents, or more than one in every three par- ticipants, provided wri�en comments. Of the 1,850 undergraduates who completed the survey, 661, or 35.7 percent, provided wri�en comments; 231 of the 553 graduate students (41.7%) completing the survey provided wri�en comments, and over half of the faculty (118 out of 229, or 51.5%) pro- vided wri�en comments. Figure 1 shows the number of comments received from each user group, compared to the number of people completing the survey. The authors decided to take a closer look at the 1,000+ wri�en comments to get a be�er understanding of the issues that were most important to University of Notre Dame library users and to deter- mine if the information gleaned from the comments supported what was learned from the quantitative data. The authors hypothesized that an examination of the free text comments would result in the identification of the same major is- sues that emerged from the quantitative analysis. Literature Review LibQUAL+™ is a useful tool for comparing library users both internal and external to any particular library. This study focused on the former, and found, as studies and surveys outside the LibQUAL+™ context have, that academic library users’ needs and their expectations of the library vary among subgroups. One would expect as much because of the different demands being made on undergraduate students, graduate students, and faculty. Ellen Hitchingham and Donald Ken- ney, from LibQUAL+™ participant Vir- ginia Tech, observed that “undergradu- ates, graduate students, and faculty con- stituents cannot be considered together to create one homogeneous entity called ‘our users’ because their perceptions of some library services are very alike but very different for other services.”14 Steve Hiller, reporting on a University of Washington’s user survey, found some similarities among graduate students FIGURE 1 University of Notre Dame 2006 LibQUAL+™ Survey: Participation by User Group 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Undergraduates Graduates Faculty User Group # of survey respondents # of respondents providing comments An Analysis of Qualitative Survey Comments 499 and faculty. Those similarities were in the following areas: the importance of print journals, use of the library remotely from an office, the values of libraries, and satisfaction in general. He also found what he described as significant differ- ences. “Faculty and graduate students in the sciences-engineering and health sci- ences were more likely to use the library remotely rather than visit, view desktop delivery as the highest priority for library support, and value journals (print and electronic) far higher than other resources such as books, archival resources, etc.”15 Brown University conducted user studies utilizing focus groups. As Eric Shoaf indicated in his article, library us- age findings were similar at Brown, in that faculty were less inclined to visit the library than graduate students were.16 At the University of Iowa, Carle�e Washing- ton-Hoagland and Leo Clougherty found that faculty wanted more print books and journals, in addition to an increase in electronic journals and remote access to the collection.17 Some studies have reported on user differences as viewed through the LibQUAL+™ survey administered at their institutions. Maria Anna Jankowska et al. found that “[g]raduate students and faculty have high minimal levels of acceptable ser- vice and desired service in the information control dimension” and “undergraduates have the highest levels of both minimal acceptable service and desired service in the library as place dimension.”18 OhioLINK LibQUAL+™ results, as de- scribed by Jeff Ga�en, revealed that their graduate students were the least satisfied group in regard to “their perceptions of service quality in relation to minimum expectations, especially on access and collection content issues as reflected in the ‘access to information’ dimension and the five OhioLINK questions.”19 This article reports on a study con- ducted by the authors to analyze the writ- ten comments provided by faculty and students participating in the LibQUAL+™ 2006 survey at the University of Notre Dame to determine what issues are most important to their users among the three primary user groups (faculty, graduate students, and undergraduates) and to test the hypothesis that an analysis of the free text comments would result in the identification of the same major issues and service manifestations that emerged from the quantitative analysis of the Ser- vice Adequacy Gap scores and the scores for the Desired Service levels. Methodology The survey comments were imported into Excel and coded by user group and discipline. Both researchers indepen- dently reviewed all the comments. Then one researcher conducted the coding, as described below, with a review by the sec- ond researcher to ensure both consistency and agreement on coding decisions. To make the best use of the wealth of information contained in each comment, a systematic analysis of the comments was made. Each comment was analyzed sentence by sentence and mapped to one of the 22 core questions or 5 local questions and to one of the three service dimensions. For purposes of this study, the five local questions were included as part of the “Information Control” dimen- sion, even though one might argue that they are relevant to other dimensions. For instance, “Library Orientation/Instruc- tion Sessions” and “Timely Document Delivery/Interlibrary Loan” are relevant to both Information Control and Affect of Service, while “Adequate Hours of Service” is relevant to both Information Control and Library as Place. Since a large number of the comments contained several different “observations” or “concepts,” many were mapped to more than one question. For example, one comment might address the library’s Web site and then shi� to talking about the li- brary’s hours of operation. This comment would be mapped twice, first to question IC-2, “A library Web site enabling me to locate information on my own,” and also to the local question, “Adequate hours of 500 College & Research Libraries November 2008 service.” This comment ap- peared twice in the Excel data- base and was counted twice. Analysis of the 1,010 com- ments resulted in 1,133 discrete observations or concepts, which were mapped to one of the 27 questions. Thirty com- ments were about the survey instrument alone and were excluded from analysis. Furthermore, 118 comments were either too broad or not specific enough to map to one of the core questions. For instance, several comments expressed overall sat- isfaction with the library or included ex- pressions of thanks but were not detailed enough to map to a specific question. Mapping each observation to one of the 27 questions and to one of the three service dimensions was designed to al- low for easy identification of those issues most important to users and also allow a comparison of these results with the quantitative analyses described earlier. Of the 1,133 observations, 763 (or 67%) fell into the area of “Information Control,” with 237 falling into the area of “Affect of Service” and 133 comments related to “Library as Place.” While not surprising, these numbers support the results from the quantitative analysis and confirm the im- portance of issues of “information control,” such as ease of use, scope of the collection, access, and timeliness, to our users. Discussion Issues of Greatest Concern for Undergraduates Undergraduates provided 641 observa- tions about the library. The Information Control questions drew the largest number of comments (416 out of 641), followed by “Affect of Service” and “Library as Place.” Nearly one-fourth of the comments dealt specifically with the local question, “library hours of service.” Clearly, ubiquity and ease of access to the library is important to undergraduates, who want the library to be open more hours, many wanting twenty-four-hour access. While some undergraduates in- dicated that they needed longer hours in order to access specialized service points (for instance, to be able to watch videos in our Audiovisual Center or to check out books from the Reserve Book Room), most of the students wanted longer hours of operation for studying. The following comment was typical of most comments concerning library hours: “My biggest, and crucial, complaint is that at least part of the library really needs to be open later than 2 am—preferably 24 hours. There is a SERIOUS LACK of 24- hour study space on campus, especially for off-campus dwellers, let alone space that is quiet and well lit. Having even a single floor remain open all hours would be a HUGE help.” While electronic access to resources has made library hours less important to students for purposes of research, access to the physical library space is evidently still a concern for undergraduates and shows that the library as a place is still important. “Willingness to help users” is another aspect of library service that is important to undergraduates, as was evidenced by the 62 observations made. Most of the comments made by undergraduates about library staff and their willingness to help users were positive. For example, TABLE 4 University of Notre Dame 2006 LibQUAL+™ Survey: Number of comments by User Group and Library Service Dimension Service Dimension Undergraduates Graduate Students Faculty Totals Information Control 416 225 122 763 Library as Place 89 32 12 133 Affect of Service 136 58 43 237 Totals 641 315 177 1133 An Analysis of Qualitative Survey Comments 501 one student wrote that “The library staff is incredibly helpful and this is very much appreciated. The topic/research librarians are an unutilized resource that should be made more well known to the student population. Many people do not know how helpful these men and women can be when starting a research project.” Similarly, another undergraduate stated that “During my 4 years at Notre Dame, I have found the Hesburgh Library employees to be extremely helpful in teaching me how to access all of the op- portunities which this wonderful library provides. Thank you very much.” These comments are representative of many of TABLE 5 University of Notre Dame 2006 LibQUAL+™ Survey: Distribution of Comments by Undergraduate Students Question Number of Comments Adequate hours of service (Local Question 5) 160 Willingness to help users (AS-8) 62 Printed library materials I need for my work (IC-3) 57 A Comfortable and inviting location (LP-3) 35 Print and/or electronic journal collections I require for my work (IC-8) 34 Making information easily accessible for independent use (IC-7) 34 Easy to use access tools that allow me to find things on my own (IC-6) 29 Employees who deal with users in a caring fashion (AS-6) 24 Quiet Space for Individual Activities (LP-2) 21 Modern equipment that lets me easily access needed information (IC-5) 20 Community space for group learning and group study (LP-5) 19 Library orientation/instruction sessions (Local Question 1) 18 Timely document delivery/interlibrary loan (Local Question 4) 17 Employees who are consistently courteous (AS-3) 16 The electronic information resources I need (IC-4) 14 Accuracy in catalog, borrowing and overdue records (Local Question 3) 13 Employees who have the knowledge to answer user questions (AS-5) 13 A library web site enabling me to locate information on my own (IC-2) 12 Dependability in handling users’ service problems (AS-9) 10 Library Space that inspires study and learning (LP-1) 8 Making me aware of library resources and services (Local Question 2) 6 A getaway for study, learning, or research (LP-4) 6 Giving users individual attention (AS-2) 5 Readiness to respond to users questions (As-4) 3 Employees who understand the needs of their users (AS-7) 3 Making electronic resources accessible from my home or office (IC-1) 2 Employees who instill confidence in users (AS-1) 0 Total 641 502 College & Research Libraries November 2008 the comments made by our students and should serve as a reminder to library staff of the importance in providing good customer service. Issues of Greatest Concern for Graduates Graduate students provided 315 observa- tions about the library. If the number of comments is an indication of importance, the most important service for graduate students is “Print and/or electronic jour- nal collections I require for my work” (IC-8), followed by “Printed library mate- rials I need for my work” (IC-3). Like the undergraduates, the Information Control questions drew the largest number of comments (71.4%), followed by Affect of Service (18.4%), then Library as Place, with 10.2 percent of the comments. Despite heavy investments by the Libraries in electronic resources over the past several years, graduate students across all disciplines are asking for more electronic access. As one graduate student stated, “I think the problem with the li- brary is not the service personnel, but the access to online journals. In the sciences, these journals are our lifeline, and quite o�en I have run into the problem that I find a fantastic article, but I can’t access it because we don’t have that particular subscription.” Similarly, another student wrote, “Please increase the availability of electronic journals because that is the future of my academic field and many others… printed materials will become less and less important as we head into the future. I hope the library is ready for this very difficult challenge.” While electronic access is increasingly important, graduate students still recog- nize and appreciate the value of the print collection. One graduate student wrote, “Please remember that technological devices are not equivalent to informa- tion. Only with adequate holdings of or access to monographs and journals can a library remain a library.” Another one stated that, “Electronic books and journals are not a suitable substitute for printed materials.” One student said that the library owned basically every book he had ever looked for, but that, most of the time, the books were checked out, making the book “totally inaccessible.” In fact, sev- eral graduate students complained about books always being checked out and expressed frustration over the library’s recall policy. Similarly, graduate students complained about the accuracy of catalog records, which would indicate a book was available when the book was not on the shelf. One graduate student writes, “I’ve frequently had problems locating books that the catalog indicates are available. Conversely, I’ve found books on the shelf, which the catalog indicates as checked out or missing.” Faculty and undergraduates voiced similar complaints. These findings support those of Cook and Heath, who, in conducting a series of 60 interviews with users of research libraries across North America in spring 2000, found that the most frequently occurring complaint was the “unavailability of books found in the catalog and noted as available.”20 Issues of Greatest Concern for Faculty Over half of the faculty (118 out of 229) completing the survey included wri�en comments. Within these 118 comments there were 177 distinct observations or con- cepts. More than two thirds of the concepts were mapped to the “Information Control” dimension. Results for faculty mirrored those of graduate students in the first two areas of greatest concern: “Print and/or electronic journal collections I require for my work” (IC-8) and “Printed Library materials I need for my work” (IC-3). Faculty members were particularly vo- cal when expressing their dissatisfaction with the collection. One faculty member in the College of Science described the lack of subscriptions to high-impact sci- entific journals as “disgraceful,” while another faculty member in the College of Engineering described the university as being in a “crisis with respect to having adequate research support for journals and technical books.” An Analysis of Qualitative Survey Comments 503 larly disappointed in the shallowness of Notre Dame’s collection.” Despite the negative comments about the collection, most faculty were apprecia- tive of the library staff’s efforts to provide library services and were quick to com- ment on the good service, despite their criticism of the collection. This comment While acknowledging the budget woes facing the library, faculty nonetheless expressed dissatisfaction with the size and scope of the collection. One faculty member in Arts & Le�ers wrote, “I real- ize that the library at Notre Dame, like all university libraries, is under tremendous financial pressure. However, I am regu- TABLE 6 University of Notre Dame 2006 LibQUAL+™ Survey: Distribution of Comments by Graduate Students Questions Number of Comments Print and/or electronic journal collections I require for my work (IC-8) 52 Printed library materials I need for my work (IC-3) 31 Willingness to help users (AS-8) 25 Making information easily accessible for independent use (IC-7) 24 Timely document delivery/interlibrary loan (Local Question 4) 24 Adequate hours of service (Local Question 5) 18 Easy to use access tools that allow me to find things on my own (IC-6) 17 A Comfortable and inviting location (LP-3) 17 Modern equipment that lets me easily access needed information (IC-5) 14 Accuracy in catalog, borrowing and overdue records (Local Question 3) 12 Employees who deal with users in a caring fashion (AS-6) 12 A library web site enabling me to locate information on my own (IC-2) 11 Employees who have the knowledge to answer user questions (AS-5) 10 The electronic information resources I need (IC-4) 9 Quiet Space for Individual Activities (LP-2) 9 Making electronic resources accessible from my home or office (IC-1) 7 Employees who are consistently courteous (AS-3) 4 Library orientation/instruction sessions (Local Question 1) 3 Making me aware of library resources and services (Local Question 2) 3 Library Space that inspires study and learning (LP-1) 3 Community space for group learning and group study (LP-5) 3 Readiness to respond to users questions (As-4) 2 Employees who understand the needs of their users (AS-7) 2 Dependability in handling users’ service problems (AS-9) 2 Giving users individual attention (AS-2) 1 A getaway for study, learning, or research (LP-4) 0 Employees who instill confidence in users (AS-1) 0 Total 315 504 College & Research Libraries November 2008 by a faculty member from Arts & Le�ers was typical of others: “The librarians at Notre Dame are knowledgeable and help- ful. Unfortunately, the library still lacks many resources in print and electronic format.” Another faculty member wrote, “The Library’s greatest failing is gaps in its collections of printed materials, and inability to purchase as broadly as is needed because of inadequate funding. The staff are almost without exception WONDERFUL.” “Timely document delivery/interli- brary loan” (Local Question) also drew a considerable number of comments. One faculty member wrote that “the Library TABLE 7 University of Notre Dame 2006 LibQUAL+™ Survey: Distribution of Comments by Faculty Questions Number of Comments Print and/or electronic journal collections I require for my work (IC-8) 29 Printed library materials I need for my work (IC-3) 26 Timely document delivery/interlibrary loan (Local Question 4) 18 Willingness to help users (AS-8) 16 The electronic information resources I need (IC-4) 11 A library web site enabling me to locate information on my own (IC-2) 10 Employees who have the knowledge to answer user questions (AS-5) 8 Making information easily accessible for independent use (IC-7) 7 Modern equipment that lets me easily access needed information (IC-5) 6 A Comfortable and inviting location (LP-3) 6 Employees who are consistently courteous (AS-3) 5 Employees who deal with users in a caring fashion (AS-6) 5 Adequate hours of service (Local Question 5) 4 Easy to use access tools that allow me to find things on my own (IC-6) 4 Quiet Space for Individual Activities (LP-2) 4 Employees who understand the needs of their users (AS-7) 3 Dependability in handling users’ service problems (AS-9) 3 Accuracy in catalog, borrowing and overdue records (Local Question 3) 2 Library orientation/instruction sessions (Local Question 1) 2 Making electronic resources accessible from my home or office (IC-1) 2 Library Space that inspires study and learning (LP-1) 2 Readiness to respond to users questions (AS-4) 2 Making me aware of library resources and services (Local Question 2) 1 Giving users individual attention (AS-2) 1 A getaway for study, learning, or research (LP-4) 0 Community space for group learning and group study (LP-5) 0 Employees who instill confidence in users (AS-1) 0 Total 177 An Analysis of Qualitative Survey Comments 505 has an astonishing lacunae in classics and literature. ILL is easy and quick and makes up for it,” while another faculty member writes, “ILL is excellent, which is good because the holdings in my field are not very strong.” These comments from faculty, as well as from graduate students, support the observation made by Cook and Heath during their inter- views conducted with users of research libraries that “for the most part, interli- brary loan is now seen as an acceptable and important component of the research process.”21 Like the graduate students, faculty also expressed frustration over the unavailability of books and the lack of enforcement of the library’s recall policy. Consequently, the library introduced a new policy to reduce the time delay between a patron recalling a book and actually having access to the book. Comparison between Quantitative & Qualitative Results If the ratings for “desired level of service” (shown in table 3) can be used to indicate the level of importance users a�ach to the various dimensions of service, the follow- ing library services are most important to Notre Dame users overall: 1. Making electronic resources acces- sible from my home or office 2. Print and/or electronic journal col- lections I require for my work 3. A library Web site enabling me to locate information on my own 4. Modern equipment that lets me easily access needed information 5. Easy-to-use access tools that allow me to find information on my own 6. The electronic information resourc- es I need 7. Making information easily acces- sible for independent use TABLE 8 University of Notre Dame 2006 LibQUAL+™ Survey: Top Ten Services by User Group Based on Number of Comments Question USER GROUP Undergrads Graduate Students Faculty Total Adequate hours of service (Local Question 5) 160 18 4 182 Print and/or electronic journal collections I require for my work (IC-8) 34 52 29 115 Printed library materials I need for my work (IC-3) 57 31 26 114 Willingness to help users (AS-8) 62 25 16 103 Making information easily accessible for independent use (IC-7) 34 24 7 65 Timely document delivery/interlibrary loan (Local Question 4) 17 24 18 59 A Comfortable and inviting location (LP-3) 35 17 6 58 Easy to use access tools that allow me to find things on my own (IC-6) 29 17 4 50 Employees who deal with users in a caring fashion (AS-6) 24 12 5 41 Modern equipment that lets me easily access needed information (IC-5) 20 14 6 40 506 College & Research Libraries November 2008 8. Adequate hours of service 9. Accuracy in the catalog, borrowing, and overdue records 10. The printed library materials I need for my work On the other hand, if the number of comments expressed by users is an in- dication of importance, then the services illustrated in the table below are most important: The large number of comments re- ceived from undergraduates, relative to the number of comments received from graduate students and faculty, skewed the results somewhat and may explain the differences between the two lists. Nevertheless, the results from the quali- tative analysis confirmed the importance of Information Control issues to all user groups. Questions appearing in both top 10 lists of important services (table 3 and table 8) are in bold print and all fall into the area of “Information Control.” Also affirmed was the importance of the library as place, as well as the importance of good service, especially library staff’s ready willingness to help users. Comparison by User Groups Earlier in this article, the five most impor- tant services by user group, based on mean average scores for the “desired” level of service, were identified. When these lists are compared to the lists ranked in order by the number of comments, some differ- ences can be observed; but the importance of information control, regardless of user group, is affirmed (see table 9). Issues of information control are equally important to undergraduates, graduate students, and faculty, although their priorities differ. For example, inter- library loan seems to be more important to faculty and graduate students than it is to undergraduates, while undergraduates want adequate hours of service, an issue not as important to faculty and graduate students. The results also show that the library as a place is more important to undergraduates than it is to faculty and graduate students. An examination of the most important services as indicated by both the “desired ratings” and by the number of wri�en comments revealed only one area that was consistent within each user group and across the three user groups: Print and/or electronic journal collections I require for my work. The clear call from students and faculty alike is for more journals, especially in electronic format, and is obviously an area that must remain a priority for the Libraries. An examination of the most impor- tant services as indicated by the wri�en comments revealed that two of the top five were the same across all three user groups: “Printed library materials I need for my work” and “Willingness to help users.” While it was expected that “Printed library materials I need for my work” would rank highly on the list of services users valued the most, the fact that “Willingness to help users” appeared in the top five most important services for each of the three user groups was surpris- ing and suggests that this is one area of service quality that is more important to users than the quantitative results alone would indicate. Conclusion The qualitative data gleaned from the LibQUAL+™ survey comments proved to be as useful, if not more useful, than the quantitative data in providing the Notre Dame Libraries with specific meaningful feedback from its users. The dimension of Information Control dominated both the quantitative and the qualitative results. The qualitative analysis allowed for a be�er understanding of the quantitative results and for a greater sense of how users differ in their desires and in the importance they assign to the various aspects of library services. Furthermore, our hypothesis that the results of the analysis of the wri�en comments would parallel those of the quantitative results is supported. The LibQUAL+™ results have been shared with library staff and users via An Analysis of Qualitative Survey Comments 507 TABLE 9 University of Notre Dame 2006 LibQUAL+™ Survey: Comparison of Most Important Services Based on Number of Written Comments vs. “Desired” Score Ratings User Group Most Important Services (Based on Desired Ratings) Most Important Services (Based on Number of Written Comments) Undergraduates Making electronic resources available from home or office (IC-1) Adequate hours of service (Local Question) Modern equipment that lets me easily access needed information (IC-5) Willingness to help users (AS-8) Adequate hours of service (Local question) Printed Library materials I need for my work (IC-3) A library web site enabling me to locate information on my own (IC-2) A comfortable and inviting location (LP-3) Print and/or electronic journal collections I require for my work (IC-8) Print and/or electronic journal collections I require for my work (IC-8) Graduate Students Print and/or electronic journal collections I require for my work (IC-8) Print and/or electronic journal collections I require for my work (IC-8) A library web site enabling me to locate information on my own (IC-2) Printed Library materials I need for my work (IC-3) The electronic resources I need (IC-4) Willingness to help users (AS-8) Making electronic resources available from home or office (IC-1) Making information easily accessible for independent use (IC-7) Easy to use access tools that allow me to find things on my own (IC-6) Timely document delivery/ interlibrary loan (local question) Faculty Print and/or electronic journal collections I require for my work (IC-8) Print and/or electronic journal collections I require for my work (IC-8) A library web site enabling me to locate information on my own (IC-2) Printed Library materials I need for my work (IC-3) Timely document delivery/ interlibrary loan (local question) Timely document delivery/ interlibrary loan (local question) The electronic resources I need (IC-4) Willingness to help users (AS-8) Easy to use access tools that allow me to find things on my own (IC-6) The electronic resources I need (IC-4) 508 College & Research Libraries November 2008 the Libraries’ Web site, advertisements in our campus newspaper, e-mail an- nouncements, and posters. Library de- partments are studying the quantitative results, as well as the wri�en comments, to identify areas where improvements can be made. A stronger recall policy was implemented to reduce the time delay between a patron recalling a book and having access to it. After meeting with several student groups, the library introduced 24-hour access to the building during fall and spring semester study days, with plans to investigate year-round 24-hour access following renovation of the first and second floors. The Graduate Study Area on the main library’s tenth floor was renovated, in addition to some short- term renovation of the first floor to make it more comfortable and inviting before funding is secured for significant, major renovations. Long-term renovation plans include a café. To better meet users’ expectations, the Libraries have focused even more on user-centered design, usability studies, log file analysis, and creating processes of continuing improvement. To address users’ problems with locating journal articles, a redesigned interface entitled “Find Articles” on the Libraries’ homep- age has been created. Marketing of library resources and services has increased through a series of rotating graphics on the Web site, a newly created electronic resources blog, attractive posters and bookmarks, and a large flat-panel monitor near the information desk (displaying a continuous slide show highlighting new resources and services). These are examples of actions the library has taken to enhance services and to be a responsive, user-centered library. LibQUAL+™ data, along with other methods of customer feedback, will continue to be used to assess the needs of users and to plan services and resources to meet their needs. Further study will in- volve examining differences by discipline to create an even greater understanding of the needs of users at Notre Dame and where the library should focus its efforts to enhance services for all users. Notes 1. Office of Undergraduate Admissions, “2006–2007 Statistics,” University of Notre Dame. Available online from h�ps://admissions.nd.edu/firstyear/statistics.cfm. [Accessed 24 September 2008]. 2. Association of Research Libraries, “Principles and Procedures of Membership.” Available online from www.arl.org/arl/membership/qualprin.shtml. [Accessed 24 September 2008]. 3. Association of Research Libraries, “LibQUAL+™: Charting Library Service Quality.” Available online from www.libqual.org. [Accessed 24 September 2008]. 4. Association of Research Libraries, “New Measures & Assessment Initiatives.” Available online from www.arl.org/stats/initiatives/. [Accessed 24 September 2008]. 5. University Libraries, “LibQUAL+™ Results,” University of Notre Dame. Available online from www.library.nd.edu/libqual/. [Accessed 24 September 2008]. 6. For background information on the identification of the original LibQUAL+™ dimensions, see: Cook, Colleen and Fred M. Heath, “Users’ Perception of Library Service Quality: A LibQUAL+ Qualitative Study,” Library Trends 49, no. 4 (Spring 2001): 548–84. 7. Ibid. 8. University Libraries, “LibQUAL+™ Results,” University of Notre Dame. Available online from www.library.nd.edu/libqual/. [Accessed 24 September 2008]. 9. Bruce Thompson, “Representativeness versus Response Rate: It Ain’t the Response Rate!” in LibQual+TM: A Total Market Survey: ALA Midwinter Workshop Reader, ed. C. Cook (Washington, D.C.: Association of Research Libraries, 2000). 10. Colleen Cook, Fred Heath, and Bruce Thompson, “‘Zones of Tolerance in Perceptions of Library Service Quality: A LibQUAL+™ Study,” portal: Libraries and the Academy 3, no.1 (2003): 113–23. 11. LibQUAL+™ Spring 2006 University of Notre Dame Survey Results (Washington, D.C.: Association of Research Libraries, 2006). An Analysis of Qualitative Survey Comments 509 12. Ibid. 13. Cook, Heath, and Thompson, “Zones of Tolerance,” 113–23. 14. Eileen E. Hitchingham and Donald Kenney, “Extracting Meaningful Measures of User Satisfaction From LibQUAL+™ for the University Libraries at Virginia Tech,” Performance Mea- surement and Metrics 3, no. 2 (2002): 57. 15. Steve Hiller, “How Different Are They?: A Comparison by Academic Area of Library Use, Priorities, and Information Needs at the University of Washington,” Issues in Science & Technology Librarianship, no. 33 (Winter 2002). 16. Eric C. Shoaf, “Using a Professional Moderator in Library Focus Group Research,” College & Research Libraries 64, no. 2 (Mar. 2003): 124. 17. Carle�e Washington-Hoagland and Leo Clougherty, “Identifying the Resource and Service Needs of Graduate Students,” portal: Libraries and the Academy 2, no. 1 (2002): 131, 133. 18. Maria Anna Jankowska, Karen Hertel, and Nancy J. Young, “Improving Library Service Quality to Graduate Students: LibQUAL+™ Survey Results in a Practical Se�ing,” portal: Libraries & the Academy 6, no. 1 (Jan. 2006): 59–77. 19. Jeffrey N.Ga�en, “Measuring Consortium Impact on User Perceptions: OhioLINK and LibQUAL+™,” Journal of Academic Librarianship 30, no. 3 (May 2004): 222–228 20. Jankowska, Hertel, and Young, “Improving Library Service Quality,” 59–77. 21. Colleen Cook and Fred Heath, “Users Perceptions of Library Service Quality: A LibQUAL+ Qualitative Study,” Library Trends 49, no. 4 (Spring 2001): 548–84. 22. Ibid. ������������ ����������� ��������� � ��� � ������������ ���������������� ������������� ���������������� ������������������ ������������������ ��������������� ��������������������� ����������������� ��������������� ���������������� �������������������� ����������������������� ��������������� ��������� ���������� ������������� ������������������� ��������������������������� ���������������� ������������������������� ��������������������������� ����������������� ������������������ ��������������������������� ������������������������ ����������������� ����������������������������� �����������������