Conference Paper

After the Data: Taking Action on ClimateQUAL® Results

Elizabeth Mengel

Associate Director, Scholarly Resources and Special Collections

The Sheridan Libraries, Johns Hopkins University

Baltimore, Maryland, United States of America

Email: emengel@jhu.edu

Judith Smith

Liaison Librarian

University of Michigan

Ann Arbor, Michigan, United States of America

Email: judsmith@umich.edu

Elizabeth Uzelac

Law Clerk, United States District Court for the Northern District of Illinois 

Chicago, Illinois, United States of America

Email: euzelac@gmail.com

2013 Mengel, Smith, and Uzelac. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – This paper discusses the actions taken by the staff development and training (SD&T) team at the Sheridan Libraries and Johns Hopkins University Museums in response to results of a ClimateQUAL survey.

Methods – The team administered the ClimateQUAL Organizational Climate and Diversity Assessment in March 2009 to the 150 staff members of the museums and libraries, and 80% responded. To get at the root of some of the results, the team conducted 23 focus group sessions over the course of two months. In each 90-minute session, 8 open-ended questions were used to probe the staff’s thoughts on the survey results and elicit concrete suggestions for moving forward. Participants were asked to discuss their personal experiences with six areas of concern: procedural justice, distributive justice, structural facilitation of teamwork, psychological safety, communication, and leadership. One year after the original ClimateQUAL survey, the team administered a one-question follow-up survey.

Results – The team analyzed and coded the notes taken during the focus group sessions and developed three discrete written summaries for each session: a brief summary of themes, a list of specific actionable suggestions, and a general description of specific scenarios aired in the sessions. From these analyses, the team developed two types of recommendations: quick tactical actions and long-term strategic recommendations. Strategic recommendations were developed in three main areas: fostering a sense of global ownership of organizational issues, improving organizational communication, and improving leadership and facilitation of teamwork. With these recommendations, the team charged managers to take broad ownership of a plan for individual actions. The results of the one-year follow-up survey were mixed. Staff perceived positive change in communication, but indicated that the areas of procedural and distributive justice, psychological safety, and transparency in decision making continued to require improvement.

Conclusion – The work of the SD&T team continues, and it is hoped that ClimateQUAL will serve as the foundation for future assessments of organizational health.

Introduction

In 2008, the Sheridan Libraries and Johns Hopkins University (JHU) Museums Staff Development and Training Team (SD&T) found itself wrestling with ways to facilitate constructive organizational change. The team was at a regrouping point in terms of its strategic direction. It sought to go beyond anecdotal evidence in identifying ways to support library and museum staff. SD&T, a small committee charged with supporting the staff of the Libraries and Museums with training and organizational development matters, did not want to make decisions based on unfounded assumptions in the course of implementing change in the organization. To ensure its programming was rooted in the actual needs of the organization, the team administered the ClimateQUAL Organizational Climate and Diversity Assessment (OCDA) in 2009 to its 150 staff members. ClimateQUAL, a confidential, third party organizational health and diversity survey, is designed to assess the shared culture of an organization. It gathers data and assesses overall staff perceptions of the organizational climate of a library. Developed at the University of Maryland Libraries in 2000, ClimateQUAL is now supported in partnership between the University of Maryland and the Association of Research Libraries (ARL).

Little did the team realize that participating in ClimateQUAL would begin a year-long odyssey of building organizational trust and championing change! The official ClimateQUAL report turned out to be merely the beginning, and the team discovered that getting results is by far the easiest part of the process.

This paper discusses the organization’s process of responding to data measuring organizational climate. After outlining the survey preparation and administration processes, the paper describes the ClimateQUAL results report. The paper then discusses the issues inherent in acting upon the report, and the processes taken to respond to these issues and act to improve the organization. It explores and explains the steps that came next in responding to data: staff focus groups, in-depth interviews with library leadership, qualitative and quantitative data analysis, re-evaluating the meaning of communicating well, finding ways to get staff and management to hear one another, and developing short and long range recommendations. The authors hope that explicating the process will aid other organizations in taking effective action in response to their own ClimateQUAL data.

Survey Preparation and Administration

The libraries and museums ran the ClimateQUAL survey March 2-23, 2009. While many organizational culture assessments exist, the team chose ClimateQUAL because of its library-based context and the support offered through a community of peers. There was an 80% return rate to the survey, a rate in line with the high return rates at other ClimateQUAL institutions. The high return rate was a byproduct of the significant preparation the SD&T team led to prepare the organization and foster a sense of both security and ownership among staff.

Prior to administering the survey, the SD&T team spent a considerable amount of time planning the rollout and grappling with organizational questions. How would we communicate the survey to staff? How would we motivate them to take it? How do we manage expectations about what can be done with the results to create change? How do we get staff to understand change is a long process and most organizational problems do not have quick fixes? The team realized that a critical success factor of the survey administration would be the existence of trust among staff in the confidential nature of the survey. As even good changes can be disruptive, part of the role as potential change agents was to find ways to effectively and comfortably discuss organizational issues without losing the trust or participation of people along the way. The team wanted to emphasize the reasons for doing the survey: assessing the libraries and museums’ organizational health, making people part of the process of improving the organization, and stimulating thinking about everyone’s role in broader organizational improvements.

Three weeks prior to the administration of the survey, SD&T team members held meetings with each department to explain the survey’s importance to the organization and outline how organizational demographics were mapped to the ClimateQUAL demographic categories. The team emphasized the survey’s safeguards to protect someone’s identity. For example, if there was a low response rate in any particular department, these responses would be rolled up into the next larger category. In these meetings, the team told staff how the raw data would be handled (i.e., no one in the institution would be able to see it or manipulate it to determine who said what), how the incentive would be administered (the team chose to have ARL administer the incentive so no one at JHU would know who submitted a survey or who won the incentive), and how the results would be distributed (the full report of everything received would be sent to staff; nothing would be held back).

These meetings were the first step in building trust with staff. Once the SD&T team established this trust, it was vigilant throughout the rest of the process to make sure we did nothing to break these bonds. Staff members showed an interest in change and looked to the team to facilitate changes many had hoped would happen. Feeling a strong sense of responsibility to the organization to do this process well, the team focused on following up on promises and finding ways to keep staff informed along the way throughout the survey administration period and beyond.

Survey Results

The SD&T team received the ClimateQUAL results report several weeks after the survey closed. The results are divided into four sections: demographics, organizational climate scales, analysis of variables, and comments.

Demographics

The demographics section provides breakdowns by Library Team, Position, Full or Part Time, Librarian vs. Non-Librarian, Age, Ethnicity, Religion, Sexual Orientation, Gender, Length of Service at JHU, and Length of Overall Library Service.

Organizational Climate Scales

After the demographics, the results provide tables with the mean, standard deviation, and standard error for the Organizational Climate Scales. The scales include measures for Organizational Justice, Leadership Climate, Diversity, Climate for Continual Learning, Climate for Teamwork, and more. Definitions of these terms can be found at: http://www.climatequal.org/concepts/core-scales/index.shtml. Because the results contain a considerable amount of statistical data, one is also provided definitions and some interpretation of mean, standard deviation, and standard error to help non-statisticians understand the significance of those measures.

Analysis of Variables

Following the tables on Organizational Climate, the results include another set of statistical tables on the demographic differences of the Climate Scales. These tables were perhaps the most confusing to people who were unfamiliar with reading and interpreting statistics especially Analysis of Variance (ANOVA) tables. In the simplest of terms, the Analysis of Variance (ANOVA) compares the means for different groups and asks whether such differences are systematic and likely to be replicated if we had another sample of respondents. If the probability is less than .05 or .01, we typically conclude that there is a systematic difference across the different groups and the difference is likely to be replicated across different samples. Even though ANOVA tests do not indicate importance, they do indicate how likely it is to see such differences replicated and can serve as an initial stepping stone for evaluating group differences for the climate scales in ClimateQUAL.

Comments

After the ANOVA tables were the comments section. Twenty-one percent of staff members supplied some type of comment. Those comments ranged from in-depth, serious discussions about issues in the library to comments on the structure and phrasing of the survey itself. Based on comments the team heard once the results were released, we suspect the comments section was the most popular with staff and where they spent the majority of their time.

Challenges

While the results were full of important data, they lacked a roadmap for our next steps. To develop a plan, the SD&T team spent focused, intensive time understanding the data, analyzing the comments, and posing questions to ourselves. What comes next? What actions are needed? One of the challenges the team faced, as is common when attempting to use any survey’s results, was determining what is actionable based on the data.

Another challenge was to determine the best way to convey the results to staff. The team had promised to provide all raw data to staff members, but there needed to be some kind of explanation to help guide them through the results, especially the statistical information. The SD&T team decided to craft an Executive Summary to the report including indications of which pages staff should spend their time investigating. The team also included information about the team’s next steps to continue the dedication to a transparent communication process.

Choosing Focus Issues

The team’s first step was to isolate specific issues on which it felt the organization should concentrate. Because the team could not address every possible issue in the report at once, it was important to prioritize. It used the analysis in the report to identify focus areas: 1) where a high percentage of staff gave positive responses; and, 2) where a high percentage of staff gave negative areas. These areas became, in turn, areas highlighted for celebration and targeted for constructive change.

The organization scored well in the ClimateQUAL areas of Benefits of Teamwork (93% of respondents giving positive responses), Task Engagement (87%), and both Valuing Diversity (84%) and Climate for Demographic Diversity (91%). In addition, the organization had welcome low scores is areas such as Organizational Withdrawal (i.e., desire to leave the organization, 13% of respondents answering affirmatively) and Work Unit Conflict at both the Interpersonal and Task level (18% and 24% respectively). Finally, the comments revealed that staff had very positive things to say about their colleagues and the dedication and work ethic exhibited daily in the organization. The team took these to be very positive signs.

Similar to other organizations, there were areas where the team needed to focus some attention. The team identified a small number of indicators for which fewer than 60% of our organization’s staff gave a positive response or where the mean score was below 5.0 on a 7-point scale. Scores for Distributive Justice (25%), Procedural Justice (55%), Structural Facilitation of Teamwork (48%), and Climate for Psychological Safety (66%, but a mean score of 4.94) all pointed to areas where organizationally we needed improvement.

To incorporate the free text comments into a plan of action, the team coded them to analyze any patterns or trends. Rather than use a grounded theory approach in analyzing the comments, the team used the ClimateQUAL Core Concept terms as provisional categories to which it mapped clauses of each comment. First, we reduced the data in the concepts by labeling them with ClimateQUAL terms relevant to what each comment conveyed (Miles & Humberman, 1994). This process focused, simplified, and abstracted the comments, enabling us to work with an organized, compressed display of what they communicated (Miles & Humberman, 1994). After analyzing the comments in relation to the ClimateQUAL Core Concept terms, we were able to conclude that the comments mirrored the focus issues identified above (Distributive Justice, Procedural Justice, Structural Facilitation of Teamwork, and Psychological Safety). In addition to these areas, the comments also showed a pattern of concern surrounding leadership and communication. Coding the comments enabled us to explicitly connect the qualitative data available to us to the quantified conclusions in the ClimateQUAL report. Moving forward, we were then able to use all of the patterns in our results to inform an action plan.

Taking Data-Driven Action

The ClimateQUAL report helped identify the most pressing organizational issues, allowing the SD&T team to avoid guessing at underlying organizational strengths and weaknesses when creating an action plan. Initial internal response to the report, though, involved far more questions than comprehension when it came to taking action. Why do people feel psychologically unsafe? How does our organization aid or impede teamwork? What does distributive justice mean at JHU? How do you reconcile positive scores on the benefits of teamwork with a lack of structural facilitation of team work? To develop specific next steps, the SD&T team, library leadership, and managers needed to better understand the specific landscapes of target issues. The SD&T team realized that it needed to dig deeper and go beyond the ClimateQUAL results. After many discussions about the best method for delving further, the team decided to engage in in-depth focus groups with staff to contextualize focus issues in the organizational landscape.

Contextualizing Issues Through Focus Groups

To get to the root of some of the ClimateQUAL results, the team conducted focus groups with each department in the organization. Members of the SD&T team paired up, one person assuming a facilitator role and the other assigned to take detailed notes. The SD&T pairs scheduled ninety-minute sessions with each department, without their manager, as well as one session each for managers, supervisors, the Staff Development and Training Committee, and those unable to attend with their department. In all, the team held 23 focus groups over the course of 2 months.

In each session, the team used structured conversations to probe the staff’s thoughts on the survey results and ask for concrete suggestions for moving forward. The facilitator opened each focus group by explaining that individual comments would be kept confidential, and asked that each attendee similarly refrain from sharing their colleagues’ comments outside the session. The facilitator also tried to manage expectations about the focus group and its purpose: the team was there to hear more, and gather ideas to inform actions, but not every idea could necessarily be implemented. Each attendee received a one-page handout listing the definitions of the six terms (Procedural Justice, Distributive Justice, Psychological Safety, Facilitation of Teamwork, Communication, and Leadership), and the questions that were to anchor the session’s conversation. The group briefly reviewed the overall results and six issues of concern from ClimateQUAL in order to set a common stage for those who may not have internalized the vocabulary and conclusions from the assessment, or were perhaps simply overwhelmed by the amount of statistical data provided.

The facilitator then asked for all suggestions, concerns, and responses in answer to the following eight questions, which anchored the conversation for the duration of the focus group sessions:

1. What did you think (were your impressions) of the survey itself (taking it, questions, timing, how it was rolled out)?
2. What were your first impressions of the results?
3. What are your expectations now, having read the results? What would you like to see done?
4. After reading the comments in the survey, is there anything else you wish had been said?
5. What makes you feel valued? (What types of rewards, recognitions, processes, or other factors?)
6. How do you feel the organization might better foster or facilitate teamwork?
7. Of the issues we identified earlier (Procedural Justice, Distributive Justice, Structural Facilitation of Teamwork, Psychological Safety, Communication, Leadership), which do you think needs to be addressed first?
8. What changes in the organization would you like to see in the organization when we repeat the survey?

The team chose to create open-ended questions to better encourage staff to explore and share their thoughts on the survey and potential follow-up actions. The eight questions were designed to progress first from helping participants remember the assessment questionnaire, to exploring their personal experiences with the six issues of concern – to discussing concrete ideas for action. Facilitators inhabited a neutral questioning role, and refrained from agreeing or disagreeing with any statements made. To help guide attendees from venting concerns to making tangible suggestions, facilitators used a series of follow-up prompts, including “If you were put in charge of fixing that issue, what would be your first step?” The team wanted to ensure that staff understood that they not only had the ability to present ideas, but that it is their responsibility to be part of the change process. This was a first step toward having staff take ownership of future organizational change.

By emphasizing confidentiality and constructive engagement, the team heard an enormous amount of information, even from individuals who had not previously felt comfortable actively engaging in global organizational issues. Overall, participants showed remarkable candor. Some staff members aired specific anecdotes of concern to them. Many were responsive to the above prompts and the discussion that followed. As the focus groups progressed, the team heard directly from staff members who were appreciative of the chance to participate so directly in organizational change. In a few cases, staff members who had at first elected not to participate in a focus group changed their mind after hearing from their colleagues about their focus group experience. Staff members passed the word along about the benefits of the focus group sessions, but fully complied with the confidential nature of the content discussed. Through their actions, they helped reinforce and perpetuate the underlying trust that the SD&T team sought to engender.

Developing Action Items

Some concerns turned out to be common to almost every focus group. There was a clear overall message from the focus groups that it would be detrimental to staff morale if action was not taken in response to issues identified by the ClimateQUAL survey. To recommend concrete actions, the team evaluated and dissected the notes of every focus group. Common themes for each question emerged alongside concrete recommendations and historical anecdotes. The SD&T team analyzed and coded the notes of each of the eight focus group questions. The team developed three discrete written summaries for each session: a brief summary of the themes and sentiment of focus groups participants, a list of specific actionable suggestions elicited during the sessions, and a general description of any specific scenarios that focus group participants aired in the sessions.

From these analyses, the team developed two major types of recommendations: quick tactical actions and long-term strategic recommendations. Long-term strategic recommendations were developed in three main areas: fostering a sense of global ownership of our organizational issues, improving organizational communication, and improving leadership and facilitation of teamwork. Many of the final quick and long-term recommendations, listed below, came from the data developed through focus group sessions.

Those recommendations were:

· To address overall organizational climate and leadership skills we need to foster a sense of ownership of organizational issues:

o Develop leadership skills on all levels of the organization.

o Facilitate conversation across the organization about leadership

· To address organizational communication:

o Develop and publish each of the following, in series: a) a complete organizational chart, b) a map of organizational workflows, and c) a matrix of how decisions are made.

o Charge Management Team with designing an explicit set of managerial communication principles and hold each other accountable to those principles.

o Charge a cross functional, cross departmental working group with developing a set of communication principles to use across the organization.

· To address both leadership and facilitation of teamwork practices:

o Bring in Talent Management and Organizational Development (an internal JHU unit which provides a suite of human resources, organizational development, and talent management services) to assist in developing the Management Team into a high performing team.

o Charge the Executive Committee and Management Team with defining delegated authority and work with Talent Management and Organizational Development to move toward organizational practices that empower teams and remove ambiguity about authority in team related issues.

o Develop institutional teamwork checklists that address issues such as participation, accountability, roles and responsibilities, team communication expectations, and annual reporting practices.

o Develop Sheridan Libraries and Johns Hopkins University Museums 101: have the organization collaboratively and openly design its own cross-training program.

The team worked to articulate the scope of what to address when specific solutions were trickier to find or outside the scope of the team’s expertise. For instance, executive and managerial level staff members were charged as a group with defining “delegated authority” to move toward practices that empower and remove ambiguity about authority in team-related situations. Additionally, managers were charged with designing an explicit set of managerial communication principles and holding each other accountable to those principles. The principles included prompt sharing of information, structuring decision making around a process based on the strategic plan, logic and data, and endeavoring to operate by consensus.

With these recommendations, the team charged managers to take broad ownership of a plan for individual actions. In cases where an issue was entwined with the daily work of managers, it was necessary to define overall expectations directed toward all managers. The team shifted into a role as a source of program-wide momentum and reporting, and managers were expected to engage with the full set of recommendations. Managers were charged with employing organizational-level thinking and were encouraged to make direct ties between the recommendations and their program areas. Managers subsequently articulated the first set of actions that would be undertaken, complete with timeline and a point person.

Engaging Executive Leadership

After the recommendations were presented to the management team, the SD&T team felt it was important to discuss them in more depth with the Executive Leadership. The team held one-hour individual sessions with the Dean and each of the other members of the Executive Council. To get the conversations started the following questions were emailed prior to the discussion.

1. Based on the ClimateQUAL Focus Group recommendations, which of the recommendations really resonates with you?
2. For us to be in a position to most efficiently attain our strategic goals and achieve success what would our library organization look like? OR How would a successful organization differ from our organization today?
3. As an executive level leader, how do you suggest starting to address some of these issues?

Executive Leadership, like other staff members, were open and honest about their perceptions and were willing to give their insights. They were able to be honest because they also knew that their comments and observations would be kept confidential, and would only go towards helping the SD&T team develop a viable action plan.

Maintaining Momentum

After the baton was officially passed to the Management Team, the Staff Development and Training team’s role changed to one of maintaining momentum. There were still many ways the team needed to continue the change process through follow-up actions. Based on comments in the survey and focus group sessions, the team discovered that staff members have a long organizational memory – especially for projects that once started with fanfare and were then not spoken of again. Although the team was not directly responsible for many of the action items, it needed to track the identified actions holistically and ensure that tasks were completed by the Management Team and communicated to all staff. The actions themselves and the continuous communication helped maintain the solid level of trust developed. Staff members looked to the team to take action, so any perception of lack of action would have left staff feeling that their confidences and trust was betrayed.

The SD&T team facilitated follow-up in multiple ways. One way was to have the Management Team report on progress at the libraries’ and museums’ Staff Exchanges (i.e., all staff meetings). Since perceptions of the staff from the ClimateQUAL survey results were that management in the library were not responsive to staff needs, it was especially important to have those responsible report and be visible to staff members. The task did not even need to be complete at the time of reporting out – there just needed to be a noticeable effort to keep staff members apprised of the project’s status.

Another initiative focused on better communication across the organization. Using the Management Teams’ communication principles as the basis, the SD&T team repurposed them to be appropriate for all staff members. The communication principles focused on the Libraries and Museums’ values, including integrity and openness, innovation and constructive engagement, and stewardship and trust. The SD&T members presented these principles at a Staff Exchange, using examples from their own work to illustrate main points at an all-staff, open meeting. Staff members eagerly participated in the Staff Exchange and responded positively, indicating that they not only enjoyed the session but felt that they learned a great deal. One staff member mentioned how grateful he was that these issues were being examined. Overall, the session bolstered the level of trust that the team had built with the staff and illustrated how we were working towards common goals.

In addition, the team became deeply involved in developing a new performance appraisal program. ClimateQUAL indicated levels of dissatisfaction with distributive and procedural justice. A way to address this issue was through a new University led Performance Partnership Program (i.e., performance appraisal system). Highlights of this new system include: a single anniversary date, a much stronger focus on year-round coaching and development, and the creation of defined, measurable goals. This new performance appraisal system was a huge initiative in the team’s workload. It required the team to evaluate a new system, allay staff concerns, and create staff “buy in.” The SD&T team utilized the communication techniques from the ClimateQUAL rollout: communicating through multiple venues; communicating repeatedly; and meeting with every department to describe the rationale for the new system. Overall, the team reinforced the idea to staff that the new system was a result of listening to their needs. We worked closely with the University’s Talent Management and Organizational Development department to train staff on the new system using a “train the trainer approach.”

One year after the original ClimateQUAL survey, the team administered a simple, one question follow up survey via Zoomerang. The question asked was: “Last year the issues below were identified from ClimateQUAL and the follow up focus groups as ORGANIZATIONAL issues that needed to be addressed. Please reflect back on the past year. How do you think the Sheridan Libraries and Johns Hopkins University Museums are doing on these issues at this point?” The issues identified were the six issues highlighted throughout this paper: Distributive Justice, Procedural Justice, Psychological Safety, Facilitation of Teamwork, Communication, and Leadership. The team also added a comment box to the survey. By administering this survey, the team wanted to better understand perceptions of organizational improvement one year after taking ClimateQUAL to help inform what items to work on next. We had a response rate of 44%, and the results were mixed. As anticipated, the perceived pace of change on issues as core as those raised by ClimateQUAL is gradual, yet staff expected faster results. The team recognized that none of these issues will be “fixed” without long term attention and willingness of the organization to change.

There were areas where staff believed there had been change, and areas that indicate a desire for more or faster improvement. Areas where staff perceived improvement include Communication, where 66% of survey respondents perceived positive change. There were also indications of areas in which we continue to need improvement, such as Procedural and Distributive Justice. Some comments indicated dissatisfaction with lack of change overall. Survey results also indicated that there have not been significant changes in Psychological Safety and Transparency in Decision Making – leading the team to note, yet again, that organizational change takes a significant amount of time and continuous effort.

Overall Lessons

Several practical lessons follow the team’s experiences with ClimateQUAL and inform how it will handle current and future data and initiatives.

Start with a Strong Team

When undertaking a large initiate such as ClimateQUAL, the importance of a proactive and dedicated team is crucial. The SD&T team is comprised of members who volunteered to be on the committee because of their avid interest in organizational development issues. Without that interest and commitment to helping staff members succeed at their jobs, this type of assessment and follow-up would not succeed. Given the issues raised by the ClimateQUAL survey, absolute discretion of each team member was critical.

Communication

So much of what the team learned throughout the ClimateQUAL implementation, analysis, and follow up is the importance of a clear, proactive, and multi-pronged approach to communication. As in real estate where the motto is “location, location, location,” the team found that it consistently returned to “communication, communication, communication” as the foundation for everything needing to be accomplished. Instead of making assumptions about staff members “hearing” the team’s message, the team started with the premise that “hearing” is challenging. The team focused on ways to have the message about the survey and its follow up activities simple and clear. People have different communication styles. The team was consistently reminded of this fact as it communicated aspects of ClimateQUAL. More often than not the team still had staff members ask us questions that in our minds, we addressed. SD&T found it critical to communicate along each step of the process in multiple ways and through multiple venues: e-mails, meetings with departments individually, postings on the wiki, and answering questions individually, or presenting at Staff Exchanges. By proactively communicating in many different ways, the team was able have people “hear” the message because the active communication built a solid level of trust. Staff members felt that there was nothing being hidden from them, and they were receptive to the information given.

Data Needs Context

Whatever results you begin with will need to be interpreted and internalized for them to have meaning. The data from ClimateQUAL provide a starting point for analyzing institutional perceptions. However, the results do not provide the analysis that only you and your colleagues can provide through your institutional lens. More discussion is often needed, as the team discovered when it held focus groups. Other organizations may find different ways to tease out important themes from their ClimateQUAL results, but our team found that having ClimateQUAL as the jumping off point for continuous discussion (not the end point) to be what propelled it forward and helped it to understand the libraries and museums strengths and areas to address.

Determine the Level of Data Desired Through the Survey

ClimateQUAL offers a range of granularity in its results data. Prior to implementing your survey, consider the level of data that you are seeking. Is it at the unit level or the broader departmental level? There are various costs associated with the results received, so it is important to determine your organizations needs ahead of time and think about the results you want in the long run. The team wanted to start with results based around very broad demographic categories at the departmental level, and because of that, there were some questions about how applicable the data was to a supervisor’s individual unit. However, because many units in the libraries and museums are very small (2-3 people) this would leave individuals’ responses overly exposed the broader organization.

Create a Clear Process

An open and defined process laid out for staff helps answer the perpetual “what now?” questions that follow such an assessment. A group-oriented process can give staff a non-confrontational group voice to management. The team also found that even with setting up a clear process and communication, there were still many questions about what was actually being accomplished, and how quickly. Staff were eager for change, and it required reporting out on expectations and continuous management of expectations about timeframe. Long-term, deep change takes time, and this idea has to be restated often.

Focus and Tenacity is Required to Repair and Build Trust

Through the course of this process, the team found that there can never be enough trust in an organization, and that it takes conscious efforts and tangible actions, such as getting “out there” and speaking to colleagues, usually face-to-face, to build or repair trust.

Organization-Level Thinking is Crucial

At all levels, but especially in leadership, a broad organizational outlook is crucial for intentional change and organizational health. To succeed in trying to assess and implement change, there needs to be a strong ”we” at all levels to move ideas forward. The team found, in conversations at all levels, that more often than not no one spoke of the organization as a whole. Staff, including management, mentioned “their team,” “their staff,” or “their department.” The team found that the concept of “we” needs to be continuously emphasized in daily communications and in larger initiatives. This change in perspective takes time, but is vital for breaking down silos and fostering deep collaborations across units.

The work of the SD&T team continues, and we hope that ClimateQUAL will serve as the foundation for future assessments for organizational health. Our plans in the future involve, not only assessments of the organization as a whole, but also evaluations of how we as the Staff Development and Training team can continue to improve to meet organizational needs.

References

Miles, M. B., & Humberman, A. M. (1994). Qualitative data analysis. Thousand Oaks: Sage.