chen.indd Organizational Learning in the Evaluation Procedures: A Qualitative Study Kuan-nien Chen and Pei-chun Lin The study investigated the nature of organizational learning within Tai- wanese Institute of Technology academic libraries, and in particular the nature of the relationship between the Institutional Evaluation of those libraries by the Ministry of Education and the organizational response of the libraries. Semistructured interviews were conducted with 24 library personnel, including directors. These interviews resulted in the identifi- cation of three themes. The results of this study show that few libraries seem to be achieving what might be considered an adequate level of organizational learning, largely, it seems, because of internal barriers and structural problems with how the institutional evaluation is implemented. It seems difficult for some libraries to use evaluation as a tool to assist with organizational learning. Libraries must engage more in a learning dialogue with their environments and enhance their capacity for effective organizational learning. Research Problem Evaluation can provide an examination of old theories of intervention and an op- portunity to help build new ones, because evaluation occurs in a se ing where it is likely that the organization will respond in terms of its knowledge, a itudes, or behavior.1 Evaluation is the link between performance and learning structures.2 The key role of evaluation with respect to organizational learning is to illuminate how the existing organization learns,3 including what framing mechanisms it uses to interpret and interact with the internal and external environments, what active memory it draws on in the operations of the organization’s work, what knowledge structures it commonly uses, and how the organization develops both individuals and itself collectively to meet changing demands.4 In the case of academic libraries, one can identify several core dimensions of effectiveness measurement. Among these dimensions are management processes, resource inputs, reference services, collection Kuan-nien Chen is Associate Professor and Senior Librarian at Kaohsiung Medical University, Kaohsi- ung City, Taiwan; e-mail: wc@kmu.edu.tw. Pei-chun Lin is Lecturer at Tzu Hui Institute of Technology, Pingtung County, Taiwan; e-mail: coogeer@yahoo.com or e1026915@yahoo.com.tw. The authors appreciate Dr. Fletcher Cole and Associate Professor Connie Wilson for their valuable comments on the study. They both are currently academic staff of Information Systems, Technology and Management at the University of New South Wales, Sydney, Australia. 36 mailto:coogeer@yahoo.com Organizational Learning in the Evaluation Procedures 37 assessment, user satisfaction, physical access, physical environment.5 All this must necessarily include the relation- ship between organizational politics and organizational commitment, and indeed this type of relationship is an emerging theme in the literature (for example, Wi et al., 2002),6 and is indicative of an in- creased interest in organizational politics beginning in the 1980s.7 Thus, the general research question of this study is: Does organizational learning occur in the procedures of library evaluation? Aim of Study The aim of the study is to describe the nature of organizational learning in academic libraries, particularly, to explore the role of evaluation. Moreover, the na- ture of the relationship between library current practice and an institutional evaluation, as well as the methods by which the libraries enhanced their learn- ing capacities, will also be investigated. Review of Literature Nature of Organizational Learning There is presently an increasing literature and indeed an abundance of case studies on the important subject of organizational learning, which generally seek to explore and suggest solutions for the problems of organizations.8 The general increase in business competition seems to have hastened the concern with organizational learning and structure. Companies ex- perimented with horizontal linkages and other structures in order to enhance their ability to change and innovate.9 A library can and should be considered as a learning organization10 that must take knowledge from different places and reorganize it so that increased value can be achieved.11 Academic libraries need their own method of achieving organi- zational learning. Just how knowledge is managed, and how this management can relate to the goal of becoming a learning organization, and what inhibitors and fa- cilitators may exist are issues that are ad- dressed by this study. What is established from the literature, however, is that such changes will be difficult, that there will need to be a more open humanistic type of culture, and that the role of manage- ment will be critical. The role of leader- ship in achieving business change has an extremely massive literature, and this has only been very partially sampled, mainly only to identify types of learning barriers and learning facilitators, and to establish the need for an open humanistic environ- ment. It is clear also from this sampling that, if libraries are to change, then some sort of transforming vision must be articu- lated. In this way, library workers can be motivated and sustained.12 Institutional Evaluation, Library Evaluation, and Organizational Learning Institutional Evaluation (IE) refers to an evaluation that is conducted by an author- ity for the purpose of the assessment of the achievements in the aspects of the gen- eral administration, the organizational operation, the financial management, the advancement and the maintenances of facilities, and teaching quality concern- ing an institution. In this study, IE is an accrediting evaluation conducted by an external team entrusted by the Ministry of Education in Taiwan. The study focuses on one of the parts of the IE, namely the library of the institution, which is princi- pally designed and utilized for inspecting and investigating the services, the space, the facilities, the members, and the collec- tions of the libraries to examine whether they have met the criteria defined for the evaluation and to provide some advice to the institutions and their libraries. Although there has been recognition of a range of different types of library evaluation, and increasing sophistication in the actual tools of evaluation, there has been insufficient attention to how library evaluation can be integrated into the library as a continuing planning and responding process.13 That is, evaluation should ideally be a built-in process, con- http:process.13 http:sustained.12 http:achieved.11 38 College & Research Libraries January 2008 ducted by the library itself. The pioneer- ing ideas of Argyris and Schon14 continue to remain relevant, as do the communica- tion directions and structures required. In libraries, however, the organizational climate is o en resistant to these types of changes. Directors—and, indeed, the whole staff—become accustomed to standard methods of carrying out library planning and operations.15 In summary, an evaluation provides useful feedback and learning chances for both staff and management of an orga- nization. In other words, an evaluation functions as a part of managing work- based learning and is in itself a learning process.16 Method The study collected information regard- ing the perceptions and the practices of the staff selected from some academic li- braries in Taiwan. Some 24 library person- nel undertook semistructured interviews from August 2002 to January 2003. These interviews explored issues arising from the survey of the related literature. Inter- views were designed and conducted by the researcher. In the study, the indepen- dent variable is Institutional Evaluation as defined by the one conducted by Taiwan’s Ministry of Education in 1997. All the interview data were analysed using two basic methods: (1) the re- sponses to the structured questions were categorised using simple content labels such as “shared decision making” and were then described using descriptive statistics; and (2) the responses to the open-ended items were analysed using a procedure derived from methods used by phenomenological and grounded theory researchers.17 Discussion of Themes Apparent in The Data Introduction Generally during the interviews the dis- cussion covered three broad aspects: (1) the existence and nature of the learning phenomena in libraries; (2) the source and nature of factors influencing new library practices; and (3) the relationship between organizational evaluation and organizational learning. The Themes The researcher identified the following three important themes that emerged from the extensive interviews: 1. The IE causes more problem than it helps in finding solutions to the existing problems at the library; 2. Leadership and communication play important roles in the organizational learning of the library; and 3. Some political factors intervene in the IE procedure. 1. The IE causes more problems than it helps in finding solutions to the existing problems at the library Many of those interviewed perceived that the IE caused more problems at the library rather than helping the library to find solutions to the problems already there. The following comments illustrate the range of thinking and the complexity of some of the communication and orga- nizational variables. The [worst] thing is that the IE was just an event instead of a continuous activity. So the influence of the IE [on the institution and the library] is really limited. I felt frustrated when the IE brought us some meaningless burdens rather than professionalism. The IE had been more concerned with the parts of the library that dealt with the users of the library or customer service. This superficiality derives from an ap- parent ignoring of the staff, their needs, and their patterns of communication, and their desire to participate in decision making: The IE is user-oriented rather than management-oriented or staff- http:researchers.17 http:process.16 http:operations.15 Organizational Learning in the Evaluation Procedures 39 oriented. As a result, the admin- istrators of the institutions, the library directors and even the staff themselves didn’t really care about how the staff members can benefit because of the IE, especially in the aspect of individual learning or self-growth. The whole IE system should be largely improved. The major im- provement should concentrate on the opinions of the librarians, par- ticularly the staff at the bo om of the library organizations. The IE should give opportunities to all the library staff to express their opinions and encourage them to share their creativities and experi- ence at work. Directors of the library saw at least some IE evaluators as unprepared and unprofessional, ill-trained to offer any viable solutions for the libraries. One director reported the lack of any demon- strable effect, while another expanded on this notion, arguing for a redesign of the “whole IE system/program”: The IE evaluators didn’t have enough time to explore the real situations and some evaluators didn’t even have suitable experi- ence and professional background to judge and consider what [are] the real meaningful points of the IE to the libraries. We can’t expect the evaluators to give us great help when the evalu- ators only stay one day in the insti- tution. If we want the IE to really do something in facilitating library organizational learning, the whole IE system/program should be re- designed. The IE then did li le to increase orga- nizational learning in the minds of the interviewees. Although months were spent in preparing for the IE’s arrival, the effort made did not seem to have any payoff. The long period of preparation and the time involved made the perceived lack of benefit appear much worse. How- ever, some interviewees thought that the library had benefited to some extent from the IE, although the benefits came about indirectly rather than as a direct result of the actions of the IE evaluators. Neverthe- less, some of the comments, arranged here in an order to elucidate the point being made, demonstrate that, in the minds of some respondents, IE can be related to organizational learning: The IE did bring us some pres- sures but that is good. The pres- sures forced us to think more and deeper. We had a consensus: doing our best to get good results [ie, to pass the IE]. As a result, we could tolerate some extra jobs and a tough leader- ship. You could find their potentials come out under pressure. I think only at that time, the staff were likely to contribute to organizational learning. Although we know that the IE was not flawless yet I would consider the IE as a kind of ‘facilitator’ which improved the whole school and library. The IE was like an engine which started our power to think deeply, not for everything but it did hap- pen…. More or less, we had some arguments, discussions, negotia- tions, and experience sharing in the groups. These made everybody [learn]. There was a positive comment saying that the IE visit o en caused the director and the staff to work together in an active manner, thereby increasing the likelihood 40 College & Research Libraries January 2008 of improved organizational learning to occur. One of the significant examples [in which] the IE affected organiza- tional learning was that we like to visit other academic libraries. This is a good opportunity to learn col- lectively. Again, it seemed to be apparent that organizational progress only seemed possible in how individuals might relate to each other and, more particularly, how management might implement its role. As head of the library, I tried to encourage the staff to study some work-related materials and to host some “reading meetings” but I soon found that they (the staff) didn’t like to do so. They didn’t think their jobs were so important that they had to learn/acquire information constantly. The findings suggest that perhaps the IE experience is be er for those groups that use the experience to communicate problems and suggest solutions for those problems. The following quotation emphasizes the dynamic and external stimulus nature of the IE, even if direc- tions are unclear. The IE evaluators wanted to see how much progress the libraries have made. Therefore the libraries have to keep pace with the IT trend, the growth of users, the improvement of other peer institutions, and so on. The next two quotations note that the increased pressure produces increased communication, and this alone may sug- gest positive effects, with the director’s comment focusing more on learning phenomenon as a viable concept: Although everybody knows that there was something unreasonable and inappropriate in the IE process, I still believe the IE caused more or less learning phenomena in our library. For instance, the communi- cation and discussion among the staff members got increased at the IE time. This moved us to think and try to solve problems by paying less time, money, and work. The IE did affect our library. Firstly, in the year the IE came, we had more meetings than usual, mostly for the IE things, and the meetings were good opportunities for all the staff members to express their opinions effectively. Secondly, we faced more problems which we had to solve and these problems brought us more chances to think more and talk more. Finally, we had more time to work closely and this led our organization to be more passionate. Directors and staff both complained about how the IE did not improve things at the library. The IE had been more concerned with the parts of the library that dealt with the users of the library or customer service. Another common perception concerning the IE is that the IE team paid less a ention to whether there was any organizational learning in the communication processes. 2. Leadership and communication play important roles in the organizational learning of the library In the interviews there appeared to be two types of leaders in the library: (1) those leaders who asked the staff for their ideas but who rarely tried to implement those ideas; and (2) those leaders who were more democratic in their approach to the staff and who consistently tried to implement the staff’s ideas. In fact, it seemed that few directors were willing to encourage suggestive opinions from their staffs, even during the period of the IE. As two staff members reported: Organizational Learning in the Evaluation Procedures 41 The key [to organizational learn- ing] here is the organizational climate rather than how much the individuals learn…. The key fac- tors in organizational climate are the leadership of the library and what the institution’s top adminis- trators consider a library to be. Organizational learning should be encouraged and strengthened from the top administrative level…. [T]he leaders are the central influential source from which the motivation for the staff to share their ideas appears. The comments below indicate a concern with the relationship between organiza- tional climate and upward communication flow. The IE preparation was stimulating but too rushed to really promote organi- zational learning. The last quotation high- lights the role of middle management: The time we prepared the document materials for the IE can be regarded as a good chance to develop or- ganizational learning because we could have more considerations, discussions, arguments, and brain- storming at that time…. We didn’t have enough time and good moods maybe translated this differently to think and communicate. Even if the IE was well intentioned in promoting organizational learn- ing such intentions were invariably frustrated by the lack of concern for learning conditions exhibited by the top administrators, and their practice of responding to the IE in their own peculiar ways. In the large-sized libraries, the “middle level” management such as head of a department and team leaders, play more important roles because they have more possibilities to ‘shape’ their teams. The interview comments suggest that management does have a role in produc- ing both a climate in which organizational learning can occur and suitable commu- nication structures for the expression of democratic ideas. There appears to be an effective component in the data, be- yond roles and structures. Staff equated “openness” with “caring” behavior from the director. An aspect of organizational culture that is integral to the development and effective operation of a learning or- ganization is very much concerned with how employees feel. Leaders who do not respect staff members are also likely to do li le to encourage organizational learning. One library director blamed the problem on the a itudes of the staff at the library: Basically, the middle management of the library has more chance to get touch in different people and things…. In this sense, I should have more chance to think and learn if I want to. Unfortunately I have more duties and pressures than my people do which cause me less eagerness to think or create something new. Such a statement shows that organi- zational learning is not assisted by poor leadership, which may be less likely to provide an appropriate climate for organizational learning. Also, from the interview data, staff can be a barrier to organizational learning. Two directors related that the staff has to be willing to share with each other for an organiza- tional learning to happen. Only when people ask others and share with others does organization- al learning happen. In other words, the more interaction among our staff members, the more possibility organizational learning occurs. The learning things are dependent on how much the staff want to learn and to make changes to the library. 42 College & Research Libraries January 2008 Two other directors took a very nega- tive view of existing organizational cli- mate, worsened, it would seem, by “low pay and lack of challenge”: As head of the library, I have a responsibility to ask my people, especially non-LIS people, to remain active and passionate about learning new things and contributing their ideas to the group… however, I find it pre y difficult to do so because the climate of a library makes the staff like robots. The strange thing is that the library staff only likes to focus on what they are familiar with instead of trying to learn something new to them…. Low pay and the lack of challenges and achievements probably are the reasons to low organizational learn- ing here. The concept is that organizational learning cannot happen when the ideas of everyone in the organization are not given any importance or imperative for action. The more interaction there is among our staff members, the more possi- bility that organizational learning occurs. The library director has power to force the staff to study and share their ideas and comments with others…. The director and the institution should be able to ask the staff to change their learning behavior. One of the important themes that emerged out of the interviews with li- brary personnel was that the hierarchy of the library seemed to reduce organi- zational communication. The theme in particular shows that, for true organi- zational learning to take place, all levels of the library staff must be thoroughly involved in the process. The hierarchy of our library more or less reduces the communication around the whole library. In this situation, the role of middle man- agement becomes more important. More communication among the dif- ferent departments of the library will increase the possibilities of learning. This is because staff members of the different departments may have some different ideas and points of view in doing things—no ma er old routines or new problems. O t h e r c o m m e n t s e x p a n d e d t h i s theme, suggesting that physical size and division of labor among departments is a variable to be considered. My library was pre y small. Except the director, everyone rotated the duties regularly. Therefore in my li- brary, we had few problems in com- munication and everyone had to learn how to face users and how to deal with their questions. This was a good chance to develop individual learning and maybe organizational learning. Throughout the literature, studies have clearly shown the importance of adequate upward and sideways movement and the management of knowledge so that all can make a contribution to a learning organization. The IE resulted in a big change in our organizational structure. The new structure includes more divi- sions (departments) and more staff members. Therefore, I think our library becomes more energized and communicated…. More obvi- ously, the personal experience and knowledge are more frequently used at our workplace. Personal factors such as general job- related motivation and friendship among Organizational Learning in the Evaluation Procedures 43 workers are variables that can be related to the type of communication in library organizations. The IE affected those who had more passions and enthusiasm on their jobs as well as those who were good at communication with other people. Friendship among the staff members was another important factor for the communication in the library. Because we were too busy (in prepa- ration for the IE) to produce our own ideas concerning certain problems, we were ignored…. All we could do was try to cooperate as much as possible with our director and the authority of our institution. This interviewee felt that middle man- agement had to act as a “bridge” between upper management and employees. The interviewees felt that this was particularly true during the stressful time of preparing for the IE. A couple of employees stated: The organizational hierarchy of our library more or less reduces the communication around the whole library. In this situation, the role of middle management becomes more important. Head of departments (the middle management) of the library should have more possibilities to play a key role in encouraging their teams to learn, or at least to express the opinions and suggestions of the staff in the bo om. In our electronic era, librarians are expected to “understand and use the Internet as a tool for the exploitation of information and the delivery of learning activities.” Here an interviewee said: I like to ask my people to talk and use e-mail at work. This is a kind of so-called “experience flow”—the experience concerning the improve- ment and the problems solution. Most data indicate that the organiza- tional structure of most libraries do not help communication flows or problem solving. Particular types of library leaders are seen as having the personal ability to give staff a voice in the whole process. 3. Some political factors intervene in the IE procedure A n o t h e r t h e m e t h a t e m e r g e d wa s that political factors (from the top ad- ministration of the institutions) were perceived to have been associated with how the IE proceeded. Underlying such an interpretation is how libraries view the IE as being politically generated, implemented, and used. Many of the library directors and the library staff actually stated that the IE was mostly politically planned. The biggest effect of the IE on or- ganizational learning here is that we become more adaptive to the changes in IE and especially IE evaluators…. The real aim of the IE should help the libraries and the in- stitutions to explore problems and solve them, not “train” the libraries and the institutions to deal with IE in such a political way. There were some “political reasons” occurring during the IE time. For ex- ample, our school recruited two new persons just half year before the IE came. This is because we had insuf- ficient staff worked for my library and that could be a serious critical point in the IE procedure. This exercise of power or political influence was generally characterized as following instructions—instructions that were at variance with how the respon- dents perceived the IE should proceed: 44 College & Research Libraries January 2008 The biggest gain we got from the IE, I remembered, was that we all learned one thing: we must put our efforts on what the IE cares about. Isn’t this ridiculous? The IE should diminish the chance for the institutional top adminis- tration to intervene in the library’s development. They always think their ways in dealing with the IE and don’t respect the professional- ism of library. Everyone must work harder when the IE was about to come…. What the library staff could do was just to follow the director’s instructions. The director followed the presi- dent’s instructions. The president basically made decisions according to the board members’ ideas…. The institutional top administration hardly respected our [librarians’] professional opinions. Other respondents were less negative in their comments and tried to explain their ideas of the purpose of the IE, which should be to assist the library to be er meet the needs of users: The IE affected us to think at a higher level. In other words, we con- sidered more about the directions of the development of the library. Al- though we probably just think and did nothing about the change but it was a really good thing to think something that we are unlikely to think at usual time. The IE should play a more sig- nificant role in “educating” the top administrators of the institutions to realize how important a library is in the campus and how helpful a library is to its users. The effects of IE, as far as the top ad- ministrators were concerned, seemed to be mainly related to the politics of resource allocation. IE forced the top administrators of the institutions to pay more a ention to the libraries, whereas, in the past, the li- braries were treated as being unimportant. This political influence is recognized in the literature (for example, by Coopey and Burgoyne),18 as indeed the role of specific individuals such as the top administrators in the institutions. In the interviews this political influence was seen as an exercise of power; but, from the employee’s point of view, this was not necessarily in the best interest of the libraries. Discussion The data display a range of perspectives toward the IE. Positive responses gener- ally involved two major comments: (1) in the preparation process, IE did cause more discussions at an organizational level; and (2) during the implementation of the evaluation, IE did help libraries to explore problems, though the responsibil- ity for change was placed upon the library staff. There is a genuine partnership be- tween the evaluation and organizational progress among the libraries. However, on the negative side, the IE was seen to be an evaluation procedure that is outside the range of purposes, procedures, and tasks of the daily life of the libraries. One of the reasons for this is that organizational structure is, of itself, insufficient. Library leadership was also a key variable. In terms of routines, library directors saw the library using routines to effectively respond to problems, all in the context of professional development, discussion, and feedback. The IE gave little direct support of this direction for change, but ironically supplied indirect support by providing a reason for increased discussion, even if it was only how to cope with the increased pressure, and the recognition that the additional workload was somehow of limited relevance to what employees might think libraries should do. In terms of the influence of IE, opinions expressed in the interviews seem far more Organizational Learning in the Evaluation Procedures 45 polarized than the survey data. A salient point of difference was that institutional valuation puts pressure on library di- rectors and staff, but this pressure can increase the potential for change. These respondents were fairly adamant that the IE system was important and did change how the library adapted to its external environment. However, other directors felt that IE had very li le real relationship to organizational learning; and, furthermore, it was not particularly important. Again, some of the differences can be ascribed to interviewees being people who consented to being inter- viewed, because they had something to share. Thus, they are more likely to take a reflective stance. The divergent positions of the directors in this area are indicative of how many libraries are characterized by a mixture of both high and low orga- nizational learning. It seems apparent that when directors reflect on how leadership, the IE, and organizational learning relate to each other, they use as a starting point the types of demands the IE makes. Where the IE relates to quantitative goals (eg, the book collection size) the director must act effectively to achieve those goals. An in- crease in the number of these quantitative related goals seems to be associated with the staff becoming busier, hence reducing the opportunity for general discussion, and was reduced for both individuals and the library as an organization. Thus, quantitative types of goals do not really improve communication in the sense of building a learning organization. Only the routines improve. Much of all this information was set in a political context. Indeed, some interviewees complained that libraries only respond to IE in a political way. IE doesn’t really change learning organiza- tional behavior; it is simply a response to a checklist. Library management does not care at all whether or not staff members benefit either personally or organization- ally from IE. IE is just one more thing to do and the whole perspective is to simply meet the IE criteria. However, even such perspectives admit to the idea that IE can have different effects on different levels of staffing. Some data indicate that preparing for the IE had a positive effect on organi- zational learning. This was because the libraries felt that some self-evaluation was necessary; for example, how to bet- ter serve library users. These types of issues automatically suggested that there might be a “be er” way to do something. The rising demand for academic educa- tion—and hence increased access to li- braries—may be a factor here. The more positive perspective toward the value of IE preparation, and indeed its implementation, was that library workers are more academic and more likely to study in their spare time. In other words, IE was viewed as an academic event for academics; therefore, it would be logical to assume that some sort of learning was taking place. Nevertheless, few inter- viewees could point specifically to exactly what organizational learning was taking place, though most felt that it did exist in their library, in some vague form. Producing an exact correspondence, then, between the IE and organizational learning was not possible in the minds of most of the interviewees. As a result of the IE, there was some sort of advance- ment, probably in stages, but a perennial vagueness seemed to cloud most of the issues. It was fairly clear that reflection on the general nature of organizational learning was not something that regularly occurred. Despite the vagueness and negativ- ity of some interviewees, it is clear that in the minds of some few, the IE did provide a starting point for some sort of organizational change, even if it might be reluctant, inefficient, and misdirected at first. All data point to the perception that institutional evaluation does put pressure on the institution and on indi- vidual staff. This pressure can result in change, and it can allow the library to become more adaptive. The library can 46 College & Research Libraries January 2008 bring about more changes, more consid- eration in different directions, and, most important, more planning. There was always some sort of qualification such that the effectiveness of the relationship depended upon staff personality, com- munication skills, and general concern of the staff about the library. In this sense, organizational effectiveness is built upon the type of staff, their own personalities, and pa erns of motivation. In terms of the model, this is an enabler or disabler of positive perceptions of environmental problems that need addressing. Roberts and Rowley19 point this out: …to consider people as individu- als within organizations, and more significantly, as unique individu- als. Aspects of psychology such as personality, perception, communi- cation and motivation, are closely inter-related and can contribute to our understanding of performance and behaviour at work. There seems to be the suggestion that there is no one preferred style of library organizational learning mode. Once the common elements of purpose, staff cul- ture, motivation, and others are satisfied, different libraries, depending on staff members, will approach organizational growth differently. The interview results are supportive of this although the perceptions of the library directors must be disentangled from the perceptions of the staff members. It must be noted that both staff members and directors considered that leader- ship can affect organizational learning. Furthermore, it was considered that new directors and new external stimuli can produce new and innovative ideas, but some staff members may be more receptive to the idea that they could develop organizational learning and be- gin a process of upward and horizontal communication. It is clear that some staff members want to do that, complaining, for example, that certain directors don’t want to change routines and that the leadership they are experiencing should be more democratic. In general, staff members were divided upon the effects of the IE pu ing stress on the whole organization, stifling indi- vidual learning and innovative responses. However, there is no suggestion that this prevented members doing further study. Indeed, those staff members who didn’t respond to organizational learn- ing opportunities were staff members characterized by directors as unwilling to challenge themselves, preferring to do nothing because organizational learning was perceived as having li le to do with their career. In general, IE was thought to have a be er effect if communication skills existed and if appropriate staff personality was in evidence. The results of this analysis of the in- terview comments suggest that the pres- ence of an open communicative learning culture can be associated with higher organizational learning. However, there appears to be no direct proportional rela- tionship. Higher organizational learning, a desirable level of organizational learn- ing, is a level that can respond proactively to the perceived need for change. Such a need, though, is more likely to emerge when it is noticed—and this means con- tinuing and open environmental scans or an outward looking perspective. Staff meetings focusing on some minimum levels to be achieved are unlikely to have such a wide perspective. Certainly, the intellectual power of all workers is not being used. It is a fact that the majority of interview comments that complained about organizational matters offered no solution whatsoever, and this sort of disempowerment was also commented upon by some of the directors. The General Implications of the Study The study expected to illuminate the re- lationship between external environment variables and organizations in the context of an academic library. More specifically, it was anticipated that the study would Organizational Learning in the Evaluation Procedures 47 contribute to an understanding of meth- ods by which libraries can become more effective learning organizations. Using the model derived from the study, it does seem that academic libraries can begin to understand themselves and respond to the environment, using, in part, the stimulus of the IE. These types of impli- cations are of significance because of the contribution that libraries make to the knowledge production, management, and maintenance within a society. This is indeed their role, but that role should be responsive to their clients and to changes in the profession of knowledge management. The model below (figure 1) identifies both constraints and catalysts and sug- gests points at which intervention and increased collaboration can be most use- ful. Importantly, the model also illustrates some of the basic themes of organizational evaluation. These relate to the previous history and tradition of the organization; the nature of the staff and its management including the leadership style; and the degree of collaborative assistance that the organization can obtain to make both strategic and cultural changes. Limitations of the Study This study employed small-scale inter- views of library directors and staff in Taiwan, within the relatively brief time context of the 1997 Institutional Evalua- tion. Thus, the findings and conclusions FIGURE 1 The Library as a Learning Organization affect Library practices and management Environmental input (eg, IE) Application of normal routines action Development of new routines and strategies action affect (decide) Constraints to learning Management disapproval of new concepts Unsupportive organizational culture, and weak worker motivation Restricted information flow High work intensity Unfavorable environmental influence (eg, coercive policies from institutional policy makers) Catalysts to learning Management facilitation of worker ideas and organizational behavior Humanistic culture and high worker motivation Unrestricted information flow Worker discretionary time available Minimum of restrictive external intervention (decide) 48 College & Research Libraries January 2008 of this study can only be generalized to academic libraries of the particular type studied in Taiwan. In addition, there are practical as well as conceptual limits to the number of variables that can be included in any such study. The variables selected for inclusion in this study are derived from an extensive survey of the literature and from the researcher’s work experi- ence in the library system. The variables selected are thought though to be quite relevant to the objectives of the study in terms of the relationships between and among the main variables. Conclusions This study considers that the libraries have not made significant movement toward becoming learning organiza- tions, simply because the external envi- ronment has not required this. In other words, the results of this study show that few libraries seem to be achieving what might be considered an adequate level of organizational learning because of internal barriers and structural prob- lems that resulted from preexisting com- munication deficiencies and, in some cases, a lack of a humanistic democratic climate. The external nature of the evaluation, and the lengthy time periods between evaluations, are two factors that make the learning response of libraries more diffi- cult. In some respects, the length of time between evaluations automatically colors perceptions of the role of the IE. Ideally, such an IE role will assist the library on a path of continuous organizational adaptation, which involves structural, communication, and climate changes. In a word, libraries must engage in a learn- ing dialogue with their environments. At present, this dialogue seems to be persistently traditional. The nature of the library management is one of the critical factors in bringing about effective organizational respon- siveness to environmental challenges. The director must build a shared vision, establish effective teams, and produce a climate of openness whereby a learning environment can be nurtured. Such an en- vironment allows individuals to construct their own meaningful roles within the organization. The findings of the study indicate that there is potential for some staff to be involved in a more effective organizational learning approach. Also, management has the responsibil- ity to identify groups of stakeholders, and barriers to change, and address their dif- ferent expectations appropriately. There are two major barriers to increased change in libraries: • the traditional top-down communi- cation structures and the lack of channels to facilitate more open learning centred communication; and • the ineffective nature of the IE as evidenced by its concern with short-term performance benchmarks, and its exclu- sion of a genuine organizational partner- ship with libraries. The IE and the libraries must have a congruent organization culture if there is a true collaboration between the two stakeholders. Organizational learning as a concept is receiving increased recogni- tion within Taiwan’s libraries, both by management and staff. It is considered as a positive direction for progress, to be achieved possibly as a result of regular Institutional Evaluation. Recommendations for Further Research Further efforts must be continued to be er understand, conceptualize, and investi- gate the dynamics of organizational learn- ing and how effective interventions and collaborations can be achieved between the organization and the environment.20 Future research will advance both knowl- edge of organizational response and the peculiarities of the library context. More suggestions are: 1. The nature of the model link between “Library practices and management” and “Development of new routines and strat- egies” and how this may be applied to organizational learning; http:environment.20 Organizational Learning in the Evaluation Procedures 49 2. The nature of the model link points governing organizational change; between “Library practices and man- and agement” and “Application of normal 3. Increased use of longitudinal and routines” and how this may affect organi- ethnographic research methods in the zational learning, especially at the choice study of libraries as organizations. Notes 1. C. Weiss, Evaluation (Upper Saddle River, N.J.: Prentice Hall, 1998). 2. K. Forss, B. Cracknell, and K. Samset, “Can Evaluation Help an Organization to Learn?” Evaluation Review 18, no. 5 (1994): 574–91; P.M. Jenlink, “Using Evaluation to Understand the Learning Architecture of an Organization,” Evaluation and Program Planning 17, no. 3 (1994): 315–325; B. Allan, Developing Information and Library Staff through Work Based Learning: 101 Activities (London: Library Association, 1999); K. Chen, “Library Evaluation and Organizational Learning: A Questionnaire Study,” Journal of Librarianship and Information Science 38, no. 2 (2006): 93–104. 3. J.B. Cousins and L.M. Earl, “Participatory Evaluation: Enhancing Evaluation Use and Organisational Learning Capacity,” The Evaluation Exchange 1, no. 3/4 (1995): 2–3. 4. Jenlink, 1994. 5. R. Cullen and P. Calvert, “New Zealand University Libraries Effectiveness Project: Dimen- sions and Concepts of Organisational Effectiveness,” LISR 18 (1996): 99–119. 6. L.A. Wi , A.L. Pa i, and W.L. Farmer, “Organisational Politics and Work Identity as Predic- tors or Organisational Commitment,” Journal of Applied Social Psychology 32, no. 3 (2002): 486–99. 7. M. Valle and L.A. Wi , “The Moderating Effect of Teamwork Perceptions on the Organisa- tional Politics–Job Satisfaction Relationship,” Journal of Social Psychology 141, no. 3 (2001): 379–88. 8. F. Blackler and M. Seonaidh, “Power, Mastery and Organizational Learning,” Journal of Manage- ment Studies 37, no. 6 (2000): 833–52; J. Coopey and J. Burgoyne, “Politics and Organizational Learning,” Journal of Management Studies 37, no. 6 (2000): 869–86; M. Crossan and I. Berdrow, “Organizational Learning and Strategic Renewal,” Strategic Management Journal 24, no. 11 (2003): 1087–1105. 9. R.L. Da , Organisation Theory and Design (Sydney: South-Western, 2004). 10. D.E. Riggs, “A Commitment to Making the Library a Learning Organisation,” College and Research Libraries 58, no. 4 (1997): 297–98; J. Rowley, “The Library as a Learning Organisation,” Library Management 18, no. 2 (1997): 88–91. 11. R.K. Lahti and M.M. Beyerlein, “Knowledge Transfer and Management Consulting: A Look at ‘The Firm’,” Business Horizon, January–February (2000): 65–74; J. Benne , “The Relationship between Team and Organizational Learning,” International Journal of Health Care Quality Assur- ance 14, no. 1 (2001): 14–20; V. Friedman, “The Individual as Agent of Organizational Learning,” California Management Review 44, no. 2 (2002): 70–89. 12. B. Green, R.J. Masi, and R.A. Cooke, “Effects of Transformational Leadership on Subordinate Motivation, Empowering and Organisational Productivity,” International Journal of Organisational Analysis 8, no. 1 (2000): 16–47. 13. Library Evaluation: A Casebook and Can-Do Guide, eds. D.P. Wallace and C.V. Fleet (Engle- wood, Colo.: Libraries Unlimited, 2001). 14. C. Argyris and D. Schon, Organizational Learning: A Theory of Action Perspective (Reading, Mass.: Addison-Wesley, 1978). 15. K. Chen, “Library Evaluation and Organizational Learning: A Questionnaire Study,” Journal of Librarianship and Information Science 38, no. 2 (2006): 93–104. 16. B. Allan, Developing Information and Library Staff through Work Based Learning: 101 Activities (London: Library Association, 1999). 17. V. Minichiello, R. Aroni, E. Timewell, and L. Alexander, In-depth Interviewing: Researching People (Melbourne: Longman Cheshire, 1990); T. May, Social Research (Buckingham, U.K.: Open University Press, 1993); M. Denscombe, The Good Research Guide: for Small-scale Social Research Projects (Philadelphia: Open University Press, 2003). 18. J. Coopey and J. Burgoyne, “Politics and Organizational Learning,” Journal of Management Studies 37, no. 6 (2000): 869–86. 19. S. Roberts and J. Rowley, Managing Information Services (London: Facet, 2004), 71. 20. C. Alderfer and L.D. Brown, Learning from Changing (Beverly Hills, Calif.: Sage, 1975); D. Cherin and W. Meezan, “Evaluation as a Means of Organizational Learning,” Administration in Social Work 22, no. 2 (1998): 1–21; T. Lawrence, M. Mauws, and B. Dyck, “The Politics of Orga- nizational Learning: Integrating Power into the 4I Framework,” Academy of Management Review 30, no. 1 (2005): 180–91.