Research purpose: The study aimed to provide information about coverage, service delivery, organisational resources, and medium-term outcomes for a supervisory development programme (SDP) in the hospitality industry.

Motivation for the study: The primary motivation was to assist programme staff to understand their programme and also to generate information that the programme staff could use to reflect on the programme’s performance and future direction.

Research design, approach and method: A mixture of quantitative and qualitative methods was employed, namely: a literature review; personal interviews; and internal programme documentation, such as e-mail correspondence, financial statements, attendance registers, and personnel records. Personal interviews were conducted with two stakeholders, namely the programme manager and programme administrator. The programme data of 69 participants in the SDP were utilised for the evaluation.

Main findings: The evaluation’s main finding concluded that the ‘implemented programme’ was not congruent with the ‘planned programme’.

Practical/managerial implications: It is recommended that programme activities and theory should be re-evaluated, as the programme is used mainly as a training programme for new appointees, rather than a management development programme to create a pool of potential supervisors. Programme uptake and output should also be closely aligned. The length of time that people in the pool of potential supervisors had to wait before they were appointed should also be reduced.

Contribution/value-add: By explicating the basic programme theory and studying programme implementation, this evaluation serves as a starting point for future evaluations of the SDP.

• the high turn-over in the hospitality industry, and at this particular hotel

• the high cost of replacing exiting staff

• lengthy delays in the appointment of new staff as a result of stringent labour legislation

• the higher workload of mid-level managers who often have to perform tasks assigned to incompetent supervisors (according to the programme manager, personal communication, June 2009).

According to the programme manager the main aim of the SDP is to create a pool of competent supervisors who will address the company’s current and future staff needs for these critical positions. There are specific objectives for the participants and two main stakeholder groups of the SDP. For the participants these are:

• gain knowledge about the company as a system

• develop a strategic outlook on the organisation

• learn the skills required for supervisory positions within their business units.

The objective for the managers of participants is improved customer service by supervisors who can exhibit the full range of activities required by their positions.

The objectives for the company are:

• more skilled employees to fill vacant supervisory positions

• improved customer service ratings

• compliance with recruitment and selection standards.

The target population for the SDP was current supervisors within the organisation (inclusive of those recently appointed from outside) and employees not yet in supervisory positions, but identified by their line managers as potential supervisors. Employees from all business units are eligible for selection into the programme, as the same management structure is implemented across the organisation.

Apart from the hotel group, an international hotel school and a private consultancy (represented by a lecturer from a local business school) participated in presenting the SDP. The duration of the SDP is three months and it is presented bi-annually. The programme activities are illustrated in Figure 1, with arrows indicating a direct relationship between activities and shaded activities indicating major assessments.

Figure 1 shows that the SDP begins with a two-hour training session for senior managers and human resource (HR) staff who are responsible for facilitating and supporting the various programme activities. In this session the external consultant introduces the principles and practice of action learning. For participants, the SDP starts with two half-day modules presented by the consultancy. On the first day, a morning module introduces the principles of action learning. During the afternoon session, a business acumen module is presented which introduces business concepts, terminology and knowledge associated with fulfilling a managerial role within a luxury hotel. At the start of the programme the participants are divided into groups (consisting of 4 to 7 employees each) for the purposes of group assignments and an action learning project. The action learning project aims to integrate the different programme activities. At the end of the programme this summative project is presented to a senior management panel who act as assessors.

The next phase of the programme consists of a series of half-day modules, presented to the participants, spaced two weeks apart. Modules include:

• HR Principles

• Financial Principles

• Principles of Food and Beverage Operations

• Principles of Rooms Operations.

A senior person within the relevant business unit conducts each of the four-hour presentations. Practical group assignments that are related to each of the module’s contents are conducted by the participants after the presentations have been made. Assignments are due two weeks after the related module.

Additional programme activities include writing an individual, reflective journal and attending two half-day coaching sessions. The journal writing activity aims to foster introspection during the participant’s development process. Journals are assessed by the programme manager who provides written feedback to participants. The coaching sessions, facilitated by a consultant, present opportunities for applying the knowledge and skills developed during the programme, and for practising questioning, listening and facilitating techniques.

Activities are assessed on an ongoing basis. Students assess their own work, as well as that of their peers. In addition, assessment is conducted by internal and external staff who facilitate the programme activities, the line managers of participants, as well as clients dealing with participants. These assessments are moderated by an international hotel school, which co-sponsors certification of the programme’s successful participants.

According to the programme manager, the SDP is based on action learning principles. A basic definition of action learning emphasises that a group of colleagues who form a mutual support structure examine actual, work-related problems and then take action to solve these problems (McGill & Beaty, 1995).

In general, programme staff hold assumptions about how their programme will change the programme’s recipients. To elicit these assumptions, the evaluators used a model proposed by Rossi, Lipsey and Freeman (2004) which delineates programme theory into three main parts:

• programme impact theory (a causal theory which describes the changes expected in the target population as a result of programme activities)

• a service utilisation plan (how the target population will interact with the programme services)

• an organisational plan (whether or not the programme has sufficient resources for effective service delivery).

Although the SDP did not have a formalised programme theory in place, the programme manager could articulate specific programme goals. The evaluators elicited these goals by means of a series of semi-structured and structured interviews with the programme staff and an analysis of programme documents. From the interviews and documents it became clear that the hotel’s senior management responded to the problems of high turnover and skills scarcity, at the supervisory level, with a management training and education programme. Figure 2 reflects the assumptions of senior management regarding the change that the SDP will bring about in the participants.

The programme’s impact theory, as set out in a causal chain in Figure 2, shows how the SDP will lead to changing the state of affairs identified as organisational problems by senior management (i.e., reduced recruitment costs and increased client satisfaction).

In order to elicit the programme’s service utilisation plan, the evaluators attempted to analyse the programme activities. The programme documentation did not contain a detailed description of these activities, apart from a diagram showing the sequence of activities and their duration. In this regard, the designers of the SDP assume that attendance of seven half-day modules on specific topics, followed by individual and group action learning projects and finally coaching sessions and reflective journal keeping, would bring about the intended changes in the participants.

In terms of the organisational plan, the following implicit assumptions of the programme manager and programme sponsors were elicited:

• in-house senior management and a consultant would constitute sufficient resources to deliver the SDP

• the length of time scheduled for the SDP would be appropriate for such a programme.

FIGURE 1: Programme activities of the supervisory development programme.

• assist programme staff to understand the assumptions of the programme better

• provide programme staff with information they can use to reflect on the programme’s current performance and future direction.

The study also provided an example of how programme theory can be developed for this type of intervention. Furthermore, the study aimed to find out whether or not such an evaluation can be used within the hospitality industry to examine management education programmes.

FIGURE 2: Impact theory of the supervisory development programme.

BOX 1: Questions the evaluators formulated for the formative evaluation of the supervisory development programme.

The following data sources were selected for the service delivery evaluation questions:

• a literature review with the search parameter, ‘action learning’

• hard copies of the conceptual flowcharts of SDP programme activities

• a final flow chart of SDP programme activities in electronic format

• hard copies of internal e-mail messages sent between programme staff during the planning phase of the first SDP in 2006

• hard copies of e-mail messages sent between the programme manager and the external consultant during the planning phase of the first SDP in 2006

• hard copies of assessed group projects collected between 2006 and 2008.

The data for questions resorting under organisational resources included financial records made available by the programme manager. Additionally, an interview with the programme manager provided data relating to time expenditure and the expertise of programme staff.

Finally, the data for the evaluation questions probing medium-term outcomes were obtained from programme records of assessment outcomes and the hotel’s personnel records.

After signing an agreement on the scope of the evaluation and preliminary evaluation questions, the programme manager introduced the evaluator to the programme administrator. Monthly meetings for the duration of the evaluation were scheduled with the programme administrator, and each lasted between 30 minutes and 90 minutes. When face-to-face meetings were not possible, the evaluator communicated with the administrator via e-mail. This was found to be less effective, as numerous e-mail messages were needed to deliver the same results as a short interview.

Meetings with the administrator were focused on developing an understanding of the programme and its activities. Through these meetings, the evaluator also gained access to relevant programme documentation and other information needed to answer the evaluation questions. Four such meetings took place before the evaluator received all the available programme documents. Thereafter, ad hoc meetings and e-mail messages, with probing or clarifying questions, were used successfully to elicit further information. This iterative process was followed until all available programme documentation and information needed to answer the evaluation questions were obtained. The frequency of communication decreased towards the end of the evaluation.

As the evaluation involved human participants, the evaluators obtained ethical clearance for the evaluation from the Research Ethics Committee of the Commerce Faculty at the University of Cape Town.

According to available programme records, 69 participants completed the SDP between 2006 and 2008, as indicated in Table 1.

Groups consisted of between 12 and 15 participants. Mean group size was slightly less than 14 (13.8) for the five SDPs evaluated. Although group size did not increase at all between 2006 and 2008, the yearly output of participants completing the SDP was significantly higher in 2007 and 2008, as compared to 2006. In 2006, with one SDP running, 15 participants completed the SDP. With two SDPs running per year, 28 and 26 participants completed the SDP in 2007 and 2008 respectively.

In order to assess completion rates, the available information on attendance, and marks for the 69 delegates who completed the SDP, were compared to the criteria set by the programme staff. The duly performed criteria and academic standards could be obtained from the marking schedules of the SDPs, and included the following:

• attending all sessions

• absence only with the HR Manager’s permission or a doctor’s note

• handing in all assignments and reflective journals

• handing in all assignments and reflective journals on time

• a minimum average for the course of 50% or more

• no more than one assignment scored between 40% and 49%

• any assignment receiving a mark below 40% would have to be resubmitted in order to obtain a minimum mark of 40% which would allow the candidate to proceed with the course.

If a participant failed to meet either the duly performed criteria or minimum academic standards, he or she would be asked to withdraw from the programme, subject to review by the SDP Academic Board.

As is evident in Table 2, the documentation of programme activities for SDP 2006(1), 2007(1) and 2007(2) was incomplete. This made it difficult to comment on the degree to which participants, who completed the SDP, adhered to the performance criteria stipulated by the programme staff for these specific programmes. In order to answer the evaluation question, an analysis of the available programme documents (attendance registers, marks, and group projects) for the 69 participants, who had completed the SDP, was undertaken. The results are presented in Table 2.

Although there were some missing data regarding marks, all recorded marks met the academic standards set by the programme staff. Significantly, this held true for the two 2008 cohorts for which between 80% and 90% of programme activities were documented. With regards to duly performed criteria:

• five participants did not submit all reflective learning journals during the 2006 SDP

• three participants did not submit all reflective learning journals during the first 2007 SDP

• three participants did not submit all reflective learning journals during the second 2007 SDP

• one participant was absent from one half-day coaching session during the first 2008 SDP

• one participant was absent from one half-day coaching session and one participant did not complete the final written assessment during the second 2008 SDP.

These findings should be interpreted cautiously for two main reasons. Firstly, the data for the 2006 and 2007 SDPs were incomplete, but results for the two 2008 SDPs could be interpreted with a higher degree of confidence because of more complete data sets. Secondly, no records were available to indicate whether or not the failures to perform duly were for reasons that the programme staff would accept as legitimate (operational requirements or illness). It is therefore possible that an even higher percentage of participants performed duly than is indicated in Table 2.

In summary, the available data indicated that the majority of participants, completing the SDP between 2006 and 2008, did so successfully as described by the programme standards. Both attendance and performance, in relation to programme standards, were taken into consideration to calculate the number of successful participants. If the standards set by the programme were adhered to strictly, a total of 55 participants successfully completed the SDP.

b. How many recipients have started but not completed the SDP and why?

All participants who enrolled in 2006 and 2007 completed the programme. Only two participants dropped out of the programme during the evaluation period, one from the 2008(1) SDP and one from the 2008(2) SDP. This constitutes a dropout rate of 2.8% (total n = 71) between 2006 and 2008. The reasons for dropout were not documented. The evaluator was also not able to interview the exiting individuals as they had both resigned from the organisation at the time of the evaluation. From the programme documentation it was clear that both participants exited the programme at an early stage.

There are several possible reasons for the low dropout rate recorded for the SDP between 2006 and 2008. Firstly, upon selection into the programme, performance criteria for the SDP are made clear to the participants. Secondly, participants sign a compulsory bonding agreement to repay the organisation R3500 should they leave the SDP before their successful completion of this programme or should they leave the organisation within six months after their successful completion of this programme. Thirdly, the SDP provides participants with an opportunity for promotion to a supervisory position. Lastly, most of the programme’s activities are scheduled during working hours and thus do not require participants to spend time after hours on developmental activities.

c. From which business units were recipients selected and were all business units equally represented?

The main business units within the hotel are:

• Rooms Division (typically comprising of Front office, Reservation, Housekeeping, Uniformed services, Telephone services and concierge)

• Food and Beverage Division (including specialty restaurants, bars, coffee shops, banqueting, lounges, catering and kitchens)

• Sales and Marketing Department

• Finance Department

• HR Department

• Maintenance Department

• the hotel’s Spa.

The majority of participants were selected from the organisation’s two biggest business units, the Rooms Division and Food and Beverage Division.

In order to answer this evaluation question, a brief literature review was conducted, focusing on the prerequisite activities of an action learning mode of delivery.

A basic definition of action learning describes a group or set of colleagues who form a mutual support structure, who work through actual work-related problems faced by the individuals within the group (McGill & Beaty, 1995). Learning focuses on taking action, as the name of this approach suggests, through the support of peers. Dotlich and Noel (1998, p. 15) describe action learning generically as ‘an alternating series of workshops and field experiences’ and as a context for rapid learning. According to this definition, action learning provides both the tools that participants need to solve business-related problems, as well as the context to engage in problem solving.

Many action learning approaches exist (Smith & O’Neil, 2003). Based on their review of action learning literature published between 1994 and 2000, these researchers identified the following generic components of action learning programmes:

• participants grapple with actual business problems as they occur

• participants work in small and stable learning groups, called sets

• sets meet regularly

• there is a fixed programme duration

• the business problems participants deal with are relevant to their own working environment

• participants take action between meetings to solve the identified business problems they face

• questioning and reflection play a major role in the learning process

• a supportive and collaborative working environment is created in the set.

In Table 3, the generic components described by Smith and O’Neil (2003) are compared with the SDP activities to ascertain the level of congruence between them.

Based on this comparison it became clear that the SDP contained the majority of the action learning components described in the literature review by Smith and O’Neil (2003). The most notable exception was that participants did not deal with actual business problems they were facing at work, but simulated problems related to the various business units. Based on this comparison of action learning literature and SDP programme activities, it can be concluded that the SDP is an action learning programme that includes the majority of activities described in the reviewed action learning literature.

However, is action learning the appropriate approach to address the problems faced by the hotel management? Is action learning a mode of delivery that is commonly used to grow a pool of supervisors, who need to think holistically about their organisation, and have a pro-active management style? Again, the available literature seems to support the choice of action learning in the context described above. Fitzsimmons and Fitzsimmons (1994) highlight the key roles of the service manager as those of independent thought and taking appropriate action. Mumford (1997) describes both planned learning exercises and seizing learning opportunities, as they emerge from the reality of managerial life, as key strategies in management development. McGill and Beaty (1995) describe action learning as a learning and reflective process that takes place with the support of a group of colleagues working with real problems, with the intent of achieving tangible results. Action learning also typically produces unintended competencies such as: gaining increased self-confidence, self-awareness, other-awareness, readiness to take responsibility and initiative, learning a more disciplined way of working, learning how to formulate more informed actions, learning to relate to and communicate effectively with others, and learning to learn (Johnson, 1998).

Action learning theorists paint a picture of action learning that appropriately matches the approach followed by the SDP. This congruence provides some evidence that action learning may be an appropriate approach for supervisory development.

b. Are the actual programme activities aligned with the intended ones?

The answer to evaluation question 4 indicated that the programme activities of the SDP, as intended, were aligned with an action learning mode of delivery. However, the evaluators also examined whether or not these intended activities were the actual activities that participants received whilst on the SDP. We compared the scheduled programme activities for the SDPs running between 2006 and 2008 and the actual activities recorded in available electronic and paper-based programme documents for the same period. It became clear that available documentation for the three SDPs running between 2006 and 2007 was limited. The reason for this was the resignation of the first programme administrator and the subsequent lack of access to her electronic programme documents. During further interviews with the current administrator, it was established that the missing programme activities did in fact take place each year. Programme expenditure (presented in Table 4) for the 2006, 2007(1) and 2007(2) SDPs supported this claim. However, besides expenditure records indicating that the external consultancy was paid for providing services for all five SDPs running between 2006 and 2008, and programme schedules indicating that planned programme activities were scheduled with specific dates, times, presenters and venues, no further records existed. There was no further documented proof that the programme activities were implemented as planned. Based on this partial evidence, it would be safe to say that the majority of programme activities between 2006 and 2007 were planned, scheduled and paid for. Unfortunately, the available programme documents did not allow a definitive statement as to the actual implementation of these activities.

The current programme administrator has been involved with the SDP since the beginning of 2008 and has been able to produce more complete programme documentation. This is reflected in Figure 3, which shows that the number of documented activities increased substantially in 2008.

The planned programme activities remained relatively consistent during the three years covered by the evaluation. Of the 18 to 20 programme activities originally planned (Figure 3) only three were changed between 2006 and 2008.

In summary, it can be concluded that the planned programme activities took place in 2008. The programme showed a high degree of stability over the three years and five programme instances evaluated, with only one new activity added and two frequency adjustments made during this time.

The programme utilised considerable financial, time and human resources. In terms of financial resources, hotel management has spent approximately R350 000 between 2006 and 2008, as Table 4 indicates.

Most of the funds were spent on fees charged by the private consultancy. Accreditation and co-sponsorship of the SDP by the international hotel school, mentioned earlier, required the second largest investment of financial resources. Although total expenditure for the SDP from 2006 to 2008 was considerable, it equated to an average of R5072.46 per participant per three-month programme. It could be argued that the SDP is a cost-effective programme.

Two factors enabled the hotel to keep programme costs relatively low. Firstly, the hotel used its own training venues. Secondly, an agreement between the external consultancy and hotel management existed to train hotel managers to facilitate programme activities and phase out the need for a consultant.

Expenditure of time is another important resource consideration, as time spent by senior managers facilitating the SDP, and participants attending the programme, converts to time being spent on non-core activities, which subsequently results in a decline in productivity. During the three months that the SDP ran, at least fourteen full workdays were spent on training, and this process involved between 12 and 15 participants, twice a year. If not properly planned for, this could add strain to the workload of senior managers facilitating the course, to colleagues of participants who may have to perform extra duties whilst participants are undergoing training, and to the participants themselves. This indirect programme cost should be accounted for in addition to the fees paid to external service providers to achieve a realistic sense of programme cost.

Two staff members were closely and continually involved in the SDP. The programme manager for the SDP is a senior HR executive within the hotel group, and the programme administrator is an HR manager within the hotel who oversaw the daily planning, running and administration of the programme’s activities.

FIGURE 3: Number of planned and documented programme activities from 2006 to 2008.

FIGURE 4:Supervisory development programme graduate appointments into vacant junior management and supervisory positions from 2006 to 2009.

FIGURE 5:Revised programme theory of the supervisory development programme.

The successful completion of all programme activities, according to the standards set by the programme staff, was chosen as an indicator of a skilled supervisor. The data are presented in Table 5.

Of the 69 participants who completed the SDP between 2006 and 2008, 45 were still employed by the organisation at the end of August 2009. If it is assumed that all those who completed the SDP have developed the required supervisory skills, then the simple answer to the evaluation question is that a pool of 45 skilled supervisors is still employed by the organisation.

However, if evidence of actual skill upon programme completion is required, as reflected in the number of trainees who have complied with all the standards and criteria of success, a different answer emerges. There was documented evidence that 55 participants (or 79.7%) adhered to all the standards set by programme staff. Of these 55 skilled participants, 35 (or 63.6%) were still employed by the hotel in 2009. When the six successful SDP graduates who had since been appointed (and, thus, do not form part of the pool of the skilled supervisors that were available for appointment when a vacancy arose) were subtracted from this number, 29 participants remained in the pool.

Within the limitations of available programme records, and the evaluation method chosen, the most accurate answer to this evaluation question is that a pool of skilled supervisors did exist and that it consisted of 29 employees who had successfully completed the SDP. The aforementioned 29 beneficiaries are still employed by the organisation and have not been appointed into vacant junior management or supervisory positions. The organisation can report back to stakeholders that their investment and efforts have resulted in the creation of a pool of 29 employees who have been trained to be competent supervisors and who are ready to be appointed to fill vacancies when they arise. Significantly, the SDP has delivered at least 35 skilled junior managers or supervisors within a period of three years. This result indicates that, to a large degree, the programme is being implemented as planned, up to this point, and has started to deliver its intended short-term outcomes.

b. Are recipients who successfully completed the SDP being appointed into vacant supervisory positions?

According to the programme records, a total of 69 SDP participants completed the programme between 2006 and 2008, of whom 35 could be labelled as having acquired the required skills, in accordance with the programme’s standards. The programme manager also indicated that in some cases newly appointed supervisors, from outside the organisation, were also enrolled in the programme. Table 6 reflects the number of participants appointed prior to and after the SDP.

Of the 55 participants who successfully completed the SDP between 2006 and 2008, 22 (40%) were already employed as supervisors whilst 6 (10.9%) were appointed after completion. Thus, the majority of these appointments took place before SDP enrolment and it could be concluded that only six participants were developed by the SDP and appointed permanently when a vacancy arose, as the programme theory stipulates. The 22 incumbents who were appointed did not fit the current understanding of how the programme was supposed to work, which is not necessarily a negative or an unintended outcome for the programme, but it should be incorporated into the formal programme theory. Furthermore, two internal appointments of non-SDP graduates and two external appointments were made during the evaluation period. These four appointments were not aligned with the programme theory at all. The continued appointment of non-SDP and external candidates could indicate a serious flaw in the implementation of the SDP, with potential repercussions for how it is perceived by the intended target group.

c. How long does a typical recipient have to wait between completion of the SDP and selection into a vacant supervisory position?

This question is only relevant to the six SDP graduates who were appointed after graduation – the 22 who were first appointed and then enrolled on the SDP, of course, experienced no waiting period. We could only find data on four appointments, and they waited on average for a rather lengthy period of 17 months, after completing the course to be appointed.

Twenty-nine successful SDP graduates were still employed by the organisation but had not yet been appointed into supervisor positions. The waiting time by the end of the evaluation period, for these beneficiaries, is indicated below:

• 2006(1) graduates not appointed by the end of August 2009 have not been appointed for 37 months (n = 3)

• 2007(1) graduates have not been appointed for 25 months (n = 3)

• 2007(2) graduates have not been appointed for 22 months (n = 7)

• 2008(1) graduates have not been appointed for 13 months (n = 8)

• 2008(2) graduates have not been appointed for 10 months (n = 8).

Although programme staff make it clear to participants that graduation from the SDP does not guarantee promotion, these findings indicated that the relatively small number of SDP graduates who had been appointed, and who were not incumbent supervisors, were appointed after approximately a year and a half. The majority of SDP graduates who had not yet been appointed into supervisory positions had been waiting for promotion for an average of 21 months.

d. Are sufficient supervisors being developed by the SDP?

This evaluation question was answered by comparing the number of skilled supervisors developed by the SDP between 2006 and 2008 with the vacancies reported by the organisation between 2006 and August 2009, when the evaluation was concluded. This information is reflected in Figure 4.

The majority of supervisory vacancies (32) were filled with employees who were at that time either current or future SDP graduates. Four vacancies were filled with individuals from within the organisation, who were not and did not become SDP graduates, whilst two external applicants were employed as supervisors.

Based on the information gathered, there are two main reasons to believe that the SDP is developing a surplus of skilled junior managers. Firstly, only 50.9% of successful graduates had been appointed into vacancies by August 2009 when this evaluation was concluded. For the 49.1% of delegates not yet appointed, the average waiting time was calculated as 21 months, with certain graduates already awaiting promotion for 37 months. It seems that there were twice as many gra duates as vacancies during the evaluation period. Secondly, interviews with the programme administrator indicated that programme staff were planning to increase the number of SDP delegates during 2009. The uptake for the first 2009 SDP was reportedly 24 participants, with a similarly sized group planned for the second 2009 SDP. This could potentially increase the number of skilled junior managers by almost 48 individuals within one year. If the number of vacancies does not increase accordingly, even more SDP graduates will form part of the pool of skilled supervisors awaiting promotion.

In summary, it could be asserted that the programme theory, as initially formulated by the programme staff, was well implemented up to the creation of a pool of skilled junior managers. Only 10.9% of SDP graduates were appointed from this pool of skilled individuals into vacant supervisory positions. The majority (40%) of supervisors were appointed into vacant supervisory positions first and then enrolled into the programme. Therefore it seems that the SDP has served mainly as a management training programme for incumbent supervisors, rather than as a management development programme for potential supervisors. Based on these findings, a revised programme theory for the SDP, as implemented between 2006 and 2008, is proposed in Figure 5.

This revised programme theory indicates that there are two groups of SDP participants, namely potential supervisors and incumbent supervisors. As indicated in Figure 5, both groups already participated in the SDP. After graduation, only the potential supervisors proceeded to the pool of skilled supervisors. These participants were ready to be appointed into supervisory vacancies and were thus able to attain both the medium and long-term outcomes specified in the original programme theory (Figure 1). After graduating from the SDP, the participants who were incumbent supervisors upon entering the programme (indicated by the grey block and perforated lines in Figure 5) did not form part of the pool of skilled participants who were available to be appointed into vacancies. These participants had already been appointed into supervisory vacancies and, thus, could not attain all medium and long-term outcomes originally specified. Although these participants may also function more efficiently, independently and productively, they did not necessarily help the organisation to reduce the cost of filling vacancies. Participants in this second group might also have been appointed from outside the organisation, thus negating the outcomes of reduced recruitment costs and reduced induction periods.

The importance of this study and its main contribution is that it provides useful information regarding programme theory, implementation and short-term outcomes within a limited time frame and limited resources (Donaldson & Lipsey, 2006). Relevant information regarding the programme’s functioning was collected, interpreted and reported in a user-friendly manner. Based on the main findings of the evaluation, aspects of the programme that required attention or improvement have been highlighted for programme staff. Furthermore, the programme stakeholders have been provided with a revised programme theory. This evaluation has also summarised the available information for the five SDPs running between 2006 and 2008. By explicating the basic programme theory and studying programme implementation, the evaluation also served as a starting point for future evaluations of the SDP.

The main results of the evaluation indicated that the programme was not implemented according to the programme plan. There was a risk that this could lead to unintended consequences, such as the participants’ perception that the organisation had violated the psychological contract (Smithson & Lewis, 2000) of development and promotion. Between 2006 and 2008, the SDP functioned mainly as a training programme for incumbent junior managers. If it was mostly used as a training programme, rather than as a management development programme, then the programme activities and theory should be re-evaluated. There may be more efficient ways than the current SDP to train incumbent managers within the organisation. At present, the evaluation results indicated that the SDPs, running between 2006 and 2008, might not have had the intended outcome of reducing recruitment costs to fill junior management and supervisory positions within the organisation.

Following Mabey and Finch-Lees (2008), this programme was treated as a management learning exercise. Our results indicated that learning indeed took place, as a pool of skilled managers was created. The theory failure (Rossi et al., 2004) that we identified had to do with what had happened further down the outcome chain, in terms of how these newly trained managers were treated.

Donaldson, S.I., & Lipsey, M.W. (2006). Roles for theory in contemporary evaluation practice: Developing practical knowledge. In I. Shaw, J.C. Greene & M.M. Mark (Ed.), The handbook of evaluation: Policies, programs, and practices, (pp. 56–75). London: Sage.

Dotlich, D.L., & Noel, J.L. (1998). Action learning: How the world’s top companies are re-creating their leaders and themselves. San Francisco: Jossey-Bass.

Fitzsimmons, J.A., & Fitzsimmons, M.J. (1994). Service management for competitive advantage. Singapore: McGraw-Hill.

Johnson, C. (1998). The essential principles of action learning. Journal of Workplace Learning, 10(6), 296–300. http://dx.doi.org/10.1108/13665629810236219

Kotter, J. (1990). What leaders really do. Harvard Business Review, 68(3), 103–111. PMid:10104518

Mabey, C., & Finch-Lees, T. (2008). Management and leadership development. London: Sage.

McGill, I., & Beaty, L. (1995). Action learning: A guide for professional, management and educational development. London: Kogan Page.

Mumford, A. (1997). Management development: Strategies for action. (3rd edn.). London: Institute of Personnel and Development.

Paauwe, J., & Williams, R. (2001). Seven key issues for management development. Journal of Management Development, 20(2), 90–102. http://dx.doi.org/10.1108/02621710110382123

Rossi, P.H., Lipsey, M.W., & Freeman, H.E. (2004). Evaluation: A systematic approach. (7th edn.). Thousand Oaks: Sage.

Smith, P.A., & O’Neil, J. (2003). A review of action learning literature 1994–2000: Part 1 – bibliography and comments. Journal of Workplace Learning, 15(2), 63–69. http://dx.doi.org/10.1108/13665620310464102

Smithson, J., & Lewis, S. (2000). Is job security changing the psychological contract? Personnel Review, 29(6), 680–702. http://dx.doi.org/10.1108/ 00483480010296465

Wexley, K.N., & Baldwin, T.T. (1986). Post-training strategies for facilitating positive transfer: An empirical exploration. Academy of Management Journal, 29, 508–520. http://dx.doi.org/10.2307/256221

Zaleznik, A. (1977). Managers and leaders: Are they different? Harvard Business Review, 55(3), 6.