key: cord-0760980-qi8gz9vl authors: Wood, Richard M.; Murch, Ben J.; Moss, Simon J.; Tyler, Joshua M.B.; Thompson, Alexander L.; Vasilakis, Christos title: Operational research for the safe and effective design of COVID-19 mass vaccination centres date: 2021-05-14 journal: Vaccine DOI: 10.1016/j.vaccine.2021.05.024 sha: 03440acbc975ddd2fcb9c3d8736f4700049732cb doc_id: 760980 cord_uid: qi8gz9vl nan With news, in late 2020, that vaccination against COVID-19 may be up to 95% effective, we have entered a new chapter in our fight against the disease [1] . Restrictions on movement and social contacts can recede as vaccine-acquired immunity reduces susceptibility to infection and (possibly) transmission. A key determinant of this is the speed at which the population can be vaccinated. To facilitate rapid dissemination, many countries are considering mass vaccination centres [2] . Ideally located in large spaces -conference venues or sporting arenas -these sites will immunise hundreds or possibly thousands of individuals each day. Crucial to their success is the safe and effective planning of demand and capacity. If more people are booked than can be seen then large and unmanageable queues will form; compromising social distancing and reducing the likelihood that people will return for their second and final inoculation. If, on the other hand, demand is too far exceeded by capacity then resources are not fully utilised; wasting vaccine and unnecessarily delaying the regression of economically-punitive social restrictions. The question is therefore, how can we safely and sustainably maximise the throughput of these sites? To this end, Operational Research (OR) can be a valuable asset. Containing a range of practically focused and mainly quantitative methods, OR has a track record in addressing questions of this very nature. While OR techniques have a history within the immunology field, e.g. in strategically optimising the extent to which influenza vaccine should be stockpiled or reactively purchased [3] , the more operational question posed here is perhaps better paired to experiences from the healthcare settingwhere modelling patient throughput along some kind of 'pathway' is commonplace [4] . In applying such models to the mass vaccination 'pathway' considered here, we demonstrate an example of the contribution that OR can make in this important next stage of our fight against COVID-19. On 2 December 2020, a live exercise was conducted at one of the sites planned to operate as a mass vaccination centre in the UK. The purpose of Exercise Panacea was to provide a safe learning environment within which to explore processes for administering COVID-19 vaccine to what would be over 1000 people each day. Such exercises are core to health emergency preparedness, supporting the identification of gaps in plans and processes [5] . The exercise involved 'flowing' a number of 'players' through the site, with services provided by members of the 'cast'. Specifically, 70 players were provided with a unique script for each attendance to ensure a range of presentations were considered, e.g. the representation of elderly people or those with hearing or mobility limitations, as well as those with adverse reactions to vaccination. Exercise Panacea took place at Ashton Gate football/rugby stadium in Bristol (UK), where a large rectangular interior hall normally used for spectator catering and entertainment had, in the days before the exercise, been converted to a space in which the four activities necessary within the mass vaccination process could be performed ( Figure 1 ). While hitherto unconfirmed, it was a consideration at the time of the exercise that, when live, there would be 1560 arrivals per 12-hour operating period facilitated by six registration assistants, 12 clinical assessors, six immunisers, and 64 seats for post-vaccination observation, and maximum safe waiting space for six vaccinees before clinical assessment and 15 before vaccination. The overarching vaccination process design had been informed by centrally-produced planning guidance (unpublished) suggesting that immunisers work in teams of two and according to fixed staffing ratios to clinical assessment. Recommendations were that each two-immuniser 'pod' could support a throughput of 520 vaccinations per 12-hour operating period (thus 1560 for six immunisers). Ultimately, the number of pods was limited to six due to spatial constraints of the Ashton Gate site. This also restricted the waiting space within the queues for clinical assessment and vaccination. While some of these operational parameters had been informed by an earlier live exercise (Exercise Asclepius, the only other live exercise of its kind before Exercise Panacea), there remained uncertainties given the novelty of the operation and the intricacies of each vaccination centre (specifically with regard to the physical layout of the site and the type and training of staff). Indeed, a key objective of Exercise Panacea was to test performance under such a configuration. Yet a robust appraisal was not fully possible, not least since only a third of the envisaged operating capacity was used during the exercise. In these situations, computer modelling can be a valuable asset in addressing any such gaps in understanding and considering 'what if' scenarios not possible to examine in real life [6] . Analysis was performed using a versatile open source simulation tool that had been previously been developed by the authors for modelling patient pathways [7, 8] . The tool employs a discrete event simulation method which is well-established in healthcare modelling [9] . This works by simulating the real-life events of vaccinees arriving at the centre, queueing (as necessary), and starting and finishing the various activities along the vaccination pathway ( Figure 1 ). These events are generated according to a given arrival rate and the capacity and service time distributions of each activity (i.e. the model inputs). Simulation outputs, calculated by running multiple (1500) replications of the model, relate to the activity-level numbers of vaccinees in service and in queue over time. With the aforementioned 'baseline' arrival rate and capacity allocations, what remained was to estimate the durations of time vaccinees would spend at each of the four activities ( Figure 1 ). This was achieved by fitting the appropriate statistical distributions to data collected from the exercise (using maximum likelihood estimation with selection through Akaike Information Criterion [10] ). The distribution of registration time was found to be fairly symmetric, and best approximated by a Weibull distribution with a mean and median of 62 seconds. Both clinical assessment and vaccination times were right- Simulation results indicate that the baseline allocation is unviable, with a bottleneck forming at the vaccination activity as characterised by a very high number in service (c.f. capacity of six) and an everincreasing queue (Table 1 , Baseline). This finding is, in fact, evident without modelling -an hourly arrival rate of 130 (i.e.1560 over 12 hours) simply cannot be sustained by a pathway containing an activity whose maximum hourly throughput is only 116 (i.e. six immunisers with 187 second estimated mean service duration). The solution is either to increase capacity or reduce arrivals. With an operational constraint limiting the number of immunisers to no more than six, the arrival rate could be lowered to the level of maximum throughput. While, at first thought, this may seem a reasonable mitigation, it does not appreciate the impact of variability in service duration, which can contribute to the formation of large queues. Although these are smaller than under the Baseline scenario, they still lead to breaches in the 15-space waiting area (Table 1 , Scenario 1). In order to safely accommodate the various peaks and troughs in service duration, the arrival rate should be sufficiently less than maximum throughput [11] . Lowering the arrival rate by 10% (i.e. from 1386 to 1247 over 12 hours), results in performance within operational limits (Table 1 , Scenario 2). It would, however, be prudent to increase the waiting space for vaccination (from 15), in order to absorb any potential 'shocks' relating to periods of elevated demand or staff shortages. Given spatial constraints of the site, this can be achieved by shifting the vaccination space into a reduced-capacity observation space (noting that observation capacity can be safely reduced since it is considerably under-utilised -as shown in Table 1 , the upper 95% CI for number in service (32.3) is approximately half the allocated capacity (64)). Registration and clinical assessment are also under-utilised, implying uneconomic use of available resource. Modelling a one-sixth capacity reduction (i.e. to five and ten workers respectively) is not shown to have an adverse performance impact (Table 1, Scenario 3); with the possible opportunity to safely make further reductions, particularly to registration capacity. Poor management of demand and capacity can result in suboptimal use of resources and excessive queueing. If available waiting space is breached then safety may be compromised as social distancing cannot be maintained. Modelling and computer simulation can provide useful insights to improve the design and operational management of mass vaccination centres. The modelling presented here has directly informed operations at the Ashton Gate site. Following our recommendations, the centre went live on 11 January 2021 with an expanded vaccination queueing area and with 1247 vaccinees booked to each 12-hour operating period (i.e. 416 vaccinees per twoimmuniser 'pod'). Site management have reported that, with such an arrival rate, a good balance appears to have been struck between maximising throughput and ensuring patient safety. As such, daily bookings were based upon the 1247 figure for the first six weeks of operation -a time in which any negative patient experience could have generated poor publicity and impacted upon the high levels of public confidence required to ensure good attendance. Beyond the analysis contained here, future work should more formally assess the impact of unforeseen 'shocks' to the vaccination process. In addition to capturing variation in arrivals and service durations (as in this study), it would be prudent to consider the resilience of any setup to the range of 'lowfrequency, high-impact' stochastic events that could be possible. For instance, staff unavailability or a road traffic accident that causes delays followed by a deluge of arrivals. Modelling could be useful in determining the necessary 'slack' in capacity required to safely absorb such shocks. Given the aforementioned intricacies of each vaccination centre, a 'one-size-fits-all' blueprint would unlikely be appropriate. Instead, those involved in setting up and managing different sites should consider the use of bespoke modelling to initialise or optimise their operation. The simulation tool used here is freely available to such ends [7, 8] . With this software, prospective users can experiment with different arrival rates and capacity configurations. The software also has additional functionality to account for time-dependent arrival rates and capacities (for instance, for use in modelling the previously mentioned shocks). As well as demand and capacity management, OR can contribute to effective mass vaccination in a number of other ways. These may include workforce scheduling, predicting no-shows and associated airline-style 'overbooking', and optimising the priority order of individuals for vaccination based upon their risk of severe illness (older people) and/or onward transmission (younger people). If any of these stages are full, then individuals queue in the dedicated waiting areas. Table 1 . Steady-state simulation results for number of vaccinees in service and in queue under the Baseline scenario and hypothetical Scenarios 1 to 3. Arrivals is the number of vaccinees arriving at the site per hour and Capacity represents the maximum number of vaccinees that can concurrently be served within registration, clinical assessment and vaccination respectively. Note, unless otherwise indicated, steady state was reached within the first hour of the 12-hour operating period. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine Vaccinating the UK against covid-19 Exploring the role of mass immunisation in influenza pandemic preparedness: a modelling study for the UK context A systematic literature review of operational research methods for modelling patient flow and outcomes within community healthcare and other settings What is the value of health emergency preparedness exercises? A scoping review study. International journal of disaster risk reduction Modelling activities at a neurological rehabilitation unit The Health Foundation. Developing a versatile tool for modelling pathway capacity in NHS organisations NHS BNSSG CCG Modelling and Analytics. PathSimR model code repository Systems modelling and simulation in health service design, delivery and decision making Information theory and an extension of the maximum likelihood principle Some problems in the theory of queues The authors acknowledge the contributions of Lucy Harries, Elizabeth Luckett, Mark Sanger, Trevor Shippey and Hayley Ware. The authors are also grateful to the anonymous referees for their helpful suggestions that have improved the quality and legibility of this article. This work was supported by The Health Foundation (Evidence into Practice award).