key: cord-0766087-alvv1a1s authors: Jain, Naveena; Moore, Corey B.; Quinn, Emma; Liu, Huei Ming; Liu, Darith; Heaton, Maria; Gehlot, Priyanka; Dhakal, Yashoda; Gupta, Leena; Hogbin, Rebecca; Eastwood, John G. title: Audit of the Sydney Local Health District Public Health Unit notification and contact tracing system during the first wave of COVID‐19 date: 2021-09-02 journal: Aust N Z J Public Health DOI: 10.1111/1753-6405.13145 sha: 767ac806decc04ac77ba8f5f90784ca0043f6e29 doc_id: 766087 cord_uid: alvv1a1s Objective: To conduct a real‐time audit to assess a Continuous Quality Improvement (CQI) activity to improve the quality of public health data in the Sydney Local Health District (SLHD) Public Health Unit during the first wave of COVID‐19. Methods: A real‐time audit of the Notifiable Conditions Information Management System was conducted for positive cases of COVID‐19 and their close contacts from SLHD. After recording missing and inaccurate data, the audit team then corrected the data. Multivariable regression models were used to look for associations with workload and time. Results: A total of 293 cases were audited. Variables measuring completeness were associated with improvement over time (p<0.0001), whereas those measuring accuracy reduced with increased workload (p=0.0003). In addition, the audit team achieved 100% data quality by correcting data. Conclusion: Utilising a team, separate from operational staff, to conduct a real‐time audit of data quality is an efficient and effective way of improving epidemiological data. Implications for public health: Implementation of CQI in a public health unit can improve data quality during times of stress. Auditing teams can also act as an intervention in their own right to achieve high‐quality data at minimal cost. Together, this can result in timely and high‐quality public health data. Due to the potential for the rapid spread of COVID-19 and high mortality rates in at-risk groups, public health measures have been implemented around the world to control the transmission of SARS-COV-2. These measures include education, social distancing and restrictions, cancellation of mass gatherings and isolation or quarantine of at-risk or positive individuals. 9, 10 These measures are informed by surveillance data, which is the systematic and continuous collection, analysis, interpretation and notification of health-related data to prevent and control health problems. 11 Surveillance data help describe the burden of the disease, monitor trends and provide data for setting priorities, designing, executing and evaluating programs and policies. [12] [13] [14] [15] [16] It is crucial that surveillance data are high quality, timely, simple to understand and representative of the population. 17, 18 In New South Wales (NSW), medical practitioners and pathology laboratories are required to report positive cases of COVID-19 under the Public Health Act 2010 (NSW) . As per NSW health guidelines, local public health units have the responsibility to follow up on all cases of COVID-19 and record this information in the state's Notifiable Condition Information Management System (NCIMS). 10 NCIMS is a secure, confidential, online database used to collect public health surveillance data in NSW. It contains information on patient demographic characteristics, laboratory results, clinical symptoms, medical management, likely source of infection, close contacts and the outcome of the disease. NCIMS is continuously updated in real-time with information from public health staff across the state, the Ministry of Health and laboratories. In March 2020, there was a rapid increase in the number of COVID-19 cases in the Sydney Local Health District (SLHD). In response to the increased workload of the district's public health unit (PHU), 73 surge staff were seconded to the PHU in March and remained there for a period of 3-4 months. This increased the number of staff working at the PHU from 58 to 131. These surge staff came from the district's population health, community health and planning teams. The majority of surge staff had no prior training in communicable disease control, case investigation or contact tracing. PHU staff were responsible for supervising and training the surge staff in interviewing, contact tracing, managing close contacts, data collection and the use of NCIMS. PHU staff were also rostered to work on-call after business hours (24/7), in addition to their usual work hours. It was recognised from the outset that the rapid escalation in activity, combined with changes in the workforce members and responsibilities, could adversely affect the quality of public health data. To initially address this issue, the PHU COVID-19 operations team ran a comprehensive staff training program that included completing HETI (Health Education and Training Institute) modules regarding emergency management, as well as surveillance and outbreak management, contact tracing videos, videos on how to use NCIMS, a review of Communicable Disease Network Australia (CDNA) national guidelines and the PHU-specific standard operating procedures, Zoom tutorials and full case simulation. The training program went through phases of the PDSA (Plan Do Study Act) cycle and constant real-time feedback from the staff and the audit team was incorporated. In addition to the staff training program, and to ensure that the epidemiological data were of high quality and timely, a continuous quality improvement (CQI) process was commenced. 17, 18 Shortell's CQI framework was applied to CQI processes as shown in Figure 1 . 19 Shortell's CQI framework is comprised of four dimensions: strategic, cultural, technical and structural. The strategic dimension includes those activities and processes that are most important to the organisation and provide the greatest opportunity for improvement, such as vision, budget priorities and long-term strategy. The cultural dimension represents the organisation's 'beliefs, values, norms and behaviours' that support or inhibit the CQI work. The technical dimension encompasses training and information infrastructure. The structural dimension refers to the ways that knowledge is acquired and dispersed throughout the organisation. The CQI was centred on daily handover meetings that included discussion of cases, management issues, dissemination of decisions and review of processes. Evidence-based presentations on topics of interest were also included. These meetings were instrumental in identifying and understanding changes in guidelines and COVID-19 operational processes and discussing the implications -implementing rapid changes in processes in the unit. To guide and evaluate changes and the performance of the unit, a real-time audit was initiated. The audit was conducted by a team staffed separately to the core COVID-19 operations team, which comprised of two supervisors (a clinical director and epidemiologist) and four other staff (an advanced paediatric trainee, clinical nurse consultant, public health unit trained personnel and registered nurse). In addition to assessing the quality of the data, any information that was identified as missing or incorrect was investigated by the audit team and rectified. The audit was performed at the SLHD PHU, located at Royal Prince Alfred Hospital. All COVID-19 cases managed by the SLHD PHU during the first wave (up until 23 May 2020) were included in the audit. These COVID-19 cases resided within the SLHD, were quarantined in hotels located within the SLHD (including overseas travellers) and had most of their follow up done by the SLHD PHU. COVID-19 cases were excluded if they were diagnosed after death, were subsequently found to have a false-positive result, or their follow up was done by another local health district. Audit of COVID-19 notification system notifications in NCIMS; email communication between the communicable disease team, Ministry of Health, and external agencies; SLHD patient electronic health records; and other electronic information stored by the PHU staff regarding the team's operations in the PHU shared drive and team emails. Where information remained missing or ambiguous, the relevant staff were contacted to clarify the information and cases were re-interviewed as required to improve the completeness and accuracy of the data. Aggregate ordinal variables for overall data quality, completeness, and accuracy were calculated for each case by summing the relevant variables listed above. A measure of PHU workload was also calculated by summing the number of other cases that were reported within three days of each case. Individual logistic regression models were generated for each audit variable, and ordinal logistic regression models were generated for each aggregate variable. The only covariates included in each model were workload and date. These were both normalised using the Ordered Quantile (ORQ) transformation. Data were analysed and visualised using the statistical package, R version 4.0.3. 20 The audit was approved by the SLHD Human Research Ethics Committee (reference LNR X20-0266). There were 304 COVID-19 cases reported to the SLHD PHU from 3 March 2020 to 23 May 2020 (82 days or 12 weeks). The first case was audited on 24 March 2020 and the last case on 12 June 2020. Of the 304 cases, 11 were excluded from this study for the following reasons: one case had an indeterminate result followed by two negative results and was declared a false positive; three cases had a positive result by RT-PCR with a subsequent negative serological result and declared to be false positives; one case was diagnosed after death; five cases were excluded due to crossjurisdictional issues as most of their follow up was done by other local health districts; and one case was lost to follow up after leaving the country. This left 293 COVID-19 cases that were included in the audit. There were 52 cases (17.7%) reported in the first quarter of the period audited, followed by a sharp increase to 217 cases (74.1%) in the second quarter, and 24 cases (8.2%) audited in the final two quarters ( Table 1 ). The number of cases that were re-interviewed by the audit team due to unknown source of infection was 41 (14.0%). Overall, there was an improvement in data quality over time, even after adjusting for workload (p<0.0001, ordinal logistic regression). On further examination, variables measuring completeness were associated with improvement over time (p<0.0001, ordinal logistic regression); whereas those measuring accuracy were inversely associated with increased workload (p=0.0003, ordinal logistic regression). Significant associations with individual audit variables are also shown in Figure 2 . Of note, completeness of linking was the most strongly associated with improvement over time (p=0.0001, logistic regression). After assessing the data for completeness and accuracy, the audit and public health teams corrected the data, resulting in 100% complete and accurate data ( Table 1 , the dashed green line in Figure 2 ). Our study confirms the challenges of achieving high-quality recorded data in a public health unit at the start of a pandemic. High-quality data are required to give an accurate epidemiological picture of cases and contacts to allow for effective and timely decisions to be made regarding controlling the spread of disease. The major challenges faced by the PHU included increased workload on existing staff, changing roles and responsibilities of these staff, new staff not trained in communicable disease control or the existing systems, increased demand on information systems, and evolving guidelines. Realising the importance of accurate epidemiological data early in the pandemic and anticipating that there could be an issue with the quality of the data, a CQI process was employed early to address this. The audit confirmed that there were issues with initial data quality and allowed a measure of the effectiveness of interventions to improve the epidemiologic data that were guiding the public health response. For example, the recording of clinical symptoms was identified as low early in the pandemic because data was being collected manually by one team member who then handed it over for data entry by another team member who prioritised it as low. Information such as this was fed back to the team during the unit's daily meetings to help improve operational processes. Not unexpectedly, our audit showed the lowest quality data at the beginning of the pandemic when these challenges were at their peak and when the number of reported cases was increasing rapidly. However, overall, there was an improvement in data quality over time, even after adjusting for the workload. This likely reflects changes due to quality processes like a comprehensive staff training program and ongoing support and supervision, leading to increased staff experience with the contact tracing and operational processes associated with the COVID-19 pandemic. Interestingly, some variables were not associated with improvement over time but were inversely associated with the workload. Variables measuring accuracy (place of acquisition and source of infection) worsened with A major challenge for the PHU was the shortage of skilled workforce trained in communicable disease control at the beginning of the pandemic. To limit the impact of this in the future, it would seem sensible that identified surge staff were trained in public health concepts and the local public health systems prior to the next wave of the pandemic. 16 However, this may not be easy to achieve politically and may come at a high opportunistic cost during normal business. Using the PDSA model, the extensive staff training program developed at the PHU helped to train the surge staff continuously to a high standard in COVID-19 operations. The staff training program would likely meet the future need to train the surge staff if there were to be another wave of the pandemic. This evaluation is the subject of a separate paper. However, our study results demonstrate that in addition to this training, having an integrated but separate audit team at the beginning of a pandemic to actively review and correct data and maintain high-quality data allows for appropriate delineation of staff roles and can ensure accurate surveillance data at a time of high activity. The audit team was able to achieve 100% complete and accurate data in nearreal-time, with an investment of only 4% of the unit's staff. There were some limitations to this study. The audit covered a period of unusual activity for a PHU, with a lot of uncertainty, rapidly changing information, and staff without a public health background working for the first time in a public health unit. Therefore, the findings from this study may not be generalisable to times with more normal activity. The study did not report on timeliness, which is an important aspect of data quality. In addition, the audit only reviewed data within one local health district of NSW. While other health districts in NSW all use the same database system to record notifiable diseases such as COVID-19 according to the same national guidelines, this information was not analysed. Therefore, our findings may not reflect the quality of data in other districts. However, the audit does provide a reference to compare other NSW local health districts. The audit was also unable to differentiate between the quality of the data entered by the PHU staff versus the surge staff, as many staff members were involved at various stages of contact tracing and data entry. CQI processes can result in improved data quality during times of rapid growth and evolving knowledge. The effectiveness of these interventions can be monitored and guided by real-time auditing. Data identified as incomplete or inaccurate from a realtime audit can be corrected with minimal resources, resulting in high quality and timely data. The SLHD PHU was able to provide highquality contact tracing services and good quality public health data on COVID-19 cases in challenging times due to early deployment of surge staff with a comprehensive training program in COVID-19 operational tasks, implementation of CQI processes guided by real-time auditing, and an auditing team that actively corrected missing and inaccurate data. The CQI processes and the real-time audit team enabled us to make more timely decisions regarding case management and contact tracing, contributing to the statewide reduction in COVID-19 cases. A novel coronavirus outbreak of global health concern Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia Austalian Minister for Health and Aged Care. First Confirmed Case of Novel Coronavirus in Australia Australian Department of Health Risk factors of critical & mortal COVID-19 cases: A systematic literature review and meta-analysis National Center for Immunization and Respiratory Diseases Division of Viral Diseases. People at Increased Risk and Other People Who Need to Take Extra Precautions COVID-19: Advice for Healthcare Workers, Staff, Students and Volunteers. Sydney (AUST): State Government of New South Wales Case-fatality rate and characteristics of patients dying in relation to COVID-19 in Italy COVID-19 National Incident Room Surveillance Team. COVID-19 Australia: Epidemiology Report 31 Isolation, quarantine, social distancing and community containment: Pivotal role for old-style public health measures in the novel coronavirus (2019-nCoV) outbreak Centre for Epidemiology and Evidence. Rapid Response: Notifiable Conditions Public health surveillance in the United States Public Health Surveillance: A Tool for Targeting and Monitoring Interventions A review of data quality assessment methods for public health information systems Electronic health information quality challenges and interventions to improve public health surveillance data and practice Clinical audit, a valuable tool to improve quality of care: General methodology and applications in nephrology Australia's public health response to COVID-19: what have we done, and where to from here? Public Health Surveillance Public Health Surveillance Systems: Recent Advances in Their Use and Evaluation Assessing the impact of continuous quality improvement/total quality management: concept versus implementation R: A Language and Environment for Statistical Computing. Vienna (AUT): R Foundation for Statistical Computing We are indebted to the staff at the public health unit and the surge staff from the population health, community health, and planning teams who were seconded to assist in the notification and tracing of patients. We extend our thanks to Dr Penelope Fotheringham for helping prepare the ethics protocol.