CONFERENCE REPORT

‘Feedback: Where data finally get thrilling’ – tools for facility managers to use data for improved health outcomes in the prevention of mother-to-child transmission of HIV and antiretroviral therapy

J Murphy,1 MPH; C-H Mershon,2 MPH; H Struthers,1 MSc, MBA; J McIntyre,1,3 MB ChB, FRCOG

1 Anova Health Institute, Johannesburg, South Africa

2 Gillings School of Global Public Health, University of North Carolina at Chapel Hill, USA

3 School of Public Health and Family Medicine, University of Cape Town, South Africa

Corresponding author: J Murphy (murphy@anovahealth.co.za)


Data use and data quality continue to be a challenge for government sector health facilities and districts across South Africa. Led by the National Department of Health, key stakeholders, such as the Anova Health Institute and district health management teams, are aligning efforts to address these gaps. Coverage and correct implementation of existing tools – including TIER.net, routine data collection forms and the South African District Health Information System – must be ensured. This conference report provides an overview of such tools and summarises suggestions for quality improvement, data use and systematic evaluation of data-related interventions.

S Afr J HIV Med 2013;14(3):131-134 DOI:10.7196/SAJHIVMED.883


There is increasing recognition of the impor­t­ance of a functional information-manage­ment system to improve health outcomes in South Africa (SA). This is gaining attention through a number of local and international policy docu­ments, including the SA District Health Management Information System (DHMIS) Policy (2011),1 the Aid Effectiveness Framework (2012) 2 and the US President’s Emergency Plan for AIDS Relief (PEPFAR) Partnership Framework.3 With ongoing evaluation and improvement of the SA District Health Information System (DHIS), patients, clinicians and policymakers are ideally positioned to benefit from the improved quality and increased use of routinely collected data at facility, sub-district and district levels. In the case of HIV services, the DHIS can be particularly valuable in determining the number of clients receiving antiretroviral therapy (ART) and in identifying gaps in the prevention of mother-to-child transmission (PMTCT) of HIV services.

The Anova Health Institute (Anova) recently gathered 160 delegates in Johannesburg for the symposium ‘Feedback: Where data finally get thrilling’, to provide an overview of best practice for information use in assessment and improvement of health services, with an emphasis on HIV treatment and PMTCT. The target audience included facility managers across Gauteng Province, with a focus on Johannesburg. Anova partnered with information and programme managers from provincial and district government, as well as a variety of non-governmental organisations (NGOs), to maximise expertise and objectivity on the issue.

Magnitude of the issue

The DHMIS Policy calls for more than just addressing data quality; it denotes that information should be used in programme planning and in clarifying the main roles and responsibilities ‘for ensuring data completeness, data quality, and data use and “ownership” at all levels of the health system.’4 One finding of this symposium was voiced by those in attendance: the DHMIS Policy is not available or followed by all facility managers, especially in the areas of data use for programme decisions and feedback between all levels (facility to sub-district/district and vice versa). The DHIS, which since 1996 has been the sole government repository of health-related data in SA, has not reached optimal levels of quality, as documented5 and anecdotally reported. This holds true for PMTCT as documented by Mate et al.,6 as well as for ART data which are not as well documented. Particular areas of concern include data accuracy, completeness and reliability. Fortunately, the National Department of Health (NDoH), facility managers, district DoH structures and NGO partners have begun the implementation of tools like TIER.net, the Prevention of Mother-to-Child Transmission Action Framework and the District Health Barometer (DHB) to interrogate and better utilise information. In this context, this conference report is not a declaration of success, but rather a brief description of the status of our progress in using tools to strengthen data quality and ease of use.

Conference content

Keynote speaker, Winnie Moleko from the Wits Reproductive Health and HIV Institute (WRHI)/NDoH, presented ‘Data feedback towards quality improvement in service delivery’. Moleko discussed the state of data quality in SA and the role that this plays in quality improvement and implementation of the National Core Standards.7 Practitioners and policy makers can use data to identify gaps in service delivery, resources and facility needs. For data to be useful, they must be correct and accurate; data that are incorrect or presented misleadingly can be detrimental to service delivery and planning. One suggestion that Moleko made, which can be implemented in service facilities, is to post the facility’s data on improvements and achievements in a public place in the facility. This allows staff and clinicians to engage the public and clients in the facility’s data-improvement process.

All presentations are available online (http://www.anovahealth.co.za/resources/entry/feedback_where_data_finally_gets_thrilling/ ). Table 1 summarises the lessons learnt for clinicians and facility managers working in the field of HIV. The body of the symposium covered three main areas: (i) review of data quality and challenges; (ii) best practice review of data use for quality improvement; and (iii) data tools available to facilities, clinicians and policy makers.


Review of data quality and challenges

Mokgadi Morokolo represented Johannesburg Health Information and gave an overview of the DHMIS. She reminded the facility managers in the audience of their responsibility for the data signoff process. This includes a review of the source data such as facility registers, critical analysis of the data outputs, and timely submission of reports and corrections. She emphasised that this is the responsibility of sub-district managers, district directors and hospital chief executive officers (CEOs). These managers are also responsible for improving their knowledge of indicators and maintaining current data-collection tools. District directors are responsible for ensuring that facilities have the current and correct stationery.

Goodwill Kachingwe and Nowinile Dube presented recent district-level data and highlighted where data can and should be used at all phases of the programme cycle (Fig. 1). Data are used in the conceptual phase to help determine what health outcomes need to be addressed through the programme. Data can be used in the planning phase to provide insight into where resources need to be distributed or to provide a baseline for future evaluation. In the implementation phase, data are used to monitor the programme implementation or to ensure that target populations are being reached by the programme. In the termination phase, data are used to evaluate the success of the programme, or to determine how the programme has contributed to district, provincial or national targets.


Fig. 1. Flowchart indicating where data can and should be used at all phases of the DHMIS programme cycle.

Best practice review of data use for quality improvement

Maria Sibanyoni from the WRHI reported on the implementation of a quality-improve­ment inter­vention in Johannesburg. 8 The intervention incorporated quality-improve­ment meetings with staff, collaborative learning workshops, process mapping and a data dashboard to improve initiation and adherence to ART. This effort succeeded in creating an inter-facility referral network and focused on data-driven processes that provided clear and achievable targets for meeting client needs. These achievements can be replicated in other locations.

Theunis Hurter, from Anova’s Cape Winelands project, demystified TIER.net reporting for the audience. TIER.net is being expanded into facilities throughout the country (Fig. 2 shows its growing use in Johannesburg). In the Winelands, TIER.net has helped clinicians and policy makers at facilities and the district level to identify defaulters, track and trace patients, and even identify PMTCT programme gaps. Specific to PMTCT, Hurter and DoH colleagues in the Cape Winelands identified, through the use of routine data, that facilities in the district had initiated ART in more under-2-year-olds than had been offered PMTCT services – a clear service-delivery gap. Like this example, one key element in using data for effective programme and data quality improvement is the presence of facility managers who empower their data capturers and others to give feedback on the data and make note of any trends, issues or remarkable issues in the data.


Fig. 2. Mapping TIER.net progress in Johannesburg, May 2012.
9


Existing data tools available to facilities, clinicians and policy makers

Existing tools, organisations and methodologies are in abundance, but greater coverage and use of these tools is still needed. The DHIS, for example, can be used to identify data quality issues through min/max out-of-range graphs and data completeness reports. The Prevention of Mother-to-Child Transmission Action Framework is effective for target-setting and monitoring programme performance. As much of this information was new to the conference audience, we suggest that raising awareness of these tools is still necessary.

Mashudu Rampilo shared the results of an informal Data Quality Audit comparing source documents (registers) to facility reports and DHIS data specific to HIV testing, the PMTCT programme and ART in Mopani, Limpopo Province. Although from a different setting, the audience was both familiar and shocked with the results. Rampilo’s results showed wide variation and regular disagreement between each of the three data points (the source, facility report and the DHIS). As noted in the DHMIS overview, data accuracy is the responsibility of the staff at facility, sub-district and district level.

One method to improve service delivery at the facility level is treatment-gap modelling. This uses baseline data, national targets and comparisons between people receiving treatment and those eligible for treatment, to estimate where the biggest gaps in service coverage exist, and where more needs to be done to meet local, provincial and national health indicator targets. This approach was adapted from the work of Barker and Venter.10

The available, but under-utilised (as remarked from conference attendees) DHB contains a comprehensive set of indicators to inform planning at all levels in the government and NGO sectors. Candy Day from the Health Systems Trust highlighted how the DHB can be used to provide an overall view of district health performance at the primary healthcare level, and to provide comparative data to monitor the overall quality of service within a district.

One final strategy for data use is the Three Tier ART Monitoring and Evaluation (M&E) Strategy, of which the ART M&E standard operating procedures (SOPs) are a key element. Catherine White presented this tool, which is essential to quality data collection and use of M&E of ART.

Recommendations

While facilitating the final discussion, Dr Cephas Chikanda, Anova’s Head of Health Systems Strengthening, and Prince Dulaze, Anova’s M&E co-ordinator for Johannesburg, solicited participant feedback to consolidate the key points that the audience had derived from the day’s presentations. The participants’ recommendations included:

• There is a need for better communication about the data within facilities between clinicians, facility managers and data collectors, as well as between the different levels of the health system. For example, facility and district managers need to communicate what the data tell them about service delivery and resources.

• Accountability for the data is the re­sponsi­bility of everyone, from facility and district data collectors, to district managers and policy makers at the national level. Accountability includes knowing the data elements, what the data reveal about health service delivery and outcomes, and how to accurately and efficiently use data to improve the health system.

• The continuous revision of data-collection tools and systems is a concern. Standardisation of tools and systems according to the DHMIS would facilitate correct and timely completion of collection tools, assist users in becoming familiar and comfortable with the data tools, and make it easier for users and collectors to identify issues and errors. Standardisation is one way to contribute to continuous quality improvement, as well as the development and use of tools and strategies for the immediate- and long-term.

• In order for the health system to use data most efficiently for its best effect, it is important to value good quality data as central to quality healthcare provision and worthy of investing time and resources. This includes sharing the results of data collection and interpretation with health services and the public. Data must also be prioritised within the system to highlight its worth as a valuable tool to improve health service delivery.

Acknowledgement. The conference was funded by PEPFAR through the United States Agency for International Development (USAID) under co-operative agreement 674-A-00-08-00009-00 to the Anova Health Institute. The opinions expressed herein are those of the authors and do not necessarily reflect the views of USAID/PEPFAR.

References
    1. National Department of Health. District Health Management Information System (DHMIS) Policy 2011. Pretoria: DoH, 2011. http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1 October 2012).

    1. National Department of Health. District Health Management Information System (DHMIS) Policy 2011. Pretoria: DoH, 2011. http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1 October 2012).

    2. National Department of Health. The Aid Effectiveness Framework for Health in South Africa. Pretoria: DoH, 2012. http://www.doh.gov.za/docs/stratdocs/2012/aideffect.pdf (accessed 1 October 2012).

    2. National Department of Health. The Aid Effectiveness Framework for Health in South Africa. Pretoria: DoH, 2012. http://www.doh.gov.za/docs/stratdocs/2012/aideffect.pdf (accessed 1 October 2012).

    3. President’s Emergency Plan for AIDS Relief (PEPFAR). Partnership Framework in Support of South Africa's National HIV & AIDS and TB Response 2012/13 - 2016/17 between the Government of the Republic of South Africa and the Government of the United States of America. Washington: PEPFAR, 2010. http://www.pepfar.gov/countries/frameworks/southafrica/index.htm (accessed 1 October 2012).

    3. President’s Emergency Plan for AIDS Relief (PEPFAR). Partnership Framework in Support of South Africa's National HIV & AIDS and TB Response 2012/13 - 2016/17 between the Government of the Republic of South Africa and the Government of the United States of America. Washington: PEPFAR, 2010. http://www.pepfar.gov/countries/frameworks/southafrica/index.htm (accessed 1 October 2012).

    4. National Department of Health. District Health Management Information System (DHMIS) Policy 2011. Pretoria: DoH, 2011. http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1 October 2012).

    4. National Department of Health. District Health Management Information System (DHMIS) Policy 2011. Pretoria: DoH, 2011. http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1 October 2012).

    5. Garrib A, Stoops N, McKenzie A, et al. An evaluation of the District Health Information System in rural South Africa. S Afr Med J 2008;98(7):549-552.

    5. Garrib A, Stoops N, McKenzie A, et al. An evaluation of the District Health Information System in rural South Africa. S Afr Med J 2008;98(7):549-552.

    6. Mate KS, Bennett B, Mphatswe W, et al. Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS ONE 2009;4(5):e5483. [http://dx.doi.org/10.1371/journal.pone.0005483]

    6. Mate KS, Bennett B, Mphatswe W, et al. Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS ONE 2009;4(5):e5483. [http://dx.doi.org/10.1371/journal.pone.0005483]

    7. National Department of Health. National Core Standards for Health Establishments in South Africa. Pretoria: DoH, 2011. http://www.sarrahsouthafrica.org/LinkClick.aspx?fileticket=YnbSHfR8S6Q%3D&tabid=2327 (accessed 1 October 2012).

    7. National Department of Health. National Core Standards for Health Establishments in South Africa. Pretoria: DoH, 2011. http://www.sarrahsouthafrica.org/LinkClick.aspx?fileticket=YnbSHfR8S6Q%3D&tabid=2327 (accessed 1 October 2012).

    8. Webster PD, Sibanyoni M, Malekutu D, et al. Using quality improvement to accelerate highly active antiretroviral treatment coverage in South Africa. BMJ Qual Saf 2012;21(4):315-324. [http://dx.doi.org/10.1136/bmjqs-2011-000381]

    8. Webster PD, Sibanyoni M, Malekutu D, et al. Using quality improvement to accelerate highly active antiretroviral treatment coverage in South Africa. BMJ Qual Saf 2012;21(4):315-324. [http://dx.doi.org/10.1136/bmjqs-2011-000381]

    9. MEASURE Evaluation. Excel2GoogleEarth (E2G). Chapel Hill: MEASURE Evaluation. http://www.cpc.unc.edu/measure/tools/monitoring-evaluation-systems/e2g (accessed 1 March 2012).

    9. MEASURE Evaluation. Excel2GoogleEarth (E2G). Chapel Hill: MEASURE Evaluation. http://www.cpc.unc.edu/measure/tools/monitoring-evaluation-systems/e2g (accessed 1 March 2012).

    10. Barker PM, Venter F. Setting district-based annual targets for HAART and PMTCT: A first step in planning effective intervention for the HIV/AIDS epidemic. S Afr Med J 2007;95:916-917. http://www.ihi.org/knowledge/Pages/Tools/SouthAfricaHAARTCalculator.aspx (accessed 1 April 2012).

    10. Barker PM, Venter F. Setting district-based annual targets for HAART and PMTCT: A first step in planning effective intervention for the HIV/AIDS epidemic. S Afr Med J 2007;95:916-917. http://www.ihi.org/knowledge/Pages/Tools/SouthAfricaHAARTCalculator.aspx (accessed 1 April 2012).