CONFERENCE REPORT
‘Feedback: Where data finally get thrilling’ – tools for facility managers to use data for improved health outcomes in the prevention of mother-to-child transmission of HIV and antiretroviral therapy
J Murphy,1 MPH; C-H
Mershon,2 MPH;
H Struthers,1 MSc, MBA; J McIntyre,1,3 MB ChB, FRCOG
1 Anova Health Institute, Johannesburg, South Africa
2 Gillings School of Global Public Health, University of North Carolina at Chapel Hill, USA
3
School of Public Health and Family
Medicine, University of Cape Town, South Africa
Corresponding
author:
J Murphy
(murphy@anovahealth.co.za)
Data use and data quality continue to be a
challenge for government sector health facilities and districts
across South Africa. Led by the National Department of Health,
key stakeholders, such as the Anova Health Institute and
district health management teams, are aligning efforts to
address these gaps. Coverage and correct implementation of
existing tools – including TIER.net, routine data collection
forms and the South African District Health Information System –
must be ensured. This conference report provides an overview of
such tools and summarises suggestions for quality improvement,
data use and systematic evaluation of data-related
interventions.
S Afr J HIV
Med 2013;14(3):131-134
DOI:10.7196/SAJHIVMED.883
There is increasing recognition of the importance of a functional information-management system to improve health outcomes in South Africa (SA). This is gaining attention through a number of local and international policy documents, including the SA District Health Management Information System (DHMIS) Policy (2011),1 the Aid Effectiveness Framework (2012) 2 and the US President’s Emergency Plan for AIDS Relief (PEPFAR) Partnership Framework.3 With ongoing evaluation and improvement of the SA District Health Information System (DHIS), patients, clinicians and policymakers are ideally positioned to benefit from the improved quality and increased use of routinely collected data at facility, sub-district and district levels. In the case of HIV services, the DHIS can be particularly valuable in determining the number of clients receiving antiretroviral therapy (ART) and in identifying gaps in the prevention of mother-to-child transmission (PMTCT) of HIV services.
The Anova Health Institute
(Anova) recently gathered 160 delegates in Johannesburg for
the symposium ‘Feedback: Where data finally get thrilling’,
to provide an overview of best practice for information use in assessment and improvement
of health services, with an emphasis on HIV treatment and
PMTCT. The target audience included facility managers across
Gauteng Province, with a focus on Johannesburg. Anova partnered
with information and programme managers from provincial and
district government, as well as a variety of non-governmental
organisations (NGOs), to maximise expertise and objectivity on
the issue.
Magnitude of the issue
The DHMIS Policy calls for more than just
addressing data quality; it denotes that information should be
used in programme planning and in clarifying the main roles and
responsibilities ‘for ensuring data completeness, data quality,
and data use and “ownership” at all levels of the health
system.’4
One finding of this symposium was voiced by those in attendance:
the DHMIS Policy is not available or followed by all facility
managers, especially in the areas of data use for programme
decisions and feedback between all levels (facility to
sub-district/district and vice versa).
The DHIS,
which since 1996 has been the sole government repository of
health-related data in SA, has not reached optimal levels of quality, as
documented5 and
anecdotally reported. This holds true for PMTCT as documented
by Mate et al.,6 as well as for
ART data which are not as well documented. Particular areas of
concern include data accuracy, completeness and reliability.
Fortunately, the National Department of Health (NDoH),
facility managers, district DoH structures and NGO partners
have begun the implementation of tools like TIER.net, the
Prevention of Mother-to-Child Transmission Action
Framework and the District Health Barometer (DHB) to
interrogate and better utilise information. In this context,
this conference report is not a declaration of success, but rather a
brief description of the status of our progress in using tools
to strengthen data quality and ease of use.
Conference content
Keynote speaker, Winnie Moleko from the Wits Reproductive Health and HIV Institute (WRHI)/NDoH, presented ‘Data feedback towards quality improvement in service delivery’. Moleko discussed the state of data quality in SA and the role that this plays in quality improvement and implementation of the National Core Standards.7 Practitioners and policy makers can use data to identify gaps in service delivery, resources and facility needs. For data to be useful, they must be correct and accurate; data that are incorrect or presented misleadingly can be detrimental to service delivery and planning. One suggestion that Moleko made, which can be implemented in service facilities, is to post the facility’s data on improvements and achievements in a public place in the facility. This allows staff and clinicians to engage the public and clients in the facility’s data-improvement process.
All presentations are available online
(http://www.anovahealth.co.za/resources/entry/feedback_where_data_finally_gets_thrilling/
). Table 1
summarises the
lessons learnt for clinicians and facility managers working in
the field of HIV. The body of the symposium covered three main
areas: (i) review of data quality and
challenges; (ii) best practice review of data
use for quality improvement; and (iii) data tools available to
facilities, clinicians and policy makers.
Review of data quality and challenges
Mokgadi Morokolo represented Johannesburg Health Information and gave an overview of the DHMIS. She reminded the facility managers in the audience of their responsibility for the data signoff process. This includes a review of the source data such as facility registers, critical analysis of the data outputs, and timely submission of reports and corrections. She emphasised that this is the responsibility of sub-district managers, district directors and hospital chief executive officers (CEOs). These managers are also responsible for improving their knowledge of indicators and maintaining current data-collection tools. District directors are responsible for ensuring that facilities have the current and correct stationery.
Goodwill Kachingwe and Nowinile Dube presented recent
district-level data and highlighted where data can and should be
used at all phases of the programme cycle (Fig. 1). Data are
used in the conceptual phase to help determine what health
outcomes need to be addressed through the programme. Data can be
used in the planning phase to provide insight into where
resources need to be distributed or to provide a baseline for
future evaluation. In the implementation phase, data are used to
monitor the programme implementation or to ensure that target
populations are being reached by the programme. In the
termination phase, data are used to evaluate the success of the
programme, or to determine how the programme has contributed to
district, provincial or national targets.
Fig. 1. Flowchart indicating where data can and should be used at all phases of the DHMIS programme cycle.
Best practice review of data use for quality improvement
Maria Sibanyoni from the WRHI reported on the implementation of a quality-improvement intervention in Johannesburg. 8 The intervention incorporated quality-improvement meetings with staff, collaborative learning workshops, process mapping and a data dashboard to improve initiation and adherence to ART. This effort succeeded in creating an inter-facility referral network and focused on data-driven processes that provided clear and achievable targets for meeting client needs. These achievements can be replicated in other locations.
Theunis Hurter, from Anova’s Cape Winelands project,
demystified TIER.net reporting for the audience.
TIER.net is being expanded into facilities throughout the
country (Fig. 2 shows its growing use in Johannesburg). In the
Winelands, TIER.net has helped clinicians and policy makers at
facilities and the district level to identify defaulters,
track and trace patients, and even identify PMTCT programme
gaps. Specific to PMTCT, Hurter and DoH colleagues in the Cape
Winelands identified, through the use of routine data, that
facilities in the district had initiated ART in more
under-2-year-olds than had been offered PMTCT services – a
clear service-delivery gap. Like this example, one key element
in using data for effective programme and data quality
improvement is the presence of facility managers who empower
their data capturers and others to give feedback on the data
and make note of any trends, issues or remarkable issues in
the data.
Existing data tools available to facilities, clinicians and policy makers
Existing tools, organisations and methodologies are in abundance, but greater coverage and use of these tools is still needed. The DHIS, for example, can be used to identify data quality issues through min/max out-of-range graphs and data completeness reports. The Prevention of Mother-to-Child Transmission Action Framework is effective for target-setting and monitoring programme performance. As much of this information was new to the conference audience, we suggest that raising awareness of these tools is still necessary.
Mashudu Rampilo shared the results of an informal Data Quality Audit comparing source documents (registers) to facility reports and DHIS data specific to HIV testing, the PMTCT programme and ART in Mopani, Limpopo Province. Although from a different setting, the audience was both familiar and shocked with the results. Rampilo’s results showed wide variation and regular disagreement between each of the three data points (the source, facility report and the DHIS). As noted in the DHMIS overview, data accuracy is the responsibility of the staff at facility, sub-district and district level.
One method to improve service delivery at the facility level is treatment-gap modelling. This uses baseline data, national targets and comparisons between people receiving treatment and those eligible for treatment, to estimate where the biggest gaps in service coverage exist, and where more needs to be done to meet local, provincial and national health indicator targets. This approach was adapted from the work of Barker and Venter.10
The available, but under-utilised (as remarked from conference attendees) DHB contains a comprehensive set of indicators to inform planning at all levels in the government and NGO sectors. Candy Day from the Health Systems Trust highlighted how the DHB can be used to provide an overall view of district health performance at the primary healthcare level, and to provide comparative data to monitor the overall quality of service within a district.
One final strategy for data use is the Three Tier ART
Monitoring and Evaluation (M&E) Strategy, of which the ART
M&E standard operating procedures (SOPs) are a key element.
Catherine White presented this tool, which is essential to
quality data collection and use of M&E of ART.
Recommendations
While facilitating the final discussion, Dr Cephas Chikanda, Anova’s Head of Health Systems Strengthening, and Prince Dulaze, Anova’s M&E co-ordinator for Johannesburg, solicited participant feedback to consolidate the key points that the audience had derived from the day’s presentations. The participants’ recommendations included:
• There is a need for better communication about the data within facilities between clinicians, facility managers and data collectors, as well as between the different levels of the health system. For example, facility and district managers need to communicate what the data tell them about service delivery and resources.
• Accountability for the data is the responsibility of everyone, from facility and district data collectors, to district managers and policy makers at the national level. Accountability includes knowing the data elements, what the data reveal about health service delivery and outcomes, and how to accurately and efficiently use data to improve the health system.
• The continuous revision of data-collection tools and systems is a concern. Standardisation of tools and systems according to the DHMIS would facilitate correct and timely completion of collection tools, assist users in becoming familiar and comfortable with the data tools, and make it easier for users and collectors to identify issues and errors. Standardisation is one way to contribute to continuous quality improvement, as well as the development and use of tools and strategies for the immediate- and long-term.
• In order
for the health system to use data most efficiently for its
best effect, it is important to value good quality data as
central to quality healthcare provision and worthy of
investing time and resources. This includes
sharing the results of data collection and interpretation with
health services and the public. Data must also be prioritised
within the system to highlight its worth as a valuable tool to
improve health service delivery.
Acknowledgement.
The conference was funded by PEPFAR through the United States
Agency for International Development (USAID) under co-operative
agreement 674-A-00-08-00009-00 to the Anova Health Institute.
The opinions expressed herein are those of the authors and do
not necessarily reflect the views of USAID/PEPFAR.
References
1. National Department of Health. District Health Management Information System (DHMIS) Policy 2011. Pretoria: DoH, 2011. http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1 October 2012).
2. National Department of Health. The Aid Effectiveness Framework for Health in South Africa. Pretoria: DoH, 2012. http://www.doh.gov.za/docs/stratdocs/2012/aideffect.pdf (accessed 1 October 2012).
3. President’s Emergency Plan for AIDS Relief (PEPFAR). Partnership Framework in Support of South Africa's National HIV & AIDS and TB Response 2012/13 - 2016/17 between the Government of the Republic of South Africa and the Government of the United States of America. Washington: PEPFAR, 2010. http://www.pepfar.gov/countries/frameworks/southafrica/index.htm (accessed 1 October 2012).
4. National Department of Health. District Health Management Information System (DHMIS) Policy 2011. Pretoria: DoH, 2011. http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1 October 2012).
5. Garrib A, Stoops N, McKenzie A, et al. An evaluation of the District Health Information System in rural South Africa. S Afr Med J 2008;98(7):549-552.
6. Mate KS, Bennett B, Mphatswe W, et al. Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS ONE 2009;4(5):e5483. [http://dx.doi.org/10.1371/journal.pone.0005483]
7. National Department of Health. National Core Standards for Health Establishments in South Africa. Pretoria: DoH, 2011. http://www.sarrahsouthafrica.org/LinkClick.aspx?fileticket=YnbSHfR8S6Q%3D&tabid=2327 (accessed 1 October 2012).
8. Webster PD, Sibanyoni M, Malekutu D, et al. Using quality improvement to accelerate highly active antiretroviral treatment coverage in South Africa. BMJ Qual Saf 2012;21(4):315-324. [http://dx.doi.org/10.1136/bmjqs-2011-000381]
9. MEASURE Evaluation. Excel2GoogleEarth (E2G). Chapel Hill: MEASURE Evaluation. http://www.cpc.unc.edu/measure/tools/monitoring-evaluation-systems/e2g (accessed 1 March 2012).
10. Barker PM, Venter F. Setting district-based annual targets for HAART and PMTCT: A first step in planning effective intervention for the HIV/AIDS epidemic. S Afr Med J 2007;95:916-917. http://www.ihi.org/knowledge/Pages/Tools/SouthAfricaHAARTCalculator.aspx (accessed 1 April 2012).