In accordance with World Health Organization Regional Office for Africa (WHO AFRO) resolutions endorsed by member states to strengthen laboratory capacity in the African region, it was recommended that National Public Health Reference Laboratories (NPHRLs) develop and implement integrated Quality Management Systems (QMS) including participation in external quality assessment schemes (EQAS) for all diagnostic and monitoring tests. Additional recommendations included the implementation of integrated, cost-effective and sustainable national EQAS. Moreover, NPHRLs strengthening of their QMS is a gateway to identifying training needs and technical support in addition to attaining accreditation based on international standards.^3,4,5,6

External quality assessment (EQA) is an objective means of assessing the integrity of the entire laboratory testing process, and aims to educate and improve performance in quality assurance (QA) issues.⁷ EQA includes, among others, on-site assessments, and blinded rechecking of previously tested specimens and/or proficiency testing.⁸ Proficiency testing (PT), an essential component of EQA, is an independent means of assessing the quality of the testing process whereby multiple well-characterised specimens are periodically sent to a group of laboratories for analysis and/or testing using routine laboratory procedures. Thus, PT programmes can assist laboratory services to identify factors contributing to errors, in addition to determining whether a laboratory can reliably perform a given test when compared to its peers.

In the African region, few public health laboratories are participating in EQAS for diagnostic and monitoring tests. These laboratories are for the most part limited to central level laboratories, mainly because schemes coordinated by international or regional providers are not able to provide sufficient advice on remedial actions when necessary. Moreover, these schemes are relatively expensive for national governments to extend participation to laboratories in the tiered laboratory network. Hence, in response to requests from member states, WHO AFRO established the regional external quality assessment schemes (REQAS) for anti-HIV serology in collaboration with the National Institute of Communicable Disease (NICD) in South Africa and the National Reference Laboratory (NRL) in Senegal to support the Anglophone and Francophone countries, respectively. The REQAS, established in March 2002, aim to assess the quality of anti-HIV-1 and HIV-2 serological testing for interested laboratories. Additionally, the REQAS allow comparison of testing facilities, in addition to evaluating the quality of serological testing using enzyme immunoassays (EIA) and HIV rapid tests. Participation in the REQAS is voluntary and at no cost to laboratories, thus allowing participation of laboratories in at least one of the REQAS components. The scope of this paper is limited to the REQAS for anti-HIV serology coordinated by NICD and aims to present results of surveys conducted from 2002 to 2010 and challenges encountered. The present article describes results of a REQAS for anti-HIV serology established for public health laboratories in the African region and discusses the implications for efforts aimed at assuring the quality of HIV testing and strengthening public health laboratories towards accreditations in the African region.

From 2002 to 2010, a total of 17 PT panel batches (001– 016), each comprised of ten serum specimens, were dispatched to laboratories. The PT panel batches 001–013, 015–016’ were comprised mainly of HIV-1 sero-positive and sero-negative samples. For PT panel batch 014, two batches (014–1 and 014–2) were dispatched: PT panel batch 014–1 was comprised of four HIV-1 and six sera-negative samples and PT panel batch 014–2 included two HIV-2, four HIV-1 and four HIV sero-negative samples. PT panel batch 014–2 was dispatched to 14 laboratories that responded to a survey confirming capacity to sero-type HIV-2. HIV-2 sero-positive samples were characterised by ELISA and confirmed using the Multispot Rapid Test kit (Bio-Rad laboratories, USA) and Western Blot using NEW LAV BLOT II (Bio-Rad laboratories, USA). All assays used in the characterisation of the PT panels were performed according to manufacturer’s instructions.

Furthermore, the env subtype of 38 HIV sero-positive samples included for PT panel batches 001 to 008 was determined by Heteroduplex Mobility Assay (HMA). Of these, 36 were env subtype C whilst 2 were env subtype B. Due to logistic and cost reasons, env sub-typing of panels was discontinued for PT panel batches 009 to 016. However, it is likely that panels sourced from the South African blood bank were either subtype B or C based on results of surveys of HIV-1 subtype conducted in South Africa.¹⁰

Characterised serum panels were aliquoted in 100µl volumes, labelled and stored at −80 °C until panels were ready for shipment to laboratories. The PT panels containing ten serum samples of known anti-HIV status together with instructions and report forms were distributed to participating laboratories twice a year, in March and October.

In 2002 and 2004, the PT panels were dispatched once to 17 and 34 laboratories, respectively. However, the panels were sent out twice yearly in 2003 and from 2005 to 2010. For each shipment, ten serum specimens in United Nations (UN) approved packages including instruction notes and reporting forms were transported to laboratories by an IATA-certified company.¹¹ To reduce shipping costs to countries with more than one participating laboratory, PT panels were shipped to the respective central level laboratories for onward distribution to other regional or peripheral laboratories. Furthermore, the central level laboratories were also responsible for collating results of the PT testing and reporting to the REQAS coordinators. Laboratories were instructed to return results by either fax or e-mail to the REQAS coordinators within 30 days of receipt of the panels. Additionally, upon receipt of data from laboratories, the REQAS coordinator forwarded the expected results and reports of individual laboratory performance scores on the PT panels. In addition, a distribution report comprising comparative data from all laboratories, complete results of the panel characterisation, test readings as well as methodologies used for testing each panel preparation was generated and forwarded to laboratories.

Of the 49 laboratories, 15 performed rapid HIV testing only whilst seven performed ELISA testing only. Fifteen laboratories performed both ELISA and rapid HIV testing. Ten of 49 laboratories reportedly used line probe assays. Of these, four laboratories used Western Blot assays, four used Inno-lia assay and two used the HIV Blot 2.2. Twelve laboratories did not report on what type of testing was used (Figure 1).

FIGURE 1: Platforms used by participating laboratories to test the proficiency testing panels 2002-2010.

FIGURE 2: Countries currently participating in the WHO/REQAS for anti-HIV serology.

FIGURE 3: Summary of EQA results for the survey period during 2002–2010.

During the survey period, 42 laboratories attained the passing score (≥ 85%) with only seven laboratories receiving unacceptable scores (Figure 3). The overall average score during the survey period was 99% (19.8/20) with a coefficient of variation (CV%) of < 10% for all distributions, with the exception of PT panel batch 005. The CV% for PT panel batch 005 was 16.2% (Figure 4).

FIGURE 4: Coefficient of variation (%) of EQA results during the survey period (2002–2010).

For PT panel batch 013, a panel sample characterised as HIV-1 sero-positive was reported as sero-negative by 12 out of 29 laboratories. To establish the root cause, the sample was included in PT panel batch 014–1 and 014–2. It was noted that 13 out of 36 laboratories still reported the sample as sero-negative. Results of the RCA indicated that laboratories failing to correctly report the sero-status of this sample for PT panel batch 013 and 014 used a fourth-generation ELISA kit as part of their testing algorithm. Additionally, some laboratories also reported faint bands with rapid tests and falsely interpreted the sample as HIV-1 sero-negative.

Thirteen out of 14 laboratories that received PT panel batch 014-2 correctly identified the two HIV-2 specimens. One laboratory failed to return results to the REQAS coordinators.

For PT panel batch 010, some laboratories reported receiving leaked or empty samples upon arrival of the PT panel shipment. However, these samples were excluded from the overall scoring. Hereafter, the REQAS coordinators discontinued the use of these tubes for subsequent distributions.

Retrospective evaluation over the nine-year survey period indicated a 98.8% overall concordance between reported and expected results from participating laboratories, with the highest percent of overall errors found in EIAs. Noticeably, during the survey period, fourth-generation (i.e. combined detection of HIV antigen and antibody) EIA test kits were observed as giving the most discordant results. Interestingly, most laboratories using the HIV Vironostika Uni-form Ag/Ab (BioMérieux, France) switched to using either the Murex HIV 1.2.0 (DiaSorin, Italy) or the Vironostika HIV Uniform II PLUS O (BioMérieux, France). With the exception of challenges with rapid tests kits used by a few laboratories reporting discordant results for PT panel batches 013 and 014, minimal errors appeared with other testing platforms.

The reason for discordant and/or equivocal results with the fourth-generation EIAs is not clear and is beyond the scope of this paper. However, PT panels were characterised using third-generation assays, Genscreen HIV 1/2 (Bio-Rad, Germany), Vironostika Uniform II Plus O (BioMérieux, France) and Murex HIV 1.2.O (DiaSorin, Italy), hence it is important to determine the process of achieving cut-offs for testing using fourth-generation assays to understand the difference in performance to third-generation assays. Other sources of false reporting may be due to inappropriate PT panel storage conditions upon receipt, test kit storage and handling, use of expired test kits as well as failure to follow the routine testing algorithm.

Testing algorithms are defined as the combination and sequence of specific test kits used in a given testing strategy; it also describes the sequence of tests to be performed. The key to achieving the final result is to always follow the sequence of the tests in the algorithm. From results of the nine-year survey, it was apparent that most laboratories failed to adhere to their national algorithms. Some laboratories relied on one test for determining HIV-1 sero-status, whilst others were performing additional testing on PT specimens found to be non-reactive on the first screening test, which often led to inconclusive overall results. Furthermore, laboratories indicated that confirmatory tests were routinely performed on site for all reactive specimens. Hence, it was noted that 95% of laboratories performed confirmatory testing on reactive samples that were part of the PT panel whilst 5% reported performing screening tests on the PT panels. Additionally, some laboratories failed to realise the importance of ongoing participation in an EQAS, proper maintenance of records and application of corrective actions to improve testing services. To address the issues of non-response by participating laboratories which were classified as either panel received but not processed, panel received but problems with testing (either routine or panel-specific) or panel processed but results not submitted or submitted late, the coordinators of the REQAS continued to encourage laboratories to report problems encountered with testing of the PT panels, in case assistance is required in resolving a specific laboratory problem. Furthermore, the REQAS coordinators established the RCA concept for laboratories. RCA aims to determine the reasons(s) for an inadequate or poor performance as well as non-responses to allow the REQAS coordinators to provide the required technical support to solve problems that exist in the laboratories. On the other hand, the RCA allowed laboratories to assess whether the PT panels were received in a satisfactory condition or whether the correct samples were tested using the appropriate method as stipulated in their site-specific standard operating procedures.

The REQAS is at no cost to participating laboratories, as the scheme is funded mainly by WHO AFRO through donor support. With increasing participation of laboratories, the REQAS using liquid serum panels may not be sustainable, especially with high cost of courier transport particularly with the requirements for cold chain transport. To alleviate these challenges, the REQAS coordinators plan to pilot dried tube specimen (DTS) technology in 2011. DTS technology is a practical and cost effective methodology for assuring the quality of serological testing in resource limited settings¹³. Another limitation of the REQAS was the distribution of mostly HIV-1 env subtype B and C panels to laboratories that routinely test non-subtype B or C samples. Henceforth, the coordinators of the scheme have considered inclusion of non-subtype C HIV-1 and HIV-2 sera as part of the PT panel batches.

In summary, analysis of results of the REQAS presented in this report provides evidence that public health laboratories in the African region are capable of accurately determining HIV sero-status albeit with challenges related to human and financial resources. Moreover, the high quality of results is evidence that the majority of African laboratories have the capacity to provide quality-assured data for clinical assessment of patients and/or surveillance programmes.

2. Birx D, de Souza M, Nkengasong JN. Laboratory challenges in the scaling up of HIV, TB, and malaria programs: The interaction of health and laboratory systems, clinical research, and service delivery. Am J Clin Pathol. 2009;131(6):849-851.

3. Burnett D. ISO 15189:2003: Quality management, evaluation and continual improvement. Clin Chem Lab Med. 2006;44(6):733-739.

4. Gershy-Damet GM, Rotz P, Cross D, et al. The World Health Organization African region laboratory accreditation process: improving the quality of laboratory systems in the African region. Am J Clin Pathol. 2010;134(3):393-400.

5. Guzel O, Guner EI. ISO 15189 accreditation: Requirements for quality and competence of medical laboratories, experience of a laboratory I. Clin Biochem. Mar 2009;42(4-5):274-278.

6. Zeh CE, Inzaule SC, Magero VO, et al. Field experience in implementing ISO 15189 in Kisumu, Kenya. Am J Clin Pathol. 2010 ;134(3):410-418.

7. World Health Organization (WHO) TCfDCaPCaTAoPHLA. Guidelines for appropriate evaluations of HIV testing technologies in Africa 2001.

8. Hertzberg MS, Mammen J, McCraw A, Nair SC, Srivastava A. Achieving and maintaining quality in the laboratory. Haemophilia. 2006;12 Suppl 3:61-67.

9. World Health Organization. Guidelines for assuring the accuracy and reliability of HIV rapid testing: Applying a quality system approach; 2005.

10. Wilkinson E, Engelbrecht S. Molecular characterization of non-subtype C and recombinant HIV-1 viruses from Cape Town, South Africa. Infect Genet Evol. 2009;9(5):840-846.

11. Pearson JE. Regulatory constraints for the transport of samples and compliance with the World Organisation for Animal Health (OIE) standards for biosecurity and biocontainment. Dev Biol (Basel). 2007;128:59-68.

12. World Heath Organization. HIV ASSAYS: OPERATIONAL CHARACTERISTICS (Phase I) REPORT 14 SIMPLE/RAPID: World Health Organization; 2004.

13. Parekh BS, Anyanwu J, Patel H, et al. Dried tube specimens: a simple and cost-effective method for preparation of HIV proficiency testing panels and quality control materials for use in resource-limited settings. J Virol Methods. 2010;163(2):295-300.