About the Author(s)


Nalia Ismael Email symbol
Instituto Nacional de Saúde, Maputo, Mozambique

Orvalho Augusto symbol
Faculty of Medicine, Eduardo Mondlane University, Maputo, Mozambique

Adolfo Vubil symbol
Instituto Nacional de Saúde, Maputo, Mozambique

Sofia O. Viegas symbol
Instituto Nacional de Saúde, Maputo, Mozambique

Fernanda Miambo symbol
Instituto Nacional de Saúde, Maputo, Mozambique

Instituto Superior de Ciência e tecnologia de Moçambiq, Maputo, Mozambique

Patrina Chongo symbol
Instituto Nacional de Saúde, Maputo, Mozambique

Nédio Mabunda symbol
Instituto Nacional de Saúde, Maputo, Mozambique

Citation


Ismael N, Augusto O, Vubil A, et al. External Quality Assessment Programme for Early Infant Diagnosis of HIV-1, Mozambique, 2011–2014. Afr J Lab Med. 2018;7(1), a664. https://doi.org/10.4102/ajlm.v7i1.664

Brief report

External Quality Assessment Programme for Early Infant Diagnosis of HIV-1, Mozambique, 2011–2014

Nalia Ismael, Orvalho Augusto, Adolfo Vubil, Sofia O. Viegas, Fernanda Miambo, Patrina Chongo, Nédio Mabunda

Received: 01 Aug. 2017; Accepted: 09 Apr. 2018; Published: 11 Oct. 2018

Copyright: © 2018. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This study evaluated a National External Quality Scheme Program for early infant diagnosis of HIV. Fourteen laboratory technicians participated and nine testing panel cycles were sent between 2011 and 2014. The response rate was 100% for the first eight panels, and the number of technicians with a test score of 100% increased during the first three panels. Based on the evaluations of the technicians, the quality of testing for early infant diagnosis of HIV improved over time in the laboratories.

Introduction

Early infant diagnosis (EID) of HIV infection assures early access to antiretroviral therapy for infected children, which significantly improves survival rates and provides substantial benefits.1,2 The antibody-specific methods, such as enzyme immunological assays used to diagnose HIV infection in adults, are not recommended for infant diagnosis, due to the passive transfer of antibodies by the mother to the baby in the uterus until the age of 18 months. Nucleic acid tests such as HIV RNA and DNA polymerase chain reaction (PCR) assays are recommended to diagnose HIV in newborns.3,4

In settings that lack adequate infrastructure and cold-chain transportation systems to process whole blood, dried blood spot (DBS) samples offers many advantages. Dried blood spot testing has simplified HIV screening in newborns, because samples are easy to collect often via heel prick or finger stick and no cold chain transportation system is required as DBS samples are stable at room temperature.5,6,7,8,9,10,11

Participation in an external quality assurance (EQA) programme has been shown to detect possible errors during testing, trace possible corrective actions and improve the quality of testing, and is thus an important tool for laboratory quality assurance.12,13,14,15,16,17,18 Available molecular tests for newborns have high sensitivity and specificity when performed correctly but when the procedures recommended by the manufacturers are not correctly performed it can result in incorrect results, which has serious consequences. Incorrect results may lead to wrong clinical follow-up and a delay in treatment initiation.19,20 To ensure the accuracy and reliability of laboratory test results, participation in an EQA programme is crucial.21

Since 2006, Mozambique has been using PCR-based methods with DBS samples to diagnose HIV in exposed newborns. In 2011, the Instituto Nacional de Saúde of Mozambique, through the National EQA Program, introduced a voluntary EQA scheme designated for each laboratory technician performing PCR-based EID on DBS samples. This report describes the results of four years of evaluation of the National EQA Program scheme in Mozambique.

Methods

Ethical considerations

The ethics committee of the Instituto Nacional de Saúde, Maputo, Mozambique approved the study (study number: 162/CIBS-INS/2017).

Implementation of external quality assurance HIV-1 DNA programme

Between 2011 and 2014, all laboratory technicians performing EID for HIV using DBS were informed about the availability of proficiency testing panels provided by the National EQA Program of Mozambique and were encouraged to enroll and participate. The panels were provided free of charge and sent to each technician for testing within four weeks of receipt. To maintain confidentiality, a unique code was assigned to each technician.

Panel description and preparation

The proficiency testing panels consisted of 20 blinded DBS samples a mixture of negative and positive samples. Negative DBS samples were prepared from a negative blood bag collected by the Blood Bank at Hospital Central de Maputo. Positive DBS samples were prepared from an HIV-positive individual’s blood used for CD4 testing at the Cellular Immunology Laboratory in the same hospital. The positive and negative samples were retested using the Amplicor HIV-1 DNA PCR test, version 1.5 (Roche Molecular Systems, Branchburg, New Jersey, United States), according the manufacturer’s instructions.

Proficiency panel description and sample organisation

Four series were prepared, and each series consisted of 20 blinded DBS samples in a different sample arrangement. Negative and positive specimens were prepared separately, dried overnight, then encoded with a sample and serial number. Each series contained 20 DBS specimens packed into a sealable plastic storage bag stored at 2°C – 8°C until shipment. All four series were sent to each laboratory, and each laboratory technician had to test one of the series sent.

Panel validation

To validate the panel, each series was selected randomly and all 20 DBS samples within the series were retested using the Amplicor HIV-1 DNA PCR test, version 1.5 (Roche, Germany) according to the manufacturer´s instructions, by two different technicians. The results were crosschecked and both results had to match. Following validation, the proficiency panels were sent and made available to all the laboratories across the country performing EID diagnosis.

Sample shipment, analysis, results and reporting

A carrier company shipped the proficiency panels at room temperature three times during 2011 and twice between 2012 and 2014. The technicians had to report the results within 30 days after panel reception. All technicians that did not report their results within 30 days after receiving the panels were considered to be non-responders.

Test scores were reported in percentages according to the number of correct results out of the 20 tests per cycle. Technicians were evaluated according to the concordance of their results with expected results established previously. After each panel, each technician was sent a report containing (1) the test scores (percentage of concordant results), (2) reported results, as well as expected results and possible causes in cases of discordant results. For technicians with results below 100%, corrective actions were conducted, including technical support through laboratory visits, emails or telephone calls.

Statistical analysis

Frequencies and proportions were used to describe the data. Trends between 2011 and 2014 were studied by means of graphs or tables. To study the association between the proportion of technicians scoring 100% on a panel and concordance among technicians, we use a beta-binomial regression with a logit link. We chose this regression to keep the fitted concordance within the 0 to 1 range and to respect the uncaptured geographic heterogeneity (over dispersion) of the data. Based on the Akaike information criterion and Bayesian information criterion, the proportion of technicians scoring 100% was included as linear and quadratic transformed. The quadratic terms were included to detect U-shaped relationships in the data. Stata 14 (StataCorp. 2015. Stata: Release 14. Statistical Software; StataCorp LP, College Station, Texas, United States) was used to perform all analysis.

Results

From 2011 to 2014, nine proficiency panels were sent to each technician. The overall number of technicians that participated increased from 10 in 2011 to 12 in 2014 (Table 1). During the course of the study, there were some withdrawals and replacements for all laboratories with the exception of Laboratory A, where the same technicians were evaluated during the whole course of the study (Table 1). For all panels, results were sent within 30 days of receipt of the panels (78.5% – 100%; average: 97.6%), with the exception of one test panel (2014-B) because three technicians did not respond.

TABLE 1: Overall number of technicians that participated in the National External Quality Assessment Program for polymerase chain reaction of HIV DNA across the nine panels, Mozambique, 2011–2014.

For Laboratory D, 18 errors were observed, and 100% of the technicians had at least one error during the nine panels (Table 2). Laboratory B had the least errors with 2, followed by Laboratory C with 10. At least one error occurred for 16.6% of technicians in Laboratory B and 75% of technicians in Laboratory C. Laboratory A had the second-highest number of errors and 20% of technicians had at least one error.

TABLE 2: Error characterisation of technicians per laboratory during the nine cycles.

The number of false positives increased during the first three panels, while no false positives were observed during the last three panels (2013-B, 2014-A and 2014-B) (Figure 1). On the other hand, the number of false negatives was constant for panels 2011-A, 2011-B, 2011-C, 2013-B, 2014-A and 2014-B. Invalid results were observed during the second panels and re-emerged during the last panel (2014-B). During the first three panels, the proportion of technicians scoring 100% increased, but decreased by 2% during the last two panels (Figure 2).

FIGURE 1: Percentage of discordant results among all participating technicians. Discordant results included false positives, false negatives and invalids.

FIGURE 2: Percentage of technicians scoring 100% versus their concordance during each test panel from 2011–2014. The line is a fit of a beta-binomial regression with linear and quadratic terms of technicians scoring 100%.

Discussion

During the expansion of EID testing and treatment programmes in Mozambique, it was necessary to optimise the quality testing for HIV DNA PCR. We report the successful implementation of the HIV-1 EID EQA Program in Mozambique during the period of 2011 to 2014. Our results show that the quality testing slightly improved over time after the EID EQA program was implemented. If progress is to be maintained, EID EQA program should be sustained.

The number of technicians participating increased during the first eight panels of the programme, with a response rate of 100%, suggesting a positive response and growing interest in the programme. Initially, training on EID for HIV conducted by the laboratory that provided the proficiency testing panels and the clear explanation of the objectives and benefits of the programme to its participants may have accounted for the increased participation. For each panel, technicians with discordant results had direct site supervision and indirect technical support (via email or telephone). The decrease in the response rate during the last panel was most likely related to annual leave during the period that the panel was sent. As the number of samples tested in the country for EID HIV-1 infection increased, the number of technicians participating in the proficiency testing increased over time.

Although corrective actions such as technical support, training and discussions with the laboratory manager were performed, the number of errors for Laboratory D, which accounted for most of the errors during the nine panels, did not decrease. For Laboratory C, most technicians had at least one error over the nine panels. During the retraining and visits, simplification of the procedures was observed, because of the large number of samples being tested. However, for Laboratory A, retraining a technician who was observed making errors helped to overcome some practical issues.

Corrective actions after each EQA panel and retraining of all technicians contributed to a significant reduction of errors over time. Technical assistance decreased in 2013 because of logistical problems faced by the reference laboratory. As a consequence, the number of errors during this period increased, showing the importance of technical assistance to ensure quality testing.

These results reinforce observations made by other studies that training and experience significantly affects the accuracy of proficiency panel testing over time.22,23,24 Furthermore, benefits such as relationship strengthening and communication improvement within the laboratories was observed after the implementation of this EQA programme, as observed in other studies.25

One of the limitations of our study is that we were not able to follow up with the technicians during all the panels and new technicians were enrolled later in the programme, which might have contributed to the increased number of errors. No sociodemographic data, such as gender, education or how long the technician had been familiar with the procedure and number of panels tested were collected. Thus, our study could not assess the association of such factors with errors observed over the time that the programme was evaluated.

A continuous national quality assessment control programme that includes international panels is important for control of the analytical aspects of EID testing for HIV. This EQA programme ensures quality testing at both the technician and laboratory levels. As indicated by this evaluation, improvement in the quality of testing for EID of HIV for the laboratories from 2011 to 2014 was observed. Our results show that both the National EQA Program for EID of HIV testing model of evaluating technicians, as well as the laboratory, are successful in ensuring the quality of results.

Acknowledgements

The authors would like to thank all the laboratories that participated in the National Quality Scheme program for Early Infant Diagnosis of HIV. In addition, they would also like to thank the staff members of the Molecular Virology Laboratory and the National Programme of External Quality Scheme of the Instituto Nacional de Saúde of Mozambique.

Competing interests

The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.

Sources of support

None.

Authors’ contributions

N.I. supported the planning and implementation of the programme and wrote the manuscript; O.A. was responsible for the statistical analysis, and revised the manuscript; A.V. and S.O.V. supported the implementation of the programme and made conceptual contributions; F.M. performed the data analysis; P.C. and N.M led the implementation of the programme and reviewed the manuscript.

References

  1. Berk DR, Falkovitz-Halpern MS, Hill DW, Albin C, Arrieta A, Bork JM, et al. Temporal trends in early clinical manifestations of perinatal HIV infection in a population-based cohort. JAMA. 2005 May 11;293(18):2221–31.
  2. Violari A, Cotton MF, Gibb DM, Babiker AG, Steyn J, Madhi SA, et al. Early antiretroviral therapy and mortality among HIV-infected infants. N Engl J Med. 2008 Nov 20;359(21):2233–44.
  3. Jani IV, Sabatier J, Vubil A, Subbarao S, Bila D, de Sousa A, et al. Evaluation of a high-throughput diagnostic system for detection of HIV-1 in dried blood spot samples from infants in Mozambique. J Clin Microbiol. Apr;50(4):1458–60.
  4. Stevens W, Sherman G, Downing R, Parsons LM, Ou CY, Crowley S, et al. Role of the laboratory in ensuring global access to ARV treatment for HIV-infected children: consensus statement on the performance of laboratory assays for early infant diagnosis. Open AIDS J. 2008;2:17–25.
  5. Sherman GG, Stevens G, Jones SA, Horsfield P, Stevens WS. Dried blood spots improve access to HIV diagnosis and care for infants in low-resource settings. J Acquir Immune Defic Syndr. 2005 Apr 15;38(5):615–7.
  6. Nyambi PN, Fransen K, De Beenhouwer H, Chomba EN, Temmerman M, Ndinya-Achola JO, et al. Detection of human immunodeficiency virus type 1 (HIV-1) in heel prick blood on filter paper from children born to HIV-1-seropositive mothers. J Clin Microbiol. 1994 Nov;32(11):2858–60.
  7. Cassol S, Salas T, Gill MJ, Montpetit M, Rudnik J, Sy CT, et al. Stability of dried blood spot specimens for detection of human immunodeficiency virus DNA by polymerase chain reaction. J Clin Microbiol. 1992 Dec;30(12):3039–42.
  8. Cassol SA, Lapointe N, Salas T, Hankins C, Arella M, Fauvel M, et al. Diagnosis of vertical HIV-1 transmission using the polymerase chain reaction and dried blood spot specimens. J Acquir Immune Defic Syndr. 1992;5(2):113–9.
  9. Biggar RJ, Miley W, Miotti P, Taha TE, Butcher A, Spadoro J, et al. Blood collection on filter paper: a practical approach to sample collection for studies of perinatal HIV transmission. J Acquir Immune Defic Syndr Hum Retrovirol. 1997 Apr 1;14(4):368–73.
  10. Comeau AM, Pitt J, Hillyer GV, Landesman S, Bremer J, Chang BH, et al. Early detection of human immunodeficiency virus on dried blood spot specimens: sensitivity across serial specimens. Women and Infants Transmission Study Group. J Pediatr. 1996 Jul;129(1):111–8.
  11. Boyle DS, Lehman DA, Lillis L, Peterson D, Singhal M, Armes N, et al. Rapid detection of HIV-1 proviral DNA for early infant diagnosis using recombinase polymerase amplification. MBio. Apr 2;4(2).
  12. Hannon WH, Lewis DS, Jones WK, Powell MK. A quality assurance program for human immunodeficiency virus seropositivity screening of dried-blood spot specimens. Infect Control Hosp Epidemiol. 1989 Jan;10(1):8–13.
  13. Hofherr LK, Peddecord KM, Benenson AS, Garfein RS, Francis DP, Ferran KL, et al. Methods for a model blind proficiency testing system. Clin Lab Sci. 1992 May-Jun;5(3):160–4.
  14. Jackson JB, Drew J, Lin HJ, Otto P, Bremer JW, Hollinger FB, et al. Establishment of a quality assurance program for human immunodeficiency virus type 1 DNA polymerase chain reaction assays by the AIDS Clinical Trials Group. ACTG PCR Working Group, and the ACTG PCR Virology Laboratories. J Clin Microbiol. 1993 Dec;31(12):3123–8.
  15. Peddecord KM, Benenson AS, Hofherr LK, Francis DP, Garfein RS, Ferran KL, et al. Analytic results of HIV-1 testing using blind proficiency testing. Clin Lab Sci. 1992 May-Jun;5(3):165–71.
  16. Reichelderfer PS, Jackson JB. Quality assurance and use of PCR in clinical trials. PCR Methods Appl. 1994 Dec;4(3):S141–9.
  17. Rickman WJ, Monical C, Waxdal MJ. Improved precision in the enumeration of absolute numbers of lymphocyte phenotypes with long-term monthly proficiency testing. Ann N Y Acad Sci. 1993 Mar 20;677:53–8.
  18. Schalla WO, Hearn TL, Taylor RN, Eavenson E, Valdiserri RO, Essien JD. CDC’s Model Performance Evaluation Program: assessment of the quality of laboratory performance for HIV-1 antibody testing. Public Health Rep. 1990 Mar-Apr;105(2):167–71.
  19. Parekh BS, Kalou MB, Alemnji G, Ou CY, Gershy-Damet GM, Nkengasong JN. Scaling up HIV rapid testing in developing countries: comprehensive approach for implementing quality assurance. Am J Clin Pathol. Oct;134(4):573–84.
  20. Mashauri FM, Siza JE, Temu MM, Mngara JT, Kishamawe C, Changalucha JM. Assessment of quality assurance in HIV testing in health facilities in Lake Victoria zone, Tanzania. Tanzan Health Res Bull. 2007 May;9(2):110–4.
  21. Hertzberg MS, Mammen J, McCraw A, Nair SC, Srivastava A. Achieving and maintaining quality in the laboratory. Haemophilia. 2006 Jul;12 Suppl 3:61–7.
  22. Learmonth KM, McPhee DA, Jardine DK, Walker SK, Aye TT, Dax EM. Assessing proficiency of interpretation of rapid human immunodeficiency virus assays in nonlaboratory settings: ensuring quality of testing. J Clin Microbiol. 2008 May;46(5):1692–7.
  23. Peeling RW, Smith PG, Bossuyt PM. A guide for diagnostic evaluations. Nat Rev Microbiol. 2006 Dec;4(12 Suppl):S2–6
  24. Chang D, Learmonth K, Dax EM. HIV testing in 2006: issues and methods. Expert Rev Anti Infect Ther. 2006 Aug;4(4):565–82.
  25. Chalermchan W, Pitak S, Sungkawasee S. Evaluation of Thailand national external quality assessment on HIV testing. Int J Health Care Qual Assur. 2007;20(2–3):130–40.