Performance of LED fluorescence microscopy for the detection of tuberculosis in Rwanda using Zeiss Primo Star
Performance of LED fluorescence microscopy for the detection of tuberculosis in Rwanda using Zeiss Primo Star
Alaine Umubyeyi Nyaruhirira1, Martine Toussaint2, Bennett Nemser3, Greet Vandebriel4, Michel Gasana2, Yanis Ben Amor3,&
1Management Sciences for Health, Center for Pharmaceutical Management, Pretoria, South Africa, 2TB Division, Rwanda Biomedical Center, Institute of HIV/AIDS, Disease Prevention & Control, Kigali, Rwanda, 3The Earth Institute, Columbia University, New York, New York, USA, 4 ICAP Columbia University, Rwanda
Yanis Ben Amor, The Earth Institute Columbia University 475 Riverside Drive New York, New York 10025 USA
Introduction: Ziehl-Neelsen (ZN) bright-field microscopy is time-consuming, with poor sensitivity, even under optimal conditions. Introduction of Primo Star iLED fluorescent microscopy (FM) may improve TB case finding at referral hospitals in Rwanda. The study aimed to determine the acceptability and effectiveness of iLED in a low resource setting.
Methods: between June 2009 and May 2010, the Rwandan TB Program and National Reference Laboratory carried out demonstration studies with iLED at a referral hospital in the capital,Kigali, and a rural district hospital in Nyamata, taking conventional FM as Gold Standard.
Results: agreement between the iLED and rechecking at the Reference Laboratory were deemed “almost perfect” (kappa = 0.81-1.00) across three of four site-phase combinations. The exception was Nyamata District Hospital during the validation phase, which was deemed “substantial” agreement (kappa = 0.61-0.80). However, the 100% concordance at both demonstration sites during the continuation phase shows technicians' rapid command of the new iLED microscope in a relatively short time. The lower overall positivity rate obtained in the rural clinic is not related to the performance of the microscope (or technicians), but is attributable to a significant increase in total number of patients and samples screened through active case finding.
Conclusion: laboratory technicians demonstrated high acceptance of iLED. Additionally, fluorescent microscopy reduces the time necessary for examination by more than half. The high level of agreement between iLED and FM during implementation in both sites provides initial evidence for iLED to replace current methods.
Sputum smear microscopy for acid-fast bacilli (AFB) using Ziehl-Neelsen (ZN) staining remains the most cost-efficient tool available to diagnose tuberculosis (TB) in low-resource countries. This method is rapid, inexpensive, and highly specific for detecting AFB in high-burden settings. However, the main limitation is its low and variable sensitivity, exacerbated in high HIV prevalence settings . High TB-HIV co-infection rates and low TB case detection impede disease control in many TB endemic settings, notably sub-Saharan Africa . Furthermore, where workloads are high, the amount of time spent examining smears compromises sensitivity . A recent systematic review demonstrated that fluorescence microscopy (FM) is, on average, 10% more sensitive than conventional bright-field microscopy in detecting AFB in clinical specimens, with comparable specificity, and takes significantly less time . However, widespread implementation of FM in disease-endemic settings remains limited due to several factors, including the short life and high cost of mercury vapor lamps; difficulty in maintaining machines; the need for a darkroom; and strict requirements for electrical power supply. Light-emitting diodes (LEDs) for FMhave been identified as an alternative to conventional FM for screening of AFB [5, 6]. LED lamps do not have the disadvantages of mercury vapor lamps, with life expectancy averaging around 50,000-100,000 hours (10-20 years) of use . They can also run on batteries [5, 7, 8]. Several commercial LED systems are now available, either as stand-alone microscopes or as add-on adapters to conventional microscopes . Data published so far on LED microscopy for TB show that results in terms of sensitivity and specificity are comparable or better with LED than mercury vapor lamps [6, 7, 10-13]. Study objectives: this demonstration project evaluated the effectiveness of employing the Primo Star iLED fluorescence microscope (subsequently referred to as iLED) for case finding of TB under routine conditions in one referral and one rural setting in Rwanda. Microscopists without prior experience in FM were solicited in order to determine operational and clinical performance, as well as acceptability of the technology to laboratory staff. Study design: this project was conducted at two sites: Nyamata district hospital (DH) in Bugesera district and the Centre Hospitalier Universitaire de Kigali (CHUK) in Kigali. Nyamata DH is a 100-bed hospital with a catchment area of about 300,000 people. CHUK is a 509-bed national referral hospital (RH) serving the capital city, Kigali, with a catchment area of approximately 1 million people. The implementation of iLED was carried out in five phases:(1) ZN baseline; (2) iLED training and appraisal (five days); (3) validation (one month); (4) implementation (three months); and (5) continuation (six months).
Ethics statement: the evaluation was approved by the Ethical Review Committee of the Ministry of Health (Kigali, Rwanda) under Protocol Number 58/RNEC/2009 and by the Institutional Review Board of Columbia University IRB-AAAC6248 (New York, NY, USA). Written consent was not obtained because microscopy for AFB smears is the standard of care in Rwanda as part of regular clinic monitoring and evaluation of Tuberculosis. During validation, all sputum results obtained though iLED were rechecked systematically by the National Reference Laboratory before results were provided to patients for management. The implementation phase was only allowed once iLED was validated as a replacement for light microscopy with similar or better sensitivity, therefore not placing patients at risk of misdiagnosis. Written consent to participate in the study was not sought from the microscopists or their supervisors because the introduction of the iLED only minimally increased the workload and only for a short duration (1 month) during the whole study. Participants were informed about the purpose and impact of the study, and microscopists readily participated enthusiastically. The need for collecting documented informed consent was waived by the IRB. Phase 1: ZN Baseline: the aim of this one-month phase was to establish a baseline, under study conditions, of false positivity and negativity rates for ZN. TB treatment decisions were based on ZN results. All incoming sputum smears were stained for ZN examination under routine conditions. Slides were read using the available conventional bright-field microscope (1000x). After reading, all slides were kept in slide boxes, which were labeled to specify the study phase, study site, box number, and slide ID. Once every two weeks, the National Reference Laboratory (NRL) study supervisor collected all boxes. NRL study technicians rechecked all slides using conventional bright-field microscopes. Discrepant slides, if any, were sent to the Supra National Reference Laboratory (SNRL) in Germany for rereading. Phase 2: training and Appraisal: a standardized five-day training course for microscopists and supervisors participating in the study was conducted. All eight participants had skills in ZN microscopy but not conventional FM. Participants after learning the purpose and impact of the study, participated readily. Following the training, all technicians involved in the project filled out a questionnaire about several features of the iLED, including installation and first use, training, and optics and handling. Phase 3: validation: the validation phase lasted one month. Each sputum sample at the study sites was stained using Auramine O and examined by the iLED at 400x magnification. Patient management was based solely on the rechecking results carried out by the NRL. Staining solutions were prepared by NRL using Merck staining reagents (Catalogue 41000, Auramine O, item number 1013010050, lot number ZC 253201532) and provided to the study sites once per month, taking into consideration the limited shelf life of Auramine O. All readings (including rechecking) were done within48 hours of staining. Results were quantified according to the scale presented in Table 1. NRL rechecked all slides using conventional FM. Rechecked results were provided to study sites the next day for timely patient management. The semi-quantitative scale for rechecking by NRL was different than the one used by study sites (Table 2). Discordant slides, if any, were sent to the SNRL for final discussion. The study sites were allowed to proceed to phase fouronly if the following performancetargets were met: (1) 95% accordance between validation results of microscopy center and supervisory site; (2) quality of Auramine O stains acceptable in 100% of slides examined; and (3) fewer than two false results in a proficiency testing panel of 10 pre-defined slides.
Phase 4: implementation phase: the procedures were the same as during the validation phase. The duration of this phasewas three months, and patient management was now based on iLEDresults. Supervision and rechecking by the NRL study supervisor were carried outusing Lot Quality Assurance Sampling (LQAS). The sample size was calculated by NTP/NRL based on the positivity and number of negative smears, but the frequency of rechecking was decreased from daily to once every two weeks. Rereading by NRL was doneusingconventional FM at 400x magnification. Discordant slides, if any, were sent to the SNRLin Germany for umpire reading. Rechecked results were provided to study sites. Phase 5: continuation and expansion: The continuation phase lastedsix months, and patient management was based on iLEDresults. Supervision and rechecking by NRL supervisors was carried out according to national Rwandan External Quality Assurance guidelines. Fifteen slides were collected quarterly and rechecked by NRL using conventional FM at 400x magnification. Discordant slides, if any, were sent to the SNRL in Germany for umpire reading. Rechecked results were provided to study sites. After the six-monthcontinuation phase, and following the availability of the compiled results of the previous phase, the demonstration project coordinator allowed all sites to use the iLED method routinely under program conditions. Data entry and analysis: all data and results were recorded in phase-specific forms andsent to NRL and NTP. An electronic database was completedon-site. Positivity agreement between methods at the study laboratories (DH or CHUK) and the National Reference Laboratory was assessed using Cohen's Kappa, which corrects for agreement by chance.Strength of agreement was evaluated using guidelines from Land is and Koch : <0 = poor; 0-0.20 = slight; 0.21-0.40 = fair; 0.41-0.60 = moderate; 0.61-0.80 = substantial; 0.81-1.00 = almost perfect.
Baseline: at the DH in Nyamata, allincoming sputum samples (100) from 37 patients, using the Spot-Morning-Spot criteria, were examined using ZN staining during the baseline phase. The positivity rate of the slides was 11% (11/100 - Table 3) with only one low false positive (LFP) as determined following rechecking. The LFP result had no negative public health consequence on the accurate diagnosis of the patient. There were no poorly stained slides reported by the NRL. At CHUK in Kigali, all incoming sputum samples (205) from 94 patients (Spot-Morning-Spot)were analyzed. The positivity rate for the samples was 7.3% (15/205 - Table 4) and two quantification errors (QEs) were reported following rechecking at NRL. The two QEs, corresponding to two samples from two different patients, had no negative public health consequence. There were no poorly stained slides reported by the NRL for CHUK during the rechecking of the baseline. For the comparison between results obtained by the DH in Nyamata and the NRL, Kappa was 0.947 [95% CI: 0.84, 1.00], which indicates an almost perfect agreement between the two laboratories(see results in Table 5). For the comparison between CHUK and NRL, Kappa was 1.000 due to full agreement (Table 5).
Validation phase: in Nyamata, all incoming samples (100) from 39 patients were screened. The positivity rate decreased to 4% (4/100 - see Table 3) and there were 1 low false positive and 1 low false negative (LFN) errorsreported. Additionally, 15 samples were reported as having poor stains. The LFN and LFP errors, corresponding to two samples from the same patient, would not have a negative public health consequence and were probably due to administrative errors. However, as per protocol, diagnosis was made solely on the basis of the rechecking by the NRL. Kappa was 0.740 [95% CI: 0.38, 1.00], which indicates substantial agreement between the laboratories (Table 5). In CHUK, all incoming samples (202) from 87 patients were screened. The positivity rate significantly increased from baseline to 22.3% (45/202 - see Table 4)and there was one LFN reported.Since only one sample was collected for that particular patient, the LFN error would have had a negative public health consequence on the accurate diagnosis of the patient.The patient was adequately treated following rechecking by NRL. Nine samples were reported as having poor stains by the NRL. Kappa was 0.986 [95% CI: 0.96,1.00'> 1.00], which indicates an almost perfect agreement between laboratories (Table 5). No errors were detected at the proficiency panel test. Both sites were allowed to proceed to the implementation phase.
Implementation phase: in Nyamata, following LQAS, 44 samples (corresponding to 44 patients) were rechecked by the NRL study supervisor. The positivity rate for samples increased from validation to 9.1% (4/44) but remained lower than in the baseline phase(Table 3). No reading/diagnosis errors were reported by the NRL. However, there were six poorly stained samples.Kappa was 1.000 due to full agreement (Table 5). In CHUK, following LQAS, 45 samples from 45 patients were rechecked. The positivity rate for samples increased further from baseline and validation to 31.1% (14/45), as shown in Table 4. Only one LFN and three poor stains were reported by the NRL. It is not possible to say whether the LFN error had a negative public health consequence, since the other two samples from this particular patient were not rechecked and could have been positive.Kappa was 0.949 [95% CI: 0.85; 1.00], which indicates an almost perfect agreement between laboratories (Table 5).
Continuation phase: in Nyamata, 16 samples were read from January through March byiLED at the site and conventional FM at NRL, and 100% concordance was observed. From April through June, an additional 11 samples were rechecked and all results concurred. At CHUK, there were 15 samples screened from January through March and 100% concordance was observed.Sixteen additional samples were screened from April through June and 100% concordance was also observed. Technicians' appraisal: the appraisaltook place after training of the technicians. Installation and first use: all technicians felt that installation of the iLED was easy and that the manual was comprehensive and easy to read and understand. Training: the technicians participating in the iLED project felt that for technicians already trained in ZN microscopy (such as themselves), an iLED training of 1-5 days was suitable. However, for technicians not familiar with ZN microscopy, an iLED training of 3-20 days was suitable. Technicians also felt that NTPs can readily use the current manual developed by the manufacturer for implementation of LED microscopy without major changes.
Optics and handling: All technicians were satisfied with the contrast, color, intensity, and signal-to-noise ratio of the iLED. All technicians were very satisfied with the resolution and depth of focus of the iLED. All technicians also felt that the field of view of the iLED is more homogenously illuminated compared to the standard view. All of them were very satisfied with the overall handling features of the microscope. All technicians also felt that it was convenient or very convenient to switch between bright field and fluorescence. Only one technician surveyed felt that the toggle field was not robust. All technicians felt that no darkroom was needed when usingiLED, a really convenient feature of iLED compared to regular FM. All technicians also agreed that the dazzling protection for the eyepieces was useful. None of the technicians surveyed reported any technical problems with the microscope overall.
Compared to classical FM with mercury vapor lamp, LED FM ismore user-friendlyand benefits from a high acceptability by technicians. Microscopes do not require warm-up and cool-down time, a considerable advantage when power supply is erratic, and the LED light source is considered safer than the mercury vapor lamp. LED systems developed for AFB smears consist eitherof modules that can be fitted to a conventional microscope, or a complete microscope with built-in LED as the light source, such as the iLED.While Partech (Münster, Germany) and Cytoscience (Fontaines, Switzerland) have both developed complete LED FM microscopes, these microscopes are less appropriate for TB than iLED since they are monocular. Using a camera and monitor might be an appropriate solution, but notin less-developed countries. iLED, a binocular FM microscope with built-in LED lamp for epi-fluorescence has produced very good results in reference laboratories . So far, very few reports on these systems exist. These rare reports, however, show excellent performance compared to ZN microscopy [15, 16]. Our study is the first direct evaluation of iLED in Rwanda. LED add-on kits have been designed for different common types of bright field microscopes. The complete installation is not difficult, but it requires slightly more time and care, which could be a disadvantage from the end-user perspective. As difficulty in acceptance by inexperienced microscopists seems to be the main obstacle to the use of FM outside referral laboratories, this may prove to be a major advantage of transmitted LED light FM, as reported earlier from Tanzania . It also remains to be seen whether complete binocular LED microscopes using epi-fluorescence, rather than transmitted light, will meet the same acceptance level with the progressive decentralization of FM to peripheral hospitals and health centers. Our study shows that the acceptability amongst the staff using iLED was extremely high and proficiency in adequate usage was rapid. Compared to traditional bright-field methods, LED fluorescence methods using Auramine O staining allows up to four times faster screening. The detection rate is also estimated to be at least 10% higher. While we did not directly compare the positivity rates between ZN and iLED, we monitored the positivity rate over time during our study.
In Nyamata, the positivity rate surprisingly decreased during the validation phase (Table 5). This result compares with previously established data from the National TB Programme, which has shown that in the last quarter of 2009 and first quarter of 2010, during which our study took place, there was an overall decrease in positivity rate that is not related to the performance of the microscope (or the technicians) but is rather attributable to a significant increase in total number of patients and samples screened. Indeed, the positivity rate dropped to 7.7% during our study, compared to 9.2% the year before. This increase in the total number of patients screened may be the result of the impact of Community Health Workers (CHWs) in Bugesera District (where Nyamata DH is located), who have been involved in active case finding, therefore casting a wider net for overall screening of TB suspects and decreasing the positivity rate at the Nyamata Center. One of the expected issues associated with the change from ZN to Auramine staining for the purpose of our study was a difficulty in preparing and then subsequently reading the slides adequately using the Auramine protocol. Indeed, a few of the smears in the various phases were reported as having poor stains. We believe these could be explained by the fact that some laboratory technicians were not completely proficient in staining the slides appropriately, as can also sometimes be the case for ZN. However, this was not an issue throughout the study as the number of slides poorly stained gradually decreased at both sites. Agreement between the iLED and rechecking at the Reference Laboratory were deemed “almost perfect” (kappa = 0.81-1.00) across three of four site-phase combinations. The exception was Nyamata District Hospital during the validation phase, which was deemed “substantial” agreement (kappa = 0.61-0.80). However, the 100% concordance at both demonstration sites during the continuation phase shows technicians' rapid command of the new iLED microscope in a relatively short time. Technicians can therefore be easily trained to switch from ZN microscopy to LED FM with a high success rate. This should be of interest to national TB control programs that are interested in improving their overall case detection rate but cannot yet invest in the newer, molecular-based technologies currently being rolled out.
The authors declare no competing interests.
AUN, MG, and YBA conceived the study and participated in its design and coordination. MT and GV participated in the design and coordination of the study. BN performed all statistical analysis. All authors helped to draft the manuscript and read and approved the final manuscript.
We would like to thank CN Paramasivan, FabriceIngabire, Sebazungu Emmanuel, Alida Ngwije, Eliane Kamanzi, Jean Baptiste Gatabazi, and Ruben Sahabofor their help with our study. Finally, we would like to thank Timmons Barbara and Catherine Mundy for the revision of the manuscript.
Table 1: semi-quantitative scale used for reading with iLED
Table 2: semi-quantitative scale for rechecking with conventional FM
Table 3: distribution of slides by outcome at different phases of the study evaluation at Nyamata Hospital
Table 4: distribution of slides by outcome at different phases of the study evaluation at CHUK
Table 5: statistical analysis of diagnostic outcomes by site at Nyamata District Hospital (DH) and at the Centre Hospitalier Universitaire de Kigali (CHUK) by either LM or iLED
|The Pan African Medical Journal articles are archived on Pubmed Central. Access PAMJ archives on PMC here|
Volume 26 (Jan - Apr 2017)
This article authors
|On Google Scholar|
Navigate this article
Tables and figures
|Table 1: semi-quantitative scale used for reading with iLED|
|Table 2: semi-quantitative scale for rechecking with conventional FM|
|Table 3: distribution of slides by outcome at different phases of the study evaluation at Nyamata Hospital|
|Table 4: distribution of slides by outcome at different phases of the study evaluation at CHUK|
|Table 5: statistical analysis of diagnostic outcomes by site at Nyamata District Hospital (DH) and at the Centre Hospitalier Universitaire de Kigali (CHUK) by either LM or iLED|
Rate this article
Popular articles in Research
|1||Epilepsy and Education in developing countries: a survey of school teachers’ knowledge about epilepsy and their attitude towards students with epilepsy in Northwestern Nigeria|
|2||Profil épidémiologique et clinique de la tuberculose dans la zone de santé de Lubumbashi (RD Congo)|
|3||Impact de la varicocèle sur le volume testiculaire et les paramètres spermatiques|
|4||Assessment of renal function and electrolytes in patients with thyroid dysfunction in Addis Ababa, Ethiopia: a cross sectional study|
|5||Implementation challenges of free maternity services policy in Kenya: the health workers’ perspective|
|6||Health care utilization among rural women of child-bearing age: a nigerian experience|
|7||Pan African Medical Journal (Pamj) and African Field Epidemiology Network (AFENET): A partnership for the future of medical publishing in Africa|