|
|
| |||||||||||||||||||
• Introduction • Research methods and design • Results and discussion • Literature search results • Global programme results • Quality System Essentials meta-analysis • Official WHO AFRO SLIPTA audits and accreditation • Service delivery indicators • Cost • Limitations to the study • Conclusion • Acknowledgements • Competing interests • Authors’ contributions • CDC disclaimer • References |
|||||||||||||||||||
Objective: We completed a systematic review of the published literature on SLMTA. The review consists of two companion papers; this article examines quantitative evidence presented in the publications along with a meta-analysis of selected results. Methods: We identified 28 published articles with data from SLMTA implementation. The SLMTA programme was evaluated through audits based on a standard checklist, which is divided into 12 sections corresponding to the 12 Quality System Essentials (QSEs). Several basic service delivery indicators reported by programmes were also examined. Results for various components of the programme were reviewed and summarised; a meta-analysis of QSE results grouped by the three stages of the quality cycle was conducted for 126 laboratories in 12 countries. Results: Global programme data show improved quality in SLMTA laboratories in every country, with average improvements on audit scores of 25 percentage points. Meta-analysis identified Improvement Management as the weakest stage, with internal audit (8%) and occurrence management (16%) showing the lowest scores. Studies documented 19% – 95% reductions in turn-around times, 69% – 93% reductions in specimen rejection rates, 76% – 81% increases in clinician satisfaction rates, 67% – 85% improvements in external quality assessment results, 50% – 66% decreases in nonconformities and 67% increases in staff punctuality. Conclusions: The wide array of results reported provides a comprehensive picture of the SLMTA programme overall, suggesting a substantive impact on provision of quality laboratory services and patient care. These comprehensive results establish a solid data-driven foundation for program improvement and further expansion.
Since its introduction in 2009, the SLMTA programme has been implemented widely throughout Africa, as well as in the Caribbean, Central and South America, and Southeast Asia.2 The primary focus of the programme thus far has been implementation and expansion; until recently, little attention has been paid to the systematic examination of programme results in order to guide programme improvement and decision making. This systematic literature review aims to compile existing results from evaluations of the SLMTA programme into a comprehensive report, in order to provide a broad view of the programme and to identify directions for the future. Because of the large volume of information collected, the review has been published in two parts. In Part 1, published separately, we present content analysis of qualitative findings and identified strategic directions for future priorities.3 In this companion paper, we compile the quantitative data presented in the publications, examine scores and indicators, and conduct a meta-analysis of selected results in order to establish a solid, data-driven foundation for programme improvement and to help guide future implementation.
The standard SLMTA implementation model includes three workshops, each of which is followed by a period of several months for laboratories to implement improvement projects, usually with onsite support and mentorship.2 Laboratories implementing the SLMTA programme are evaluated through audits based on the Stepwise Laboratory Quality Improvement Process Towards Accreditation (SLIPTA) checklist.4 Audit scores are categorised into star ratings, with zero stars corresponding to a score of 0% – 54%, one star 55% – 64%, two stars 65% – 74%, three stars 75% – 84%, four stars 85% – 94%, and five stars 95% – 100%. The checklist items are divided into 12 sections that represent the 12 Quality System Essentials (QSEs) as defined by the Clinical and Laboratory Standards Institute (CLSI).5 These QSEs can be grouped by stages of the quality cycle: Resource Management (equipment; facilities and safety; organisation and personnel; purchasing and inventory), Process Management (client management; documents and records; information management; process control and internal/external quality assessment) and Improvement Management (corrective action; internal audit; management reviews; occurrence management).6 To assess progress, baseline and exit audits are conducted before and after SLMTA implementation, respectively, using the SLIPTA checklist. ‘Surveillance’ audits are also often conducted after the exit audit in order to monitor continued improvement and assess sustainability. Several studies provided scores by individual QSEs. We combined these data and conducted a meta-analysis in Microsoft® Excel 2013 so as to determine common areas of strength, weakness and improvement. For studies reporting only median or mean QSE data for multiple laboratories, laboratory-level data were solicited from authors to further enhance the analysis. All cost estimates reported in local currency in published articles were converted into US dollars, based on the official exchange rate as of August 1, 2014. Percent changes in indicator results were calculated from published results if not reported directly in the papers.
Global programme results Data from all laboratories implementing the SLMTA programme were collated and summarised in a single paper describing the global results of the programme to date.33 In total, 617 laboratories in 47 countries on four continents have implemented SLMTA in 65 training cohorts, with nearly 2000 laboratory staff trained in the programme. Most of the laboratories were at the district (38%), regional (27%) or national (18%) levels. The authors report that the starting level of laboratory quality in developing countries was very low, with 84% of SLMTA laboratories scoring below the one-star level at baseline. The 302 laboratories that had completed the programme had an average improvement of 25 percentage points; 70% achieved at least one star at exit audit and 22% of laboratories increased three or more star levels. Estimates of the number of laboratory tests conducted by SLMTA laboratories suggested that the 617 laboratories enrolled in SLMTA conduct more than 100 million tests annually and that whilst only 16% of these tests were conducted by laboratories with at least one quality star before SLMTA, 68% were done by laboratories with at least one star after SLMTA implementation. That translates to approximately 58 million tests conducted by laboratories with little to no QMS prior to SLMTA which now have at least a basic quality system in place.33 Quality System Essentials meta-analysis Examining individual SLIPTA checklist scores for each of the 12 QSEs enables laboratories to pinpoint strengths, weaknesses and areas of improvement. QSE data have not been compiled systematically on a global scale. From the published papers, QSE data were presented for 126 laboratories in 12 countries.8,11,12,14,15,18,20,21,22,25,26 Individual studies reported substantial variability in high- and low-scoring QSEs. For example, some laboratories scored 0% for five of the 12 QSEs at exit audit, whereas others scored 100% for the same five QSEs.At baseline, the weakest areas overall were in the Improvement Management stage of the quality cycle, including internal audit (5%), occurrence management (16%), corrective action (25%) and management reviews (29%) (Figure 1). At an average of 20%, this stage scored less than half of the other two stages, namely, Resource Management (42%) and Process Management (40%). None of the 12 QSEs had mean baseline scores above 55%; the highest scores were in information management (51%), facilities and safety (47%), purchasing and inventory (42%) and process control and internal/external quality assessment (41%). At the exit audit, the four Improvement Management QSEs still showed the lowest scores, ranging from 32% – 50% (average 42%) (Figure 1). The Resource Management and Process Management stages had higher scores ranging from 58% – 74% (average 65% for Resource Management and 63% for Process Management). The greatest improvements were in documents and records (34 percentage points), client management (29 percentage points), and facilities and safety (27 percentage points). Each of the three stages had the same average improvement of 23 percentage points.
Based on results from five laboratories, Maina et al. found that the laboratories with the greatest overall score increases had focused on internal audit and corrective action; they then hypothesised that an improvement in these areas may be a catalyst for overall improvement in other areas.14 Meta-analysis results suggest that the corrective action QSE may be the most predictive of overall improvement; laboratories in the top quartile of overall improvement outperformed those in the bottom quartile by 62 percentage points for the corrective action QSE, compared to a median of 40 percentage points for the other QSEs. CLSI defines corrective action as an ‘action to eliminate the (root) cause of a detected nonconformity or other undesirable situation’.34 In the SLIPTA checklist, corrective action is assessed through four questions about how the laboratory deals with occurrence reports, nonconformities and discordant results.4 The International Organization for Standardization (ISO) confirms the importance of corrective action, saying that ‘the corrective and preventive actions system is the most critical element for an efficient quality system’.35 Additional work is needed to verify priority areas of improvement, as well as to delineate the set of essential improvement projects that will result in meaningful laboratory quality improvement. Official WHO AFRO SLIPTA audits and accreditation A July 2009 survey of accrediting body registers identified 340 accredited laboratories in sub-Saharan Africa; only 28 (8%) of these laboratories were located outside of South Africa and nearly all were private, parastatal or donor-supported research facilities.36 By early 2013, little progress had been made, with 380 laboratories accredited in the region; only 35 (9%) laboratories outside of South Africa were accredited and three quarters of the 49 countries in the region had no accredited laboratories.37 However, the impact of SLMTA is beginning to show; as of September 2014, six laboratories enrolled in SLMTA in Kenya, the Bahamas, Vietnam and Zimbabwe have been accredited, at a median of 31.5 months after starting the SLMTA programme.10,11,33 Several laboratories have been recommended for accreditation or are in the process of application.11,18,20,28 Ninety-seven SLMTA laboratories have received official WHO AFRO SLIPTA audits conducted by representatives from the African Society for Laboratory Medicine,33 including 11 laboratories in published reports included in this review.7,18,25,26,29 Service delivery indicators In addition to audit scores, many of the studies reported improvements for indicators reflecting testing and customer and clinician satisfaction (Table 2). Three studies reported reductions in turnaround time for testing,10,20,28 with times decreasing by 19% – 95%. Patient and clinician satisfaction were commonly measured using surveys. Four studies showed relative improvements in patient satisfaction ranging from 30% to > 100%,9,10,25,28 although in one laboratory complaints from patients increased, possibly as a result of staff attrition.17 Two studies reporting on clinician satisfaction found improvements of approximately 80%.17,28Indicators for laboratory management and overall functioning also showed improvements (Table 2). One laboratory reported a 65% decrease in corrective actions,10 five laboratories in the Caribbean Region reported decreases in nonconformities of 50% – 66%11 and two laboratories showed improvements in external quality assessment results of 67% – 85%.10,17 In a Kenyan laboratory, staff punctuality increased 67% and the need for equipment repairs decreased 63%.17 A Botswana laboratory successfully reduced losses resulting from expired reagents from $18 000 in 2010 to $40 in 2013;28 and three studies showed reductions in specimen rejection rates of 69% – 93%.10,17,25 When SLMTA was adapted and implemented at a hospital in Cameroon, patient wait times decreased 67% – 83%, infection rates and stillborn rates decreased (83% and 80%, respectively) and the number of patients and hospital revenue increased.9
Cost The reported costs per laboratory of implementing various components of SLMTA have varied widely (Table 3). Much of this variability is because of differences in what was included in the cost estimates, as well as location-specific factors, such the price of fuel, salary levels and distances to participating laboratories. The estimated cost of conducting the three-workshop SLMTA series has ranged from $1482 per laboratory in Zimbabwe using local facilitators in a central location31 to $21 480 in Cameroon using decentralised training.22 Mentorship cost per laboratory has ranged from $5689 in Zimbabwe30 to $24 000 in Ghana.25 The cost of implementing improvement projects has ranged from $10 000 in Ghana25 to $36 500 in a Kenyan laboratory seeking accreditation.10
Three studies have compared the cost of various SLMTA implementation models. One study of 19 laboratories in Zimbabwe found that mentorship and supervision costs for four different models were similar ($5689-$9601 per laboratory), recommending that ‘countries should carefully consider which mentorship model or models would be best suited to their individual situation’.30 Another study in Zimbabwe found that implementing SLMTA using local (in-country) facilitators is more expensive than external facilitators for the first SLMTA cohort because of the costs associated with conducting an in-country training-of-trainers; however, over the course of national scale-up in 120 laboratories, use of local facilitators would save the country nearly 50% ($580 000 vs. $322 000).31 A Cameroonian study found that the cost per laboratory of centralised training was approximately the same as decentralised training ($21 122 vs. $21 480, respectively); centralised training required less trainer time, whilst decentralised training allowed more staff to participate.22 No published studies to date have reported a thorough examination of the cost of implementing the entire SLMTA programme, including each of the major components (training of mentors, trainers and auditors; conducting SLMTA workshops; mentorship, supervisory visits and implementation of improvement projects; and conducting audits). In addition, a more extensive cost-benefit analysis taking into consideration the value of laboratorians’ time (i.e., opportunity cost) to participate in the programme and implement changes in the laboratory along with tangible and intangible benefits of the programme is needed.31 Limitations to the study This review is subject to several limitations. Firstly, whilst 28 studies on SLMTA were identified and summarised, these reflect only 18 (38%) of the 47 countries and 211 (34%) of the 617 laboratories that have implemented the programme. Their results may not be representative of the programme as a whole, or a comprehensive account of all laboratories’ experiences. Secondly, whilst audit results were available for all laboratories because of the use of the SLIPTA checklist, the other indicators presented here were available in few of the published studies; in addition, methodologies varied between the studies, limiting the ability to combine and compare results directly.Authors of the studies published thus far also point out several limitations. Firstly, the SLMTA programme as a whole is too young to allow an assessment of the long-term sustainability of results.14,33 Secondly, all of the published studies were observational; several studies examining the effect of mentorship or training methodologies note that laboratories were not assigned randomly, but were rather selected purposively based on convenience or other programmatic considerations. Thus there may have been other factors that could account for some of the differences.8,15,20,30 Similarly, none of the studies included control laboratories upon which to base a comparison.22 Thirdly, there is a lack of consistency in the qualifications of auditors; whilst the SLIPTA checklist is designed to help standardise the audit process, some variability between auditors may remain.8,29 Finally, several authors noted that their published studies are based on a small number of laboratories14,15,20,30 and some indicators were either not measured systematically9 or not measured at baseline.9,28 Conclusion In their summary of global-level findings, Yao et al. point out that ‘few [other] management and leadership development programmes have been implemented on a such a large scale with results-oriented outcome measures’.33 The wide array of results reported provides a comprehensive picture of the SLMTA programme overall, suggesting a substantive impact on provision of quality laboratory services and patient care. The full potential of the programme can be realised only if the lessons learned lead to informed action among laboratory workers, healthcare providers and policy makers toward the ultimate goal of providing quality patient care.
This research has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the CDC. Competing interests The authors declare that they have no financial or personal relationship(s) that may have inappropriately influenced them in writing this article. Authors’ contributions E.T.L (CDC, Atlanta) analysed the data and wrote the manuscript. K.Y. (CDC, Atlanta) and J.N.N (CDC, Atlanta) provided substantial input to the revision of the manuscript. CDC disclaimer The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC.
2.Yao K, Maruta T, Luman ET, Nkengasong JN. The SLMTA programme: Transforming the laboratory landscape in developing countries. Afr J Lab Med. 2014;3(2), Art. #194, 8 pages. http://dx.doi.org/10.4102/ajlm.v3i2.194 3.Luman ET, Yao K, Nkengasong JN. A comprehensive review of the SLMTA literature part 1: Content analysis and future priorities. Afr J Lab Med. 2014;3(2), Art. #265, 11 pages. http://dx.doi.org/10.4102/ajlm.v3i2.265 4.World Health Organization Regional Office for Africa. WHO Guide for the Stepwise Laboratory Improvement Process Towards Accreditation in the African Region (with checklist) [document on the Internet]. c2011 [cited 2014 Jul]. Available from: http://www.afro.who.int/ en/clusters-a-programmes/hss/blood-safety-laboratories-a-health- technology/blt-highlights/3859-who-guide-for-the-stepwise-laboratory- improvement-process-towards-accreditation-in-the-african- region-with-checklist.html 5.Clinical and Laboratory Standards Institute. Application of a quality management system model for laboratory services; Approved Guidelines – Third Edition. CLSI document GP26-A3 [ISBN 1-56238-553-4]. Wayne, PA: Clinical and Laboratory Standards Institute; 2004. 6.Datema T, Oskam L, van Beers SM, et al. Critical review of the Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA): Suggestions for harmonization, implementation and improvement. Trop Med Int Health. 2012;17(3):361–367. 7.Andiric LR, Massambu CG. One laboratory’s progress toward accreditation in Tanzania. Afr J Lab Med. 2014;3(2), Art. #202, 4 pages. http://dx.doi.org/10.4102/ajlm.v3i2.202 8.Audu RA, Onubogu CC, Nwokoye NN, et al. Improving quality in national reference laboratories: The role of SLMTA and mentorship. Afr J Lab Med. 2014;3(2), Art. #200, 7 pages. http://dx.doi.org/10.4102/ajlm.v3i2.200 9.Eno LT, Asong T, Ngale E, et al. Driving hospital transformation with SLMTA in a regional hospital in Cameroon. Afr J Lab Med. 2014;3(2), Art. #221, 5 pages. http://dx.doi.org/10.4102/ajlm.v3i2.221 10.Gachuki T, Sewe R, Mwangi J, et al. Attaining ISO 15189 accreditation through SLMTA: A journey by Kenya’s National HIV Reference Laboratory. Afr J Lab Med. 2014;3(2), Art. #216, 9 pages. http://dx.doi.org/10.4102/ajlm.v3i2.216 11.Guevara G, Gordon F, Irving Y, et al. The impact of SLMTA in improving laboratory quality systems in the Caribbean Region. Afr J Lab Med. 2014;3(2), Art. #199, 9 pages. http://dx.doi.org/10.4102/ajlm.v3i2.199 12.Hiwotu TM, Ayana G, Mulugeta A, et al. Laboratory system strengthening and quality improvement in Ethiopia. Afr J Lab Med. 2014;3(2), Art. #228, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.228 13.Lulie AD, Hiwotu TM, Mulugeta A, et al. Perceptions and attitudes toward SLMTA amongst laboratory and hospital professionals in Ethiopia. Afr J Lab Med. 2014;3(2), Art. #233, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.233 14.Maina RN, Mengo DM, Mohamud AD, et al. Progressing beyond SLMTA: Are internal audits and corrective action the key drivers of quality improvement? Afr J Lab Med. 2014;3(2), Art. #222, 7 pages. http://dx.doi.org/10.4102/ajlm.v3i2.222 15.Makokha EP, Mwalili S, Basiye FL, et al. Using standard and institutional mentorship models to implement SLMTA in Kenya. Afr J Lab Med. 2014;3(2), Art. #220, 8 pages. http://dx.doi.org/10.4102/ajlm.v3i2.220 16.Maruta T, Yao K, Ndlovu N, Moyo S. Training-of-trainers: A strategy to build country capacity for SLMTA expansion and sustainability. Afr J Lab Med. 2014;3(2), Art. #196, 7 pages. http://dx.doi.org/10.4102/ajlm.v3i2.196 17.Maruti PM, Mulianga EA, Wambani LN, et al. Creating a sustainable culture of quality through the SLMTA programme in a district hospital laboratory in Kenya. Afr J Lab Med. 2014;3(2), Art. #201, 5 pages. http://dx.doi.org/10.4102/ajlm.v3i2.201 18.Masamha J, Skaggs B, Pinto I, et al. Working toward a sustainable laboratory quality improvement programme through country ownership: Mozambique’s SLMTA story. Afr J Lab Med. 2014;3(2), Art. #253, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.253 19.Mataranyika MN, Beukes LK. View from the top: Involvement of Namibia’s health ministry in laboratory quality improvement. Afr J Lab Med. 2014;3(2), Art. #195, 2 pages. http://dx.doi.org/10.4102/ajlm.v3i2.195 20.Mokobela KO, Moatshe MT, Modukanele M. Accelerating the spread of laboratory quality improvement efforts in Botswana. Afr J Lab Med. 2014;3(2), Art. #207, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.207 21.Mothabeng D, Maruta T, Lebina M, et al. Strengthening laboratory management towards accreditation: the Lesotho experience. Afr J Lab Med. 2012;1(1), Art. #9, 7 pages. http://dx. doi: 10.4102/ajlm.v1i1.9 22.Ndasi J, Dimite L, Mbome V, et al. Decentralised facility-based training as an alternative model for SLMTA implementation: The Cameroon experience. Afr J Lab Med. 2014;3(2), Art. #231, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.231 23.Nguyen TT, McKinney B, Pierson A, et al. SLIPTA e-Tool improves laboratory audit process in Vietnam and Cambodia. Afr J Lab Med. 2014;3(2), Art. #219, 5 pages. http://dx.doi.org/10.4102/ajlm.v3i2.219 24.Nkengasong JN, Birx D. Quality matters in strengthening global laboratory medicine. Afr J Lab Med. 2014;3(2), Art. #239, 4 pages. http://dx.doi.org/10.4102/ajlm.v3i2.239 25.Nkrumah B, Van der Puije B, Bekoe V, et al. Building local human resources to implement SLMTA with limited donor funding: The Ghana experience. Afr J Lab Med. 2014;3(2), Art. #214, 7 pages. http://dx.doi.org/10.4102/ajlm.v3i2.214 26.Nkwawir SC, Batumani NN, Maruta T, Awasom CN. From grass to grace: How SLMTA revolutionised the Bamenda Regional Hospital Laboratory in Cameroon. Afr J Lab Med. 2014;3(2), Art. #203, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.203 27.Noble MA, Martin R, Ndihokubwayo J-B. Making great strides in medical laboratory quality. Afr J Lab Med. 2014;3(2), Art. #256, 2 pages. http://dx.doi.org/10.4102/ajlm.v3i2.256 28.Ntshambiwa K, Ntabe-Jagwer W, Kefilwe C, et al. Translating a National Laboratory Strategic Plan into action through SLMTA in a district hospital laboratory in Botswana. Afr J Lab Med. 2014;3(2), Art. #209, 5 pages. http://dx.doi.org/10.4102/ajlm.v3i2.209 29.Nzabahimana I, Sebasirimu S, Gatabazi JB, et al. Innovative strategies for a successful SLMTA country programme: The Rwanda story. Afr J Lab Med. 2014;3(2), Art. #217, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.217 30.Nzombe P, Luman ET, Shumba E, et al. Maximising mentorship: Variations in laboratory mentorship models implemented in Zimbabwe. Afr J Lab Med. 2014;3(2), Art. #241, 8 pages. http://dx.doi.org/10.4102/ajlm.v3i2.241 31.Shumba E, Nzombe P, Mbinda A, et al. Weighing the costs: Implementing the SLMTA programme in Zimbabwe using internal versus external facilitators. Afr J Lab Med. 2014;3(2), Art. #248, 6 pages. http://dx.doi.org/10.4102/ajlm.v3i2.248 32.Yao K, McKinney B, Murphy A, et al. Improving quality management systems of laboratories in developing countries: An innovative training approach to accelerate laboratory accreditation. Am J Clin Pathol. 2010;134(3):401–409. 33.Yao K, Luman ET, SLMTA Collaborating Authors. Evidence from 617 laboratories in 47 countries for SLMTA-driven improvement in quality management systems. Afr J Lab Med. 2014;3(2), Art. #262, 11 pages. http://dx.doi.org/10.4102/ajlm.v3i2.262 34.Clinical Laboratory Standards Institute. Management of nonconforming laboratory events; approved guideline. CLSI document GP32-A. Wayne, PA: Clinical Laboratory Standards Institute; 2007. 35.International Organization for Standardization. ISO 9000 resources: corrective and preventive actions high level review [page on the Internet]. c2014 [cited 2014 Sep 18]. Available from: http://www.iso9000resources.com/ba/corrective-preventive-actions.cfm 36.Gershy-Damet G, Rotz P, Cross D, et al. The World Health Organization African region laboratory accreditation process: Improving the quality of laboratory systems in the African region. Am J Clin Pathol. 2010;134(3):393–400. 37.Schroeder LF, Amukele T. Medical laboratories in Sub-Saharan Africa that meet international quality standards. Am J Clin Pathol. 2014;141(6):791–795. |
