Study setting

LDHF training for PAC was implemented in 16 health facilities (Health Center IIIs, IVs, and Hospitals) in the semi-urban districts of Wakiso and Mukono in the Central Region. Traditional PAC training included trainees from 18 health facilities in another Central Region district, Mpigi. In Uganda, illegal and often unsafe abortions contribute both to the need and complexity of PAC delivery [12]. As these PAC complications contribute significantly to maternal deaths, training has been recommended for high and midlevel health workers [13,14]. PAC is restricted further by absenteeism and task-shifting, which often leaves midwives stretched to provide services [15].

LDHF training for Pediatric HIV/AIDS management was implemented in 8 health facilities across four districts: Hoima and Kibaale in Western Uganda and Kyankwanzi and Kiboga in Central Uganda. Traditional training on pediatric HIV/AIDS was delivered for a separate group of providers representing 7 other facilities within the districts. Uganda’s high HIV general population rate at 7.4% in 2012 has been the focus of significant response resources, and the most recent National Strategic Plan outlines additional steps for prevention of mother to child transmission, provision of antiretroviral therapy (ART), strategies for early detection, treatment initiation, and management of ART especially for child cases of HIV [16-19]. Failures in early diagnosis and care linkages led to 72% of children presenting, in a 2012 study, with advanced disease at the initial visit to an HIV clinic and only 41% of eligible children receiving ART in 2013 [16,20]. The Ministry of Health has specified efforts to combat Pediatric HIV with communication strategies, community mobilization, and training for health care workers [21].

Program description

The PAC program began in May 2015 with a two-month development phase. Start-up activities lasted one-month, which included District Health Team meetings, facility visits to select training sites, training of trainers (TOT) for selected district trainers, orientation and PAC content introduction for trainees, and baseline assessment data collection. Implementation spanned four months, during which the 12 district trainers led LDHF capacity-building for the 20 trainees – one-hour continuing medical education (CME) and one-hour hands-on practice – twice monthly until 6 training modules had been successfully completed. Final evaluation of trainees included a structured written knowledge assessment proctored by AOGU staff as well as an Observed Structured Clinical Exam (OSCEs) with five simulated case stations which tested providers’ skills.

Pediatric HIV/AIDS care LDHF training program began March 2015 with a two-month development phase including orientation meetings with the District Health Teams and facilities, baseline assessment, and LDHF curriculum development. Start-up activities included a two-week TOT for district mentors. The implementation phase included biweekly district mentor led LDHF sessions - CME trainings for 1 hour for 4 months (8 visits) with 2 months accounting for planning, delays, and follow up -open to all staff but targeted to 54 mentees. An additional 5 mentoring sessions were scheduled for district mentors to see patients with mentees, practice application of knowledge and provide feedback. The LDHF training targeted all cadres of staff involved in pediatric HIV/AIDS care services. The endline assessment included a written knowledge test and direct observation of skills during patient care visit using a scored checklist. Table 1 defines the differences in the training programs, Table 2 details activities differences of the LDHF training approaches, and Table 3 summarizes which activities occurred between the programs’ different LDHF and traditional training approaches.

Measurement

We prospectively monitored costs to determine full training program cost of the LDHF and traditional workshop approaches for each partner from March to December 2015. Costs were disaggregated into three distinct windows of time: development (two months), start-up (one month) and implementation (six months). Development and start-up activities were considered investments with an estimated useful life of 2 years and were amortized, or reduced and calculated to be used gradually over time, using a 3% discount rate; the analytic time horizon of 6 months reflects the implementation phase. Development and start-up activity costs included initial stakeholder meetings, adapting the curricula for LDHF, procuring medical supplies, holding the district trainer trainings and orientations, and costs of personnel. Implementation phase activities of the program included costs of personnel time, LDHF training sessions at facilities, evaluation of trainees as well as building and support costs. Furniture and equipment, which did not have a local cost data source, were valued by standards of the region set forth by WHO CHOICE [22]. Final costs presented in 2015 US$ for a 9-month analytic time horizon aligned with program implementation.

We additionally sought to assess opportunity costs of provider participation in trainings through a participating provider survey conducted as a quasi-independent study with Jhpiego Uganda and Johns Hopkins School of Public Health. The survey included quantitative questions on demographics and training participation and open-ended questions to elicit qualitative responses about preferences and perceptions of training.

Sampling

PAC training with the LDHF approach was delivered to 20 health providers in 16 facilities while the traditional workshop approach was utilized to train a comparator group of 20 health providers from 18 facilities. Once identified as target health facilities for post-abortion complication referrals, the nurse, physician, or manager ‘in-charge’ of the facility selected one or more health providers to attend the training.

LDHF training in Pediatric HIV/AIDS management was delivered to 54 health professionals in 8 facilities, while 21 providers in 7 separate facilities were trained through a traditional workshop based approach. The implementing organization selected eligible staff with facility leadership recommendation; participants included clinical officers, registered nurses, laboratory technician/assistants, enrolled nurses, midwives, or records assistants that were responsible for clinical care and management of HIV/AIDS care.

The provider survey sampling sought 100% of training participants (139) from all roles of LDHF trainee, traditional trainee, and district mentors by visits of research assistants to health facilities and districts; scheduling issues and provider leave contributed to an 8% loss to follow up. Of the 119 providers surveyed (48 from PAC and 71 from Pediatric HIV), 58 were LDHF trainees, 20 were LDHF district mentors, and 39 were traditional trainees.

Data collection

All training program activities and resources were defined through semi-structured interviews by external costing experts with personnel from each training program. Cost data were then collected by training program personnel using a standardized Excel tool developed externally based on the reported activities outlined through semi-structured interviews with program personnel. Cost data were sourced from financial cost reports, receipts, accounting systems, or other data sources identified by the training partner staff who had participated in program costing methods instruction.

The training participant provider survey was developed, pilot tested, and programmed a tablet-based, 45-minute to 1 hour in-depth interview conducted in English with local language clarifications by Ugandan midwives trained as research assistants in qualitative data. The survey collected key data on provider income, financial incentives of trainings, and time spent on and travel to either training. Open-ended responses were captured in handwritten form by the interviewer and entered digitally into the tablet after the interview concluded to ensure interviewers focused on responents rather than devices.

Analysis

Cost analysis was conducted by costing experts Johns Hopkins Bloomberg School of Public Health in MS Excel (Microsoft Inc, Seattle WA, USA) to summarize and evaluate cost drivers and activity resource allocation for each implementing partner program. Cost analyses included assessing the costs and cost-efficiency of conducting in-service training through LDHF approach and traditional workshop approaches in both PAC and Pediatric HIV training programs. Provider survey data were collected in the Mobile Data Studio platform, transferred to excel, and analyzed in Stata Version 13.1 (Stata Corp, College Station TX, USA). Data from the open-ended provider surveys were analyzed following general qualitative thematic analysis guidelines. Responses were manually coded through an iterative process; themes and sub-themes were identified and responses were re-analyzed in order to code presence of these defined concepts in each response.

Ethics approval

The institutional review board (IRB) at the Johns Hopkins University, Bloomberg School of Public Health in Baltimore, Maryland, USA determined the study protocol to be non-human subjects research. This IRB determination was shared with local partners and each conducted their own ethics review process as well.

Costs of in-service training for PAC

PAC LDHF training had an estimated financial cost of US$52 680, including US$30 761 to develop and start-up program activities and US$21 919 to implement the LDHF training sessions (Table 2). Of total cost for the training program, one-time development and start-up main costs were medical supply procurement (16%), District Trainer TOT and Trainee Orientation (16%), curriculum adaption (8%), personnel (7%), and initial stakeholder meetings (4%). Of the total program costs, implementation costs were driven by LDHF facility based training (11%), personnel costs (8%), supportive supervision (5%), final stakeholder meetings (5%), monitoring and evaluation (4%), endline provider evaluation (4%), and other costs (Table 4). When amortizing development and start-up phase costs over the expected 2-year lifespan of the training, the total cost for the LDHF training was US$29 957 and the cost per person trained was US$936. For each training approach, details on total financial cost are found in Table 4 and details on total annualized costs found in Table 5.

PAC workshop training corresponded to a total financial program cost of US$10 808, including $348 for one-time start-up activities and US$10 460 for implementation. The program had two main activity cost drivers; 78% of traditional training costs were incurred during the workshop (US$8378) and 11% of costs (US$1149) were incurred during a shared final day evaluation (Table 4). Personnel salary cost to support traditional training was low as staff had only a short time devoted to planning. Salaries were US$975 or 9% of the traditional training cost (Table 4). After amortizing the start-up activities over the expected 2-year lifespan of the training, the estimated cost for a 6-month implementation period for traditional training was US$10 551, with a corresponding cost per training recipient of US$527 (Table 5).

Provider incentives and costs of participating for PAC

The cost to participate – travel, transport, and opportunity cost of income that could have been generated in the time spent on the training program – for the PAC LDHF training differed between district trainers and LDHF trainees. District trainers had a total cost to participate of $192 and were given a total facilitation incentive of US$376, including the 6-day training of trainers and 6 LDHF sessions conducted. The LDHF trainees had a total cost to participate of $49 and were given a total facilitation incentive (lunch, travel) of US$66. The facilitation incentives were given by the training organization and are included in the programmatic costs listed above; the cost to participate is drawn from the provider survey data for district trainers and LDHF trainees (Table 6).

The provider cost to participate as a trainee in the PAC workshop training was a total of $38. Workshop trainees were given a total facilitation incentive (lunch, travel) of US$34. The facilitation costs were included in the programmatic costs listed above, whereas the cost to participate indicate the opportunity cost for traditional workshop trainees.

Costs of in-service training for Pediatric HIV

Pediatric HIV/AIDS training through the 9-month LDHF program had a total financial cost of $64 712, including development and start-up activities with an estimated cost of US$31 183 (48% of overall program cost) and the cost of LDHF training session implementation which was US$33 529. The cost driver of one-time start-up activities was the two-week District Mentor TOT at US$15 000 – 23% of the overall program cost. Among recurrent costs, personnel salary and benefits were the leading drivers; accounting for 39% of the total program cost (Table 4). The provision of training itself was conducted through 8 visits by district mentors at a cost of US$5081 (8% of program cost) and 4 supportive supervision visits costing an estimated US$4396 (7% of program cost) (Table 4). When amortizing development and start-up phase costs over the expected 2-year lifespan of the training program, the 6-month implementation of the LDHF training cost an estimated US$41 677 and the cost per training recipient US$631 (Table 5).

Pediatric HIV/AIDS training through the traditional workshop corresponded to a total of $20 220 in financial costs over the course of 1 month, including $2118 used for start-up activities and $18 102 for implementation activities. Over half of the total program budget, US$13 125 (65%), was spent on the conduct of traditional training workshop. Another cost driver during the implementation phase was personnel (11%) and endline provider assessment (9%), which took place at the facility and was conducted by staff (Table 4). After annualizing development and start-up activities, the estimated cost for a 6-month implementation period for traditional training was US$18 656, with a corresponding cost per training recipient of US$888 (Table 5).

Provider incentives and costs of participating in-service training for Pediatric HIV

The cost to participate – travel, transport, and opportunity cost of income that could have been generated in the time spent on the training program – for the Pediatric HIV LDHF training differed between district trainers and LDHF trainees. District trainers had a total cost to participate of US$220 and were given a total facilitation incentive of US$429, including a 10-day training of trainers and average 8 LDHF sessions conducted. The LDHF trainees had a total cost to participate of US$33 and were given a total facilitation incentive (lunch, travel) of US$17. The provider cost to participate as a trainee in the pediatric HIV workshop training was a total of US$36. Workshop trainees were given a total facilitation incentive (lunch, travel) of US$34 (Table 7).

Provider preferences for LDHF training

The responses by training program participants provided insight on provider preferences and perspectives regarding LDHF training and traditional workshop training. Among those who had direct experience with the LDHF approach, 70% reported preferring LDHF as compared to traditional training. Most providers that preferred LDHF felt that it improved learning methods for transferring knowledge and critical skills, provided a forum for including and engaging many staff, and was beneficial for its facility location, allowance of hands-on learning, and interaction with supervisors for the mix of theory and practical experience (Figure 1).

Figure 1.  Thematic preferences among providers that prefer low dose, high frequency (LDHF) over traditional training.

Providers felt LDHF facilitated course focus in small manageable quantities and accommodated practice.

Respondents noted greater transfer of knowledge and skills among providers in the facility and helpfulness of frequent follow-up and repetition.

Whereas traditional trainings often select only one or two providers from a health facility, LDHF was perceived to benefit more participants and promote teamwork at the facility. Another location-based benefit relayed by respondents noted that LDHF training on-site saved patients’ time, avoided service provision interruptions, minimized provider absenteeism, and gave hands-on learning opportunities with outside facilitators.

Holding the training at the facility saved travel time and allowed more time to apply concepts to patient care.

Facility-based LDHF sessions were seen to provide opportunities for interaction and adequate time to address an individual’s learning needs or provide immediate feedback.

Respondents emphasized that LDHF provided a better balance of theory and patient-based application of skills in the clinical setting than traditional workshops.

Provider preferences for traditional workshop trainings

Despite the many advantages of LDHF, 45% of all survey respondents favored traditional training. Primary reasons included uninterrupted time to concentrate during a workshop, refreshing change of environment, valuable peer exchange, better incentives, and access to new resources with centralized workshop structure (Figure 2).

Figure 2.  Thematic preferences among providers that prefer traditional training over low dose, high frequency (LDHF).

Many preferred traditional training off-site due to time and concentration away from patient interruptions.

They also noted that a brief reprieve from work and chance to leave the facility was a refreshing change.

The chance to meet, interact, build friendships and network professionally was a key theme among respondents who preferred traditional workshop training. Shared experience with new colleagues was highly valued.

Respondents cited the higher incentives of the traditional training were preferred to those of the LDHF approach.

Providers noted other traditional workshop structure benefits: specialists and multiple trainers, new equipment accessible in central locations, and easier to maintain participation and convey all content at once.

Comparison with other training programs

Continuing medical education can be very effective in capacity building, however the many moderating variables and poor quality of cost-effectiveness studies which assess costs and measured outcomes for multiple training alternatives have inhibited the development of clear and concise guidelines on the optimal training delivery methods for different cadres and practicing sites [25-28]. Comparisons across the processes and structures of different LDHF training programs and traditional training alternatives has led to a more robust understanding of the feasibility of large-scale LDHF programs. LDHF training principles have been incorporated with promising results for maternal and newborn health in Uganda and Ghana as well as in high-income country hospitals for a training dose-response in higher retention of CPR skills with completion of additional on-site trainings [10,11,29]. Additionally, the literature suggests that training environments that are similar to those in which health providers practice are most efficient for the uptake of knowledge and skills and interactive, repetitive trainings improve learning as compared to didactic, singular trainings [30]. These benefits of LDHF were noted by providers in their perspectives of the approaches, but without acknowledging and accounting for provider preferences for peer networking, appropriate incentives, and chances to access expertise and new equipment these benefits may not be fully realized in future training programs.

Limitations of study

The PAC program provider assessments could not be used to measure attributable effect of LDHF training on knowledge and competency due to changes in the written exam, modification of competency testing methods, and different providers selected to assess at baseline vs endline. The absence of this robust training effectiveness data and the short window of program implementation pose limitations for our study. Additional challenges during LDHF on-site training – for both the PAC and Pediatric HIV programs – such as absenteeism or human resource transfers affected attendance of training sessions. On average, PAC training participants attended 73% of LDHF sessions and pediatric HIV participants attended 60% of LDHF sessions, which may have affected knowledge and competency. Training organizations did collect some facility health service delivery data, but none of these data were used to measure the effectiveness of the training program due to the short window of implementation and challenges with attribution. Beyond challenges with program evaluation, funding and program delays influenced the work of all partners and led to vastly differing implementation and assessment timeframes for the LDHF program and the comparison traditional workshop. Future research studies of training approaches should adopt a longer analytic time horizon, evaluate a possible moderating effect of health worker turnover or absenteeism on knowledge and competency gains, and optimally measure value for money along with changes in service delivery and related health outcomes such as deaths or infections averted.