Eighty public BEmONC facilities were purposively selected to participate in each of four phases of the AMANAT program during 2015-2017. In each phase, selection of facilities was based on assessments of facility readiness for the AMANAT program, including 1) a minimum of six nurse-mentees available for training in each facility, 2) an average load of ≥100 deliveries each month, 3) basic readiness in terms of infrastructure and equipment, and 4) willingness of the facility management to participate. Decisions regarding facilities’ inclusion into various AMANAT phases also considered the proximity of intervention-ready facilities. Nurse-mentors with BSc degrees in nursing and at least two years of experience providing BEmONC services were recruited from across India in partnership with the Christian Medical Association of India through a rigorous interview process. They were familiarised with the training modules by master trainers holding master’s degrees in nursing. Pairs of nurse-mentors rotated weekly through four facilities over a period of 6-8 months. They used structured learning sessions to cover a range of topics (ie, basic nursing procedures, infection prevention, essential obstetric and newborn practices, management of complications such as postpartum hemorrhage or pre-eclampsia, teamwork and communication, documentation and reporting), as well as bedside mentoring related to the management of normal and complicated labor and delivery. The approach was learning-by-doing to the extent possible, utilising opportunities presented by the women admitted daily for delivery in each facility and enabling supervised improvements such as organising labor rooms for greater efficiency, ensuring correct storage and use of drugs and sterilisation of equipment, correct and complete clinical documentation, and periodic clinical case reviews. Despite about 50 days of intermittent exposure to mentoring, there was only limited exposure of individual mentees to the clinical management of even common complications such as postpartum hemorrhage and birth asphyxia. To make up for this gap, high-fidelity simulation training using inexpensive props, video recordings, and debriefing techniques designed by Pronto International were embedded into the AMANAT intervention and used for team-training facilitated by experts from the University of California San Francisco and University of Utah — these were focused primarily on the identification and management of intrapartum complications [6,10-12], and are described in detailed in another article in this series [11]. Highly utilised were MamaNatalie and NeoNatalie, simulation models used to demonstrate obstetric and neonatal practices. Mentoring and training were conducted in groups, in the local Hindi dialect. Nurse-mentors worked with the nurse-mentees to observe, provide instruction, and demonstrate through co-management of cases. In addition, they were available by phone during non-mentoring weeks (ie, between the week-long mentoring visits) to offer patient management guidance and enable nurses to practice per protocol.

For this analysis, we use data from the last three of four implementation phases of the AMANAT intervention, each covering blocks of 80 public facilities offering BEmONC services between September 2015 and January 2017; owing to their formative nature, baseline data from the first implementation phase were judged not suitable for pre-post intervention comparisons. Evaluation data were collected the week before the intervention (baseline) and approximately a month after completion of the intervention (endline) in all facilities where the intervention was implemented. In each facility, all nurse-mentees posted in the labour room were administered knowledge tests with 11 common questions and 19 content-similar questions at baseline and endline. General infection control measures in place in each facility were captured on separate checklist-based tools at baseline and endline. To assess maternal intrapartum and newborn clinical practices, CARE employed direct observation of deliveries by nurse-mentors, using a pre-formatted, pilot-tested tool to record the execution of key care steps and procedures. For each phase of 80 facilities, nurse mentors assigned to mentor the facilities conducted baseline observations before beginning the training over the period of one week. After the end of the mentoring period of 6-8 months, a different set of nurse mentors conducted endline observations for that phase over the period of one week. Only normal deliveries conducted by nurse mentees on Monday to Friday, 8 am-5 pm during the week when the evaluation team was present in the health facility were selected for observation. Complicated deliveries were not included in either pre- or post-assessments.

Outcomes of interest for our analysis were: 1) nurse-mentees’ knowledge score measured as the sum of correct responses to the 30 knowledge questions; 2) facility infection control score measured as the sum of routinely (ie, throughout the week of assessment) performed infection control measures out of nine measures assessed in each facility; 3) intrapartum clinical practices measured as the sum of correctly performed practices out of 22 practices assessed for all deliveries observed; and 4) newborn clinical practices measured as the sum of correctly performed practices out of 13 essential newborn practices assessed for all newborns observed. Items assessed represent essential elements of intrapartum and newborn care and were scored 1 if performed and 0 if not performed as per accepted clinical guidelines in India [13]. Given that only essential practices performed during normal labour and delivery were assessed, score items were weighted equally.

Across the three phases with data analysed here, the AMANAT intervention was implemented in 240 public facilities. Due to fieldwork problems (eg, travel delays, absenteeism and overlapping evaluation timing for nurse-mentors coming from outside Bihar; Figure 1), only 206 facilities had available data for all outcomes of interest at both baseline and endline. Because randomisation of deliveries and newborns observed was not feasible, additional information was collected during the evaluation and used in analyses. The facility type was categorised as PHC or higher-level; 2014 facility delivery volumes were obtained from the Health Management Information System (HMIS) [14] and used in analyses as a continuous variable. A measure of both available and functional facility equipment was created based on assessment of 10 types of equipment needed to perform essential intrapartum and newborn procedures (blood pressure apparatus, fetal doppler, baby weighing machine, radiant warmer, suction machine, neonatal ambu bag, neonatal mask size 0, neonatal mask size 1, oxygen concentrator, oxygen cylinder). For each equipment that was available and functional at the time of assessment, the facility received a score of 1; the equipment score is a summed score of available and functional equipment ranging between 0 and 10. Nurse-mentees’ experience was categorized as <5, 5-9, and ≥10 years. For all observed deliveries, observers captured patients’ age and parity, the time in relation to delivery when the observation started, the time when the woman was taken to the delivery room (before or after 2pm) as a proxy for nurses’ shifts, and the type of other providers present in the room at the time of delivery (doctor/staff nurse vs ANM) to account for the potential heightened attention and higher care quality offered by nurse-mentees when higher-qualified providers were present.

Figure 1.  Implementation of the AMANAT intervention between September 2015 and January 2017. AMANAT – Apatkalin Matritvaevam Navjat Tatparta, translated emergency obstetrical and neonatal readiness. Blue dots indicate start of data collection; orange dots indicate end of data collection for each phase; each of the three phases covered 80 facilities offering basic emergency obstetric care. The evaluation was on-going, with baseline data collection occurring before the intervention and endline data collection occurring about one month after the end of the intervention.

Nurse-mentee and observed delivery characteristics were compared between baseline and endline using χ² tests for proportions, t tests for means, and Wilcoxon tests for medians. Subsequently, baseline-endline changes in nurse-mentees’ knowledge were assessed as proportions of mentees who answered knowledge questions correctly, overall and by content (ie, biomedical waste, maternal intrapartum care, newborn care). The statistical significance of baseline-endline knowledge differences was tested using χ² tests for proportions and t tests for means. We estimated proportions and corresponding 95% confidence intervals for routine performance of nine infection control measures at the facility level, and of 22 intrapartum and 13 newborn practices at the observed delivery and newborn levels. Lastly, we fitted four linear regression models to explore the impact of the AMANAT intervention (endline vs baseline) and, separately, nurses’ knowledge, infection control, intrapartum, and newborn care practices. To ease the interpretation of results, outcome scores were linearly transformed to a 0-100 scale for use in regression analyses. All models were adjusted for type of facility, 2014 delivery volume, equipment score, AMANAT implementation phase, and clustering at the nurse-mentor pair level given that the same pair trained nursing staff in multiple facilities. The model fitted for nurse-mentee knowledge was also adjusted for nurse-mentee’s experience measured in years. Models fitted for intrapartum and newborn practice performance were also adjusted for nurse-mentee’s experience, observed patient’s parity, the time when the delivery observation started, the time when the woman was taken to the delivery room (before or after 2pm), and the highest qualification of the other providers assisting the delivery.

This study is part of the Ananya Bihar program and is registered at ClinicalTrials.gov number NCT02726230. The study was approved by the Institutional Committee for Ethics and Review of Health Management Research Office of the Indian Institute of Health Management Research in Jaipur, India. Written informed consent or left thumb impression was obtained from each participant before the interview or care observation. We used STATA 15 (StataCorp, College Station, TX, USA) to conduct the analyses.

The 240 AMANAT facilities in the three implementation phases analysed here represent 36 of 38 districts and 48.5% of facilities offering BEmONC services in Bihar (Table 1). Our analytic sample includes data collected in 206 (85.8%) of these 240 facilities, four-fifths of which were PHCs. The median number of deliveries performed in these facilities in 2014 was 2124. Overall, 1892 and 1822 nurse-mentees completed knowledge assessments at baseline and endline, respectively. More deliveries were observed at baseline than endline (mean 3.7 vs 3.0 deliveries per facility). Patient observations started closer to their arrival/admission time at endline than baseline, and considerably more deliveries were observed in the morning than the afternoon shift at both baseline and endline.

Nurse-mentee knowledge

Nurse-mentees’ knowledge improved significantly after the intervention. Only 17% of nurse-mentees interviewed at baseline compared with 93% of those interviewed at endline answered >50% of questions correctly; none of them answered >70% of questions correctly at baseline, while 51% did so at endline (Figure 2, Panel A). On average, nurse-mentees answered correctly 38% of questions at baseline and 68% of questions at endline (P < 0.001), with knowledge of neonatal care aspects being relatively higher than knowledge of maternal intrapartum care or general biomedical waste management (Figure 2, Panel B).

Figure 2.  Nurse-mentee knowledge before and after the AMANAT intervention. Panel A. Changes in the proportion of nurses responding correctly to knowledge questions between baseline and endline. Panel B. Changes in proportion of questions correctly answered between baseline and endline by content. AMANAT – Apatkalin Matritvaevam Navjat Tatparta, translated emergency obstetrical and neonatal readiness. All shown differences between baseline and endline proportions are statistically significant at P < 0.001.

Infection control practices

All nine infection control measures assessed were significantly more prevalent at endline than baseline (all P < 0.006; Figure 3). Seven of these measures were practiced in >50% of facilities at endline, of which two (ie, daily preparation of 0.5% hypochlorite solution for soaking medical instruments and disposal of placenta in correct containers) were observed in >80% of facilities.

Figure 3.  Infection control practices before and after the AMANAT intervention. AMANAT – Apatkalin Matritvaevam Navjat Tatparta, translated emergency obstetrical and neonatal readiness. Columns are proportions and corresponding 95% confidence intervals of facilities performing each of the infection control measure shown. All differences between baseline and endline are statistically significant at P < 0.006 or better.

The AMANAT intervention was an emergency response to a health workforce capacity crisis in the face of a rapid increase in the number of women delivering in public facilities. It was not aimed to function as a structural change or a replacement for in-service or other training methods for EmONC providers in Bihar. Rather, it was designed to overcome the pitfalls of other capacity building methods by being minimally disruptive to service delivery and by including a mentoring component that motivates providers and amplifies behavior and clinical practice changes. Offered as part of a comprehensive health system strengthening and quality improvement initiative in public facilities, this intervention also aimed to instill a sense of professional pride and shared interest among EmONC providers to offer quality services [6].

AMANAT led to increased adherence to evidence-based care and improved performance of essential intrapartum and newborn practices by nurses in public BEmONC facilities in Bihar. Exposure to the intervention improved nurse-mentees’ knowledge and performance of infection control and intrapartum practices by about 30%, and their performance of essential newborn practices, which was already higher at baseline, by 24%.

Among facility-level infection prevention measures, the appropriate cleaning and disinfection of delivery instruments saw the most impressive uptake between baseline and endline assessments. Also, provider-level practices to control infection before and during delivery (ie, hand washing; use of disinfected, sterile instruments) were considerably more prevalent after the intervention. Especially encouraging are changes observed in the initial assessment (ie, vital sign checking, fetal heart rate monitoring) of pregnant women upon arrival at the facility and before delivery – without a comprehensive initial examination, it is difficult to recognize or prevent potential complications. Importantly, with postpartum hemorrhage being a key contributor to maternal mortality in India [15], we found promising changes in the active management of third stage of labor practices known to reduce occurrence of severe postpartum hemorrhage by 60%-70% [16]. After the intervention, in about 72% of deliveries observed, oxytocin was given in the proper dose (10 IU) intramuscularly within 1 minute of delivery; controlled cord traction was practiced in 74% of deliveries observed; and uterine massage after the delivery of the placenta accompanied about 86% of deliveries observed. These represent changes of about 50%, 56%, and 366%, respectively, as compared to baseline. Areas of intrapartum care that were slower to change included use of personal protective equipment (ie, apron and masks) with all deliveries, episiotomies and related use of local anesthesia, temperature checking during initial assessment, checking of placenta for completeness, and cleaning the perineum after delivery. Also notable are the observed improvements in newborn cord care, placement on the mother’s abdomen immediately after birth, and early initiation of skin-to-skin care.

Our study is not without limitations. Given facility readiness differences, a cluster-randomised design evaluation was deemed infeasible. The pre-post design and the use of a uniform process of selecting facilities in each AMANAT phase minimised the effect of between-facility variations and allowed for facilities to be reached within the stipulated time. Baseline and endline data were not available for all outcomes in all facilities where AMANAT was implemented. However, available state-level data on key facility characteristics are reassuring with regard to the statewide representativeness of our results. Of the 495 public BEmONC facilities in Bihar, 83% are PHCs; they are served, on average, by 12 health workers and offer about 2200 deliveries each year. By comparison, among AMANAT facilities in our analysis, 78% were PHCs, they had an average of 11 health workers, and performed 2296 deliveries per year [14]. Only normal deliveries that occurred during the day were observed at both baseline and endline. This was because pregnancy complications were not frequent enough to be reliably observed in all facilities and in order to minimise the potential influence of complication severity and differences in 24/7 availability of human resources on our findings. Observations were only conducted Monday-Friday during daytime hours, which may have overestimated the effects of the intervention on essential maternal and newborn practice uptake given potential staff shortages that are more likely to occur during weekends and evenings.

There is limited documentation of experience with and impact of mentoring and training interventions on improving providers’ EmONC knowledge and skills in low- and middle-income countries [5,7,8,10-12,17-19]. For such skill building, they appear to be superior to traditional, short-duration trainings away from health facilities because they offer the option of both simulated and bedside learning and practicing [5], but their effectiveness varies based on context, barriers and facilitators to implementation [12]. Nonetheless, all studies to date furnished positive evidence regarding the acceptability of such interventions. Most recently, a qualitative study in Karnataka, India found that mentors provided training and support in a non-threatening manner, and mentees perceived them as helpful, trusted resources; nurse-mentees frequently contacted nurse-mentors between trainings to seek advice or confirm clinical decisions, which helped ingrain clinical skills and adherence to guidelines into daily practice [17]. Our results add to this body of evidence.

There are lessons to be learned from the AMANAT intervention in Bihar. First, there is value in having started with a sufficiently large, pilot program that provided information on feasibility, acceptability, design, and potential success of the intervention upon scale-up. This is demonstrated by the little variation in outcomes by implementation phase (data not shown). Second, the intervention could not have been implemented with fidelity at such large scale without the full collaboration of the GoB and of providers working in public facilities. Mentors’ ability to train and prompt nurse-mentees’ changes in knowledge and practices could have been completely undermined if medical officers or facility leadership were not in favour of the intervention. Third, the intervention was offered within a larger, statewide quality improvement framework in public facilities. As a result of this initiative, we found improvements in available and functional equipment in intervention facilities between baseline and endline assessments, and in regression analyses, the equipment score was found to be a significant positive predictor of nurse-mentees’ knowledge and practice scores. Fourth, joint mentoring by 2 nurse-mentors in each facility, despite its cost implication, allowed them to divide responsibilities and engage more intensively with the nursing staff. Busy facilities were expected to require more mentoring support than lower volume facilities [17]. Yet, we did not find that delivery volumes influenced mentees’ knowledge or performance in a significant manner. On the other hand, we found significantly higher scores of mentees’ knowledge and practice of general infection prevention measures in higher than PHC-level facilities. This may be due to having better trained nurses (ie, more GNMs) in these facilities or to having a higher proportion of complicated cases in these facilities and thus more opportunities for nurse-mentees to learn and practice. Fifth, some elements of the AMANAT intervention should be considered in the design of future training interventions for EmONC providers in Bihar. The mentoring model can assist with scalability and sustainability of future training interventions as mentees not only become agents of practice and quality culture changes in the facilities where they work, but potentially beyond if further trained to serve as nurse-mentors. This will also address the limitations with having to bring nurse-mentors from outside the state. Given the challenges with finding nurse-mentors in Bihar for training, the intervention was modified early in 2018 to no longer rely on availability of mentors from outside the state. Professional organisations worldwide recommend the use of simulations for in-service training in obstetrics and neonatology [20] – simulation should be incorporated in future EmONC provider trainings in Bihar if funding is available. Simulation not only gets the team involved but allows them to practice in a more realistic environment where there is no risk to actual patients. To build skills in recognising and managing pregnancy complications, mentees could be rotated through high-level facilities to observe and practice in different environments. Inclusion of team-based problem solving and accountability techniques in future training curriculum may lead to wider adherence to clinical guidelines than accomplished by AMANAT. Also, use of debrief meetings that potentially involve labor and delivery units’ leadership can help share progress and action plans for continuous quality improvement. Finally, despite positive results, AMANAT endline scores represented about 72%, 61%, 57%, and 74% of maximum knowledge, infection control, intrapartum, and neonatal practice scores, respectively. Considerable gaps in knowledge and practice remained after the AMANAT intervention in facilities exposed to the intervention. Emergency response interventions like AMANAT are resource-intensive, cannot fully solve broad health workforce capacity problems, and cannot replace either other training methods or the need for close oversight of the quality of intrapartum care. Quality improvement initiatives that include teamwork and communication as well as simulation trainings are expected to close the gaps in knowledge as well as the gap between knowledge and practice.

Learning from the AMANAT intervention and building on a much stronger health system in Bihar in 2019 than 2011 [6], future mentoring and training interventions need to address current knowledge and practice gaps among EmONC providers and institutionalise a focus on quality of care and patient safety.