Program description

The program approach aimed to address demand, supply, and policy barriers to ORS and zinc uptake in both public and private sectors. The interventions were categorized into four major areas: (1) policy revision and partner coordination; (2) market shaping to improve availability of affordable, high-quality ORS and zinc; (3) provider training and mentoring; and (4) caregiver demand generation. In Table S1 in Online Supplementary Document, we describe in detail the program activities under each of these four areas roughly separated across two program phases: between the baseline and midline surveys and between the midline and endline surveys.

For policy revision and partner coordination, the program partnered with the FMOH, NPHCDA, and state Ministries of Health to establish coordination mechanisms at national and state levels. Led by the government, the National Essential Medicines Coordinating Mechanism (NEMCM) consisted of various government departments, private industry, and non-governmental partner organizations [7]. The aim of the NEMCM was to mobilize resources for implementation of Nigeria’s Essential Childhood Medicines Scale-up Plan and to align and coordinate key stakeholders around the scale-up plan and updates to national treatment policies. By 2014, the NEMCM partners supported government efforts to revise to the national treatment guideline to recommend ORS and zinc for childhood diarrhea, add zinc to the national and state Essential Medicines Lists (EML), and broadly communicated the policy allowing zinc to be available over-the-counter. The NEMCM also conducted a partner mapping exercise to identify investments by non-governmental stakeholders and coordinated those investments to expand the reach of program implementation to reach underserved areas. For example, the Sustaining Health Outcomes through the Private Sector (SHOPS) program selected Benue and Kebbi state for additional investments in ORS and zinc scale-up based on the partner mapping exercise [8].

For market shaping, the program engaged local manufacturers to encourage investments in production, promotion, and sales of optimal zinc and ORS products— including zinc dispersible tablets (DT), the WHO-recommended low-osmolarity ORS formulation, and co-packaged ORS and zinc. Market intelligence on Nigeria’s demand and planned orders was provided to suppliers through regular forums and reports. In addition, suppliers received one-on-one technical assistance on product registration, cost reduction, marketing, and product and packaging optimization strategies. This support led to the introduction of four new low-osmolarity ORS, six zinc DT, and seven co-packaged ORS and zinc products into the Nigerian market. Beyond the technical support to local suppliers, the program worked with Nigeria’s National Agency for Food and Drug Administration and Control (NAFDAC) to encourage suppliers to switch to a low-osmolarity ORS formulation. Prior to the program, 21 different ORS products were already available in the Nigerian market, though there was only one low-osmolarity ORS registered in the country, no zinc DT, and no co-packs. By the end of the program, there were 35 low-osmolarity ORS, 12 zinc DT, and 10 co-packs registered with NAFDAC. Table 1 presents the number of low-osmolarity ORS, zinc DT, and co-packs registered with NAFDAC prior to the program and by program’s end. While the program partnered directly with a handful of willing suppliers to strengthen the local supply base of ORS and zinc, the program was overall agnostic to the brand of ORS and zinc used by patients and aimed to increase the overall use of ORS and zinc by patients.

Furthermore, the program worked along the entire private-sector supply chain in Nigeria to expand the reach of ORS and zinc in rural areas. The program used innovative private sector strategies and streamlined distribution models targeting wholesalers, sub-distributors, and retailers. For example, the program placed brand-agnostic marketers at wholesale outlets to distribute educational materials and encourage private retailers patronizing wholesale outlets to stock ORS and zinc. The market shaping activities in the private sector led to a large increase in ORS and zinc sales by major ORS and zinc suppliers in Nigeria. Table 2 presents annual sales of ORS, zinc, and co-packs from seven major local suppliers during the program period.

The program also supported the state governments to optimize planning and procurement of ORS and zinc, including introduction of an ORS and zinc co-pack, into public facilities. The program provided technical assistance to quantify ORS and zinc demand in public facilities and advocated for inclusion of these commodities in the state procurement budgets. Additionally, the program worked with state drug management agencies to include ORS and zinc in the state drug revolving funds (DRFs) and negotiated reduced prices. By the end of the program, we observed an increasing preference by state governments for procuring co-packs. As shown in Table 2, public sector procurement gradually shifted from single ORS and zinc to co-packs.

The program aimed to switch the practices of health care providers to use ORS and zinc for treatment of pediatric diarrhea through multiple interventions. The program implemented targeted activities in the public and private sector. First the program conducted diarrhea management trainings with leaders of health worker professional groups, such as the Nigerian Medical Association (NMA), Pharmaceutical Society of Nigeria (PSN), National Association of Nigerian Nurses and Midwives (NANNM), and National Association of Proprietary Patent Medicine Vendors (NAPPMED) to secure buy-in. The program subsequently rolled out state-level trainings to nearly 20 000 health care providers in the public sector. In the private sector, the program partnered with the NAPPMED and the Pharmacists Council of Nigeria (PCN) to conduct state-level trainings to nearly 19 000 patent and proprietary medicine vendors (PPMVs). Following up on the state-level trainings, the program worked with existing government supervisory structures to provide individual mentorship of public health care workers on diarrhea case management. Similarly, in the private sector, the program worked with NAPPMED and PCN to conduct shop-to-shop detailing with their network of PPMVs and pharmacists.

The fourth area of intervention was to directly reach caregivers and encourage them to seek care early and provide them with education on ORS and zinc. The program worked with key influencers in the community, such as religious leaders, female vanguard associations, Islamiyah schools, and local health care workers, to incorporate messages in their communication platforms. A radio campaign was also implemented in selected states.

The first half of the program focused on increasing procurement of ORS and zinc in the public sector and getting negotiated prices, mapping private sector wholesale and retail networks and implementing activities to push ORS and zinc through the private sector supply chains, and large-scale training programs for both private and public sector providers to orient them to ORS and zinc and link them to suppliers. Limited community engagement occurred during the first half of the program. In the second half of the program, the focus switched to conducting multi-contact interventions that leveraged sustainable channels, such as government supportive supervision activities and pharmaceutical detailing. The majority of community engagement activities also occurred in the second half of the program, such as utilizing key influencers, facility-based “health talks”, and radio.

Study objectives

The purpose of this study is to evaluate the program and to present the results of the household surveys conducted at baseline, midline, and endline using analytical techniques. To estimate the potential program impact, we focused on five outcomes: 1) the change over time in care-seeking for children with diarrhea in the past two weeks; 2) the change over time in treatment with ORS for children with diarrhea in the past two weeks; 3) the change over time in treatment with combined ORS and zinc; 4) whether the change over time in treatment with ORS was modified by the source of care (ie, home, public sector, private sector, other sector, or multiple sectors); and 5) whether the change over time in treatment with combined ORS and zinc was modified by the source of care.

Evaluation study design and setting

This evaluation consisted of repeated cross-sectional, multi-stage, cluster-based household surveys carried out separately in Bauchi, Cross River, Kaduna, Kano, Katsina, Lagos, Niger, and Rivers (Figure 1). The eight states represent a wide variety of cultures, health, and economic outcomes and cover five out of the six geopolitical zones (only South East is not represented). According to the 2006 census, out of the 36 states and the Federal Capital Territory of Abuja, these 8 states combined represent 36% of the under-five population and 34% of the total population [9]. The state-level under-five mortality varies and estimates ranged from the highest in Bauchi state (186 per 1000 live births) to the lowest in Lagos and Rivers (74 and 96 per 1000 live births) [10]. Across all states, diarrhea is a leading cause of deaths among children under five, and the eight states represent 40% of the total pediatric diarrhea burden in Nigeria. Prior to the program, the UNICEF MICS 2011 survey estimated ORS coverage to be the lowest in Bauchi (12%) and highest in Lagos (62%) [11].

Figure 1.  Map of program states.

Study population

The study population was children under five who resided in one of our eight states within Nigeria where the program was implemented and had diarrhea in the past two weeks.

Sampling design

The sampling design aimed to obtain a state-representative sample of households with children under five. For logistical and financial purposes, we performed clustered sampling by randomly selecting households within randomly selected census enumeration areas (EAs). The sampling frame for the study was the 2006 national census. Working with Nigeria’s National Bureau of Statistics (NBS), we first stratified all EAs by state, and urban and rural areas. Next, we randomly selected EAs by using a random number generator in Microsoft Excel (Microsoft Inc, Seattle WA, USA). Population figures were not available for census EAs so we could not conduct probability proportional to size sampling. The sizes of the EAs varied, and we incorporated sampling weights to account for differences in EA sizes which are described later in the paper. Within each EA, trained enumerators listed all households living in the EA and screened for households with at least one child under five living in the household. After completion of the listing, the interviewers conducted a systematic random sample of households with children under five. Based on our experience during piloting, we found that sampling 20 households with at least one child under five per EA was achievable even in the smallest EAs in all states except for Cross Rivers and Rivers where we reduced the number to 15. Due to security issues during data collection, we excluded certain EAs from the sample selection. A new sample was created for each wave of data collection. Figure 2 presents the number of households interviewed at each wave and the number of children under five and diarrhea episodes included in the sample.

Figure 2.  Sample results.

Data collection procedures

Independent research firms were hired to conduct the data collection: Nielsen Nigeria for the baseline and Practical Sampling International for the midline and endline. The research firms hired enumerators who were familiar with the local context, could speak the local languages in the study states, and had experience conducting household surveys. Both male and female enumerators were hired, though depending on the state context, a greater number of female enumerators were hired to comply with local norms on conducting one-on-one interviews with primarily female caregivers. Enumerators were trained for five days prior to the launch of each data collection period. Training covered background on the project, household listing procedures, review of all survey questions, research ethics, use of the electronic data collection software SurveyCTO (Dobility Inc, Cambridge MA, USA), and a day of field practice. Teams consisting of four interviewers and one supervisor were formed.

The target respondent for the household survey was the primary caregiver of children under five in the household. The household survey included questions regarding household characteristics, respondent characteristics, characteristics of the child, and if the child had diarrhea in the two weeks preceding the survey, detailed information regarding the resolution of the diarrhea, such as source of care and whether the child was given ORS and zinc. Flip charts of local ORS, zinc, and other diarrhea treatments were used to aid the respondent’s recall. The survey was translated into Hausa, Pidgin, and Yoruba, and back translated to English to ensure the question’s intent was not lost. The baseline survey was conducted on paper and double entered into CSPro (US Census Bureau, Washington DC, USA). Discrepant entries were resolved by a third data entrant. The midline and endline surveys were conducted electronically using mobile devices programmed with SurveyCTO.

Data collection was also organized by funding source. Activities in Kano, Lagos, and Rivers, including the program evaluation, were funded by Norad while activities in Bauchi, Cross River, Kaduna, Katsina, and Niger were funded by GAC. Due to differences in donor reporting timelines, data collection for Norad-supported states were different than GAC-supported states. Additionally, the presidential election in 2015 also affected the timing of the Norad midline survey as Nigerians often return to their home state to vote rather than vote in their usual place of residence. The program was also concerned with any potential disruptions that may have resulted from the presidential elections. To avoid effects of population movement during the elections and potential disruptions after the elections, the program decided to conduct the midline prior to the elections. Table 3 presents the timelines for data collection by funding source.

Sample size calculations

Sample size calculations were designed to ensure adequate power to detect change in ORS and combined ORS and zinc coverage in each state. We used the following formula to calculate the sample size for the study:

where N is the desired sample sizes of children with diarrhea in each state (assuming one child per household), Deft is the design effect due to clustering which we assumed to be 1.5 [12], p1 is the 2011 state coverage estimates (the most recent coverage data available at the time) [11], and p2 is the endline estimates necessary to see a 25% difference over time. We use a two-sided t test with 95% confidence, 80% power, and equal variances. We took into account the prevalence of diarrhea found and allowed for a 5% non-response rate [11]. To simplify training and field work management, assuring better data quality, we used the largest sample size required (ie, Lagos) as the sample size for all states. In each state, the sample size was 940 households with a child under five. Based on the lower density of population within Cross Rivers and Rivers, the sample size was reduced to 930 households with children under five.

Statistical analyses

We first appended the survey data sets from each state and survey wave into a single data set. We then merged the survey data set with the EA listing data set and constructed probability selection weights and post-stratification weights to account for unequal probability of selection. Additional information on calculations for developing the survey weights is included in Appendix S1 in Online Supplementary Document. Descriptive statistics for characteristics of the child, caregiver, and household were calculated for each survey period and were compared using χ-2 tests.

To evaluate whether the five outcomes changed during the program period, we constructed models using generalized estimating equations with the logit link function and computed crude and adjusted odds ratios and 95% confidence intervals. For our fourth and fifth outcomes, we examined if the associations between the survey period and the odds of a child receiving ORS or combined ORS and zinc were modified by source of care by including an interaction between the dummy variables for survey period (ie, baseline, midline, and endline) and source of care (ie, home, public sector, private sector, other sector, or multiple sectors).

For all models above, we identified potential confounders using a priori knowledge; these were factors that we believe may be related to use of ORS or combined ORS and zinc as well as factors that may have changed over time. We first conducted bivariate tests between the outcome and each covariate. The final multivariable models included characteristics of the child, caregiver, and household. For child characteristics, we included the child’s sex, child’s age categorized into 1-year age intervals, and source of care for the diarrhea episode. Sources of care were categorized into 5 mutually exclusive categories: did not seek advice or care outside the home, sought care in the public sector only (ie, government hospital or clinic), sought care in the private sector only (ie, private doctor, PPMV, pharmacist), sought care in other source only (ie, market, traditional healer, friend or family), and sought care from multiple different sources. For caregiver characteristics, we included the caregiver’s sex, caregiver’s age categorized into 10-year age intervals, and whether the caregiver had attended any level of schooling. For household characteristics, we included variables for the state the household was located in, urban or rural classification, size of the household, whether the household had access to an improved water source as defined by the Demographic and Health Survey (DHS), and the wealth status. For each survey period and state, wealth scores were independently generated using principle component analysis of survey questions on household assets and categorized into quintiles [13].

To illustrate our model results, we used the multivariable interaction models to estimate predicted probabilities and conducted contrast tests to further evaluate whether there were statistically significant differences in predicted probabilities of treatment by survey period and source of care.

All results incorporated probability selection weights and post-stratification weights to account for unequal probability of selection; additionally, the modeling accounted for the clustering at the census enumeration area and household levels. All analyses were conducted in Stata 14 (StataCorp, College Station TX, USA).

Ethical approval

The study was reviewed and approved by the National Health Research Ethics Committee of Nigeria (NHREC) (protocol number NHREC/01/01/2007). Ethical approval was also obtained from each state government research ethics committee. Written informed consent was obtained from all participating respondents.

Characteristics of caregivers, children, and households

We completed 23 295 interviews resulting in a total of 37 886 children under five listed and a study sample of 5416 children under five with diarrhea in the two weeks preceding the survey (Figure 2). The two-week prevalence of diarrhea was found to be 11% at baseline, 13% at midline, and 16% at endline. Prevalence of diarrhea at endline was significantly higher than at baseline (P < 0.01). We also found in our descriptive analysis a statistically significant increase in the primary study outcomes. At baseline, we found 38% (95% CI = 34%-42%) of diarrhea episodes received ORS and 4% (95% CI = 3%-5%) received both ORS and zinc. At endline, we found 55% (95% CI = 51%-58%) received ORS and 30% (95% CI = 27%-33%) received both ORS and zinc (Table 4).

Characteristics of diarrhea episodes in the last two weeks preceding the survey at each time period were generally similar (Table 4). There were no statistically significant differences between endline and baseline in the sex of the child with diarrhea, the schooling of the child’s caregiver, and the household size, wealth status, and access to improved drinking water. A few notable exceptions were age of the child, age and sex of the caregiver respondent, and location of the household. At endline, 22% (95% CI = 20%, 24%) of diarrhea episodes were among children 0-11 month compared to 12% (95% CI = 10%, 15%) at baseline. Conversely, there were less children age 36-47 months and 48-59 months at endline than at baseline. Only 15% (95% CI = 13%, 17%) of diarrhea episodes in our sample at endline were among children 36-47 months compared to 19% (95% CI = 17%, 22%) at baseline, and children 48-59 months old made up 9% (95% CI = 8%, 10%) of our sample of diarrhea episodes at endline compared to 20% (95% CI = 17%, 22%) at baseline. We found that more of our caregiver respondents were female at endline compared to baseline. At endline, 94% (95% CI = 92%, 96%) of our caregiver respondents were female compared to 90% (95% CI = 88%, 92%) at baseline. The sample of diarrhea episodes at endline was less rural than our sample at baseline. At endline, 55% (95% CI = 51%, 59%) of diarrhea episodes came from rural areas while at baseline the figure was 65% (95% CI = 60%, 69%). Lastly, a lower proportion of diarrhea episodes came from Rivers state (P < 0.01) and a greater proportion of diarrhea episodes were found in Cross River at endline compared to baseline (P = 0.03).

Association between survey period and seeking care for diarrhea

Table 5 presents crude and adjusted odds ratios (aOR) for our first outcome of seeking advice or care outside the home by survey period. From the multivariable model, the adjusted odds of caregivers seeking care at midline were not significantly greater than at baseline (aOR: 1.16; 95% CI = 0.91, 1.50; P = 0.24). At endline, however, the adjusted odds of care-seeking were 1.43 (95% CI = 1.12, 1.82) times greater than baseline (P < 0.01).

Association between survey period and treatment with ORS and zinc

Table 6 presents crude and adjusted odds ratios for our primary study outcomes: treatment with ORS and treatment with combined ORS and zinc by survey period and source of care. Adjusting for all other covariates, the overall odds of diarrhea episodes being treated with ORS were 1.80 (95% CI = 1.37, 2.38) times greater at midline than baseline and 1.88 (95% CI = 1.46, 2.43) times greater at endline. The adjusted odds of diarrhea episodes being treated with ORS and zinc combined was 9.63 (95% CI = 6.10, 15.21) times greater at midline and 15.14 (95% CI = 9.82, 23.34) times greater at endline.

When we included the interaction term to investigate whether the odds ratios between the endline and baseline survey were modified by source of care, we found statistically significant results for treatment of ORS among diarrhea episodes that sought care in the public and private sector. For diarrhea episodes that did not seek care, the adjusted odds ratio of receiving ORS at endline compared to baseline was 0.84 (95% CI = 0.54, 1.46). Relative to this, the odds of receiving ORS is further increased by a factor of 4.49 (95% CI = 2.21, 9.11) when seeking care in the public sector and 2.43 (95% CI = 1.31, 4.52) when seeking care in the private sector. For receipt of combined ORS and zinc, we found the odds ratio for endline was also significantly modified by seeking care in the public sector and private sectors. Among cases not seeking care, the adjusted odds ratio at endline compared to baseline was 4.24 (95% CI = 1.59, 11.27). The adjusted odds ratio was further elevated by a factor of 3.94 (95% CI = 1.16, 13.41) times when seeking care in the public sector and 5.64 (95% CI = 1.67, 19.09) times for cases seeking care in the private sector (P < 0.01).

To further illustrate how ORS treatment by survey period is modified by source of care, Table 7 presents predicted probabilities from the interaction models for treatment with ORS and treatment with ORS and zinc combined by survey period and source of care. Among cases not seeking advice or care outside the home, the predicted probability of receiving ORS was 26% (95% CI = 18%, 33%) at baseline and 24% (95% CI = 19%, 28%) at endline with no statistically significant difference between the survey periods (P = 0.64). Among cases seeking care in the public sector, the predicted probability of receiving ORS increased from 57% (95% CI = 50%, 65%) to 83% (95% CI = 79%, 87%) between baseline and endline (P < 0.01). The predicted probabilities also increased among cases seeking care in the private sector from 41% (95% CI = 34%, 48%) to 58% (95% CI = 54%, 63%). There was no statistically significant difference in the predicted probability for receiving ORS among cases seeking care from other sources or multiple sectors.

For receipt of combined ORS and zinc, there was a statistically significant increase in all sources of care. Among cases not seeking advice or care outside the home, the predicted probability increased from 2% (95% CI = 0%, 4%) to 8% (95% CI = 6%, 11%). Cases seeking care in the public sector saw the largest increase in predicted probability from 7% (95% CI = 3%, 11%) to 51% (95% CI = 47%, 56%). Among cases seeking care in the private sector, the predicted probability increased from 2% (95% CI = 1%, 3%) to 29% (95% CI = 26%, 33%).