Interventions included in the review

This review focuses on life–saving interventions across the RMNCH continuum of care that are directed against major causes of maternal, newborn, and under–five mortality and stillbirths, and for which there is clear evidence of effectiveness. The list of interventions included in the Global Investment Framework for Women’s and Children’s Health provided a starting point for identifying these interventions [1]. We considered both biomedical interventions and behaviors, such as the practice of exclusive breastfeeding or sleeping under a bednet (often treated as interventions for global monitoring purposes). The “essential newborn care” intervention was broken into its component practices, including thermal care, immediate breastfeeding, and chlorhexidine for umbilical cord cleansing. Water and sanitation interventions were added based on evidence of their effectiveness in reducing under–five morbidity and mortality [6]. The appendix lists the references of published peer–reviewed articles that describe the underlying evidence base. Typically, this evidence is a systematic review of the published literature on effectiveness, but occasionally it is based on consensus among experts, for example where interventions are established in practice and an evaluation of effectiveness has not been conducted, or where the lack of clinical equipoise has led to such evaluations being considered unethical. We consider measurement issues separately for the following groups of interventions: periconceptional (reproductive), antenatal, intrapartum, postnatal, feeding, under–five, and cross–cutting environmental.

Types of data that are the focus of the review

For each life–saving intervention addressed by this review, we indicate the possible mode(s) of delivery for the intervention (facility–based, community–based, outreach, and/or behavioral), and identify current sources of population–based coverage data and, for facility–based interventions, readiness or quality of care data that could be linked with careseeking data to produce coverage estimates. For coverage measurement, included data sources must provide representative information on both the numerator (individuals in need of an intervention who received it) and denominator (all individuals in need of an intervention).This review only considers population–based data from surveys and other sources that are administered regularly on a large scale (generally at national level) in low– and middle–income countries. More specialized, bespoke surveys (for example, special surveys conducted for effectiveness or efficacy studies) are not included, as these surveys typically provide data for only one country (or more commonly a sub–national area within a country) and are not a useful data source for most countries seeking to track their progress toward RMNCH goals. The review does not address the practical details of survey design such as sampling strategies and detailed sample size issues.

Population–based coverage data

Household surveys are the major source for population–based intervention coverage data in low– and middle–income countries. These surveys are particularly valuable because they typically seek to interview a representative sample of the population, and thus provide measures of coverage that take into account the entire population and for which uncertainty estimates can be calculated. This review includes only surveys with a representative sampling design that provide data at national scale and at regular intervals, the Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS) [7,8]. These are the largest international household survey programmes on population and health, and the two main sources for survey–based coverage estimates used in global databases [9]. We consider both survey programmes in this review. DHS has coordinated more than 325 nationally representative surveys in 91 countries since 1985, and MICS has carried out 279 surveys in 109 countries since 1995. Survey questionnaires are defined and revised through consultative processes that include stakeholders at global and country level. Over time, the two survey programmes have included an increasing number of coverage indicators along the continuum of RMNCH, including all the categories addressed by this review. In addition to measuring the coverage of biomedical interventions, the survey programmes measure the prevalence of behaviors such as feeding practices, as well as the coverage of water and sanitation interventions. Both programmes provide estimates for internationally agreed–upon indicators for monitoring progress in RMNCH.

Assessment of coverage measurement

For each intervention, we first assessed whether it would be theoretically possible for a representative household survey to establish the coverage denominator, ie, the population in need of the intervention. Our assessment was based on the indications for receiving a particular intervention (for example, whether the intervention is to be given to all children within a particular age range or only to children with a particular diagnosis). We considered that for preventive interventions targeted based on age or other conditions (eg, pregnancy), it would generally be possible to establish an appropriate denominator in a household survey, whereas for treatment interventions requiring a diagnosis or recognition of specific symptoms, it would be possible to establish a denominator only for easily recognizable symptoms such as diarrhea.

We reviewed the questionnaires from MICS Round 5 and DHS Phase 6 for each RMNCH intervention to determine whether the surveys provided measures of the numerator and denominator for the coverage indicator. We also noted the reference period for the coverage indicator, that is, the time period over which the indicator is measured and calculated, generally expressed as an interval of time preceding the survey interview.

Routine health system and program data

Routine data collected via the health system or by implementing programmes may also have some potential for use in estimating RMNCH intervention coverage. Potential advantages of routine data include their availability at a relatively low cost, on a continuous basis, and at facility or district level. In addition, routine data have the potential to provide information on services in greater detail than can be ascertained from respondent recall in household surveys.

However, routine data also have important limitations. Denominators are limited to those who are in contact with the health system, and therefore do not represent the population as a whole. Numerators may be over–counted, especially for services like vitamin A or immunizations that may be delivered both in facilities and through community–based activities or child health days. Many RMNCH indicators of interest are simply not available through routine data, because the numerator, denominator, or both are not collected. Routine health systems in most low– and middle–income countries are also characterized by poor data quality and completeness, and do not include important variables needed to assess equity. Some routine data may be out of date, or may only be updated irregularly. For these reasons, routine data have not been recommended in many settings for tracking key outcome and coverage indicators, and are not considered as a source for intervention coverage data for the purposes of this review.

Readiness and quality of care data

Data on service provider readiness and quality of care are typically collected through a survey or census of health providers – which may include health centers, referral facilities, and community health workers. We define readiness as the presence of the necessary drugs, commodities, and/or trained and supervised staff to administer the intervention to individuals in need. Measurements of quality require an observation–based assessment of whether an intervention was actually received by individuals in need of the intervention, but readiness variables are often used as proxies for quality. Health provider surveys record information on readiness components, and may also include observations of service provision with or without an independent assessment of the client’s need for the intervention. For this review, we sought to include assessments of the provision of RMNCH interventions that are administered regularly in multiple countries and at national scale. We excluded one–time or single–country assessments, as well as special assessments conducted for a specific study. There was substantial variation in the type of data collected by readiness assessments; we included any assessment that collected data on the availability of the necessary drugs and commodities to deliver the interventions in this review.

To identify provider assessments meeting these criteria, we hand–searched a 2009 review of health facility survey methods [10] as well as the presentations from a technical consultation on linking household surveys and provider assessments [11].

We identified five provider assessments that met our inclusion criteria: the World Health Organization (WHO)’s Service Availability and Readiness Assessment (SARA), the DHS Program’s Service Provision Assessment (SPA), MEASURE Evaluation’s Rapid Health Facility Assessments (R–HFA) and Quick Investigation of Quality (QIQ), and WHO’s IMCI quality of care assessments (previously the IMCI–MCE Health Facility Survey) ( Table 1 ).

For each intervention, we reviewed the questionnaires from these provider assessments to determine whether they assessed readiness, observation–based quality of care, or neither. Interventions not able to be measured through provider assessments were excluded. These include non–health sector interventions, such as the availability of improved water sources and improved sanitation, and interventions that are limited to use or ownership of a commodity, such as insecticide–treated bednets (ITNs). In addition, many behaviors do not lend themselves to measurement through provider assessments, although interventions seeking to influence the behavior (eg, counselling on breastfeeding practices) may be amenable to measurement in provider assessments.

Feasibility for linking study

For each intervention, we assessed the potential to measure population–based intervention coverage through an approach linking household survey data to provider assessment data, at either the individual level or aggregate community level. This approach makes use of population–based data from a household survey to generate a representative estimate of those in need of the intervention (the denominator). The numerator makes use of both population–based data (to estimate the number of individuals who sought care from a particular provider) and data from service provision assessments to determine whether the provider was “ready” to provide the intervention, information that is not available from a household survey. Feasibility of linking was assessed and categorized as highly feasible/potential/infeasible, by considering whether careseeking data for the intervention could be obtained through a household survey, and whether readiness or quality of care for that intervention could be measured through a provider assessment. Interventions for which either careseeking or readiness/quality of care could not be measured were considered infeasible for a linking study. Interventions for which readiness could not be measured but quality of care might be assessed through observation were considered potential candidates for a linking study. Interventions for which both careseeking and readiness could be measured were considered highly feasible candidates for a linking study. For example, magnesium sulfate for treatment of pre–eclampsia/eclampsia was categorized as highly feasible because careseeking (ANC consultations and facility delivery) is measured via household surveys, and readiness to deliver the intervention (availability of magnesium sulfate, dipstick for urine protein/acetic acid and flame for heating, blood pressure apparatus, and trained staff) is currently collected in provider assessments. On the other hand, treatment of neonatal sepsis with antibiotics was categorized as infeasible, because careseeking for neonatal sepsis is not currently measured in household surveys due to the difficulty in establishing a valid denominator (newborns with signs of sepsis) using a survey questionnaire.

Measurement of intervention coverage through household surveys

Twenty–four, or fewer than half, of the included interventions are currently measured through regular household surveys (DHS or MICS). Of those interventions not measured through household surveys, all but one (handwashing) are delivered at health facilities; five can also be delivered at community level and two via outreach. Two of the measured interventions are proxies for intervention coverage and actually measure careseeking, ie, skilled birth attendance and postnatal visits, rather than interventions. Many of the measured interventions fall in the under–five and environmental categories, with 11 of 18 under–five interventions and five of six environmental interventions measured in MICS and DHS. Within the under–five category, however, there are gaps with respect to measuring treatment of malnutrition and neonatal infections. Along the continuum of care, the intrapartum period stands out as the highest risk period for women and babies, and yet none of the included interventions for this period is measured in surveys beyond service contacts. Similarly, a relatively low proportion of antenatal (two of 13) and periconceptional (one of five) interventions are measured through MICS and DHS.

Measurement of readiness and quality of care

Of the 49 interventions that can be delivered at a health facility, provider assessments currently measure readiness for 27 interventions, and readiness and observation–based quality of care for 10 interventions. Those interventions not currently addressed by provider assessments are primarily periconceptional and antenatal in nature (for example, safe abortion services, calcium supplementation, and detection and management of fetal growth restriction). The WHO’s SARA and the DHS Program’s SPA are the main sources of these data. These two assessments provide data for most of the same interventions. SPAs provide a more complete assessment of health worker training and supervision, as well as the quality of services.

Feasibility of measuring coverage through linked provider assessments and household surveys

Estimating intervention coverage using a linked approach requires the ability to measure careseeking through a population–based household survey and provider readiness to deliver the intervention (or quality of delivery of the intervention) through a health provider assessment. These two sources of information then must be linked, either by matching each individual in the household survey to a particular facility, or by associating everyone in the household survey within a catchment area to a particular facility. We estimate that a linking approach would be highly feasible for 22 interventions, 13 of which are not currently measured in household surveys – five antenatal, six intrapartum, one postnatal, and one under–five intervention. For another five intrapartum and postnatal indicators, a linking approach might be feasible if observation–based provider assessments were used.

Gaps in coverage measurement

In general, we found that household surveys are not good sources of coverage data for interventions that require caregiver or respondent knowledge of specific clinical details such as a diagnosis. The exception is conditions for which biomarkers are available. Although new biomarker tests are increasingly available, their use in large–scale surveys is restricted to a few indicators and, where they are available, their use can be complicated and expensive. Household surveys are generally well–suited for measuring preventive interventions and assessing careseeking based on symptoms that can easily be recognized and recalled by mothers. There is a clear measurement gap for interventions delivered during pregnancy and around the time of birth. Household surveys primarily measure careseeking for these periods, and therefore cannot currently be used to track progress in the coverage of most reproductive, maternal, and neonatal interventions. Moreover, many of these interventions are not appropriate for measurement through household surveys, because they require a diagnosis such as pre–eclampsia or preterm premature rupture of membranes that cannot be readily established through a survey questionnaire. Household surveys are also not suited to measuring coverage of interventions needed by very small numbers of individuals (such as antibiotics for preterm premature rupture of membranes), as household surveys typically cannot achieve adequate sample sizes to provide precise coverage estimates. This gap is of particular concern given the importance of the period around and immediately after birth for the health of mothers and babies: most maternal and newborn deaths occur during childbirth and in the day following birth [13], and neonatal deaths represent a growing proportion of under–five deaths [14]. Tracking the coverage of interventions that protect against common causes of maternal and neonatal deaths is thus critical to ensuring progress in RMNCH, and is not possible at present.

Another important gap is the lack of data on the accuracy [3], precision, and reliability of the coverage data collected through household surveys. Where data on indicator validity exist, they suggest that although household surveys can provide accurate coverage measures for some interventions, such as treatment of fever with an ACT [15], other interventions such as antibiotics for pneumonia are not well measured through such surveys [16]. The question of whether coverage measurements are reliable over time and across countries is of central importance if survey data are to be used to track progress in coverage of RMNCH interventions. There is an urgent need for research to better understand which health interventions household surveys can provide accurate, precise, and reliable population–based coverage measures, and for which interventions alternative measurement approaches should be explored. A few recent studies have explored the validity of a range of coverage measures for the intrapartum and immediate postnatal period with mixed results [17]. A clear alternative to measuring careseeking (ie, skilled birth attendance) has not yet emerged for the intrapartum period.

Limitations

This review has a number of limitations. Our list of interventions was based on those in the Global Investment Framework for Women’s and Children’s Health, and included only interventions with published effectiveness estimates (see Appendix S1 in Online Supplementary Document). However, there may be interventions, particularly emerging interventions for which the body of evidence is still developing, that have been omitted. As new interventions emerge over time, there will be an ongoing need to consider whether and how to measure their coverage.

Our process for assessing the feasibility of using a linking approach to estimate the coverage of each intervention was somewhat subjective. Although we attempted to establish clear criteria for each level of feasibility, it is possible that another group might come to somewhat different conclusions. There are ongoing efforts to implement the linking approach using existing and new data. When complete, these studies will provide additional information about the feasibility of linking for various interventions.

Finally, we note that household and provider surveys and routine data continue to evolve. This review provides a snapshot of the gaps and opportunities at a particular point in time. We expect that some of the gaps identified here will be filled over time as data collection instruments are revised and routine health information systems improve.

Research and practice agendas

Providing valid, population–based estimates of coverage for RMNCH interventions at national and sub–national levels is essential to achieving reductions in maternal, newborn, and child deaths and stillbirths, and must be a priority for the RMNCH research and practice community. We recommend three parallel streams of action to improve the availability and quality of data on intervention coverage for RMNCH.