Of the 27 experts that proposed research questions, approximately 26% were based in LMICs in Africa, Asia, and South America, 19% were program experts, 70% were researchers, and 11% were involved in both research and programing. In contrast, 25% of the 20 respondents who provided scores from LMICs, 5% were program experts, 85% were researchers, and 10% were involved in both. The characteristics of the study participants are summarized in Figure 2 .

Figure 2.  Background characteristics of respondents. The inner graphs indicate characteristics of the 27 experts who proposed research questions. The outer graphs indicate characteristics of the 20 experts who provided scores.

The scorecard contained 855 fields in total, and across the 20 scorecards returned, experts completed an average of 76.3% (652.4) of fields. Table 2 shows the 23 research questions with a research priority score above 85.00, and Annex I includes the complete list of ranks and scores for all 57 questions. Both tables present the perceived likelihood that each research question will comply with each of the five chosen priority–setting criteria. The research priority scores ranged from 61.01 to 93.52, with a median of 82.87. The average expert agreement scores ranged from 0.50 to 0.90, with a median of 0.75. Similar to past CHNRI exercises, average expert agreement showed a strong positive association with research priority score, as evidenced by a Pearson correlation coefficient of 0.967 (P < 0.0001). This finding indicates that there was strong agreement among experts about what were considered priority research questions.

The 3 top–ranked research questions were: i) “How can interventions and packages to reduce neonatal mortality be expanded to include ECD and stimulation interventions?”; ii) “How does the integration of ECD and MNCAH&N interventions affect human resource requirements and capacity development in resource–poor settings?”; and iii) “How can integrated interventions be tailored to vulnerable refugee and migrant populations to protect against poor ECD and MNCAH&N outcomes?”. The fourth highest–ranked question – ‘What are the benefits, if any, of linking ECD programs with microcredit or conditional cash transfer programs?’ – received a perfect score for the effect on equity criterion.

The 23 highest–ranked questions varied across the continuum of care, with explicit mention of all populations of interest: neonates (question #1), infants and children (#5, 6, 10, 12, 17, 20, 23), adolescents (#7), and mothers (#19, 20, 23). There was also a particular emphasis on at–risk populations, including: refugees and migrant workers (#3), small for gestational age infants (#5), children with nutritional and cognitive deficits (#10), and mothers and children vulnerable to violence (#20, 23). Moreover, research questions pertaining to capacity development and responsibilities of community health workers (#2, 8, 11), responsive and complementary feeding (#6, 14), and cost–effectiveness and financial incentive programs (#4, 18, 22) were identified as top priorities. Mobile phones and media were proposed as a potential delivery platform for integrated interventions (#15), and there were two highly ranked research questions about determining the parameters for quality assessment of integrated programs (#13, 17).

The present exercise engaged a diverse group of global health experts with knowledge and experience across the continuum of care for MNCAH&N strategies as well as ECD relevant interventions and delivery platforms. The CHNRI method’s systematic ranking of proposed research priorities against predetermined criteria made apparent some of the strengths and weaknesses of competing research investment options, and offered greater replicability and transparency than Delphi or other consultative processes [17].

The research question that received the highest research priority score pertained to the integration of ECD packages to interventions to reduce neonatal mortality. Programs that address neonatal mortality and morbidity provide an opportunity to intervene early to optimize development, and thus fulfill a major principle in addressing risks to child development. Moreover, the comprehensive list of highly–ranked research priorities also highlighted key elements of scaling up coverage of integrated interventions in LMICs, including: human resource and capacity development, cost–effectiveness and incentive schemes to reduce financial barriers, and quality assessment of integrated health programing. These three central themes, along with questions about harnessing the capacity of information technology and mobile health platforms, feature heavily across implementation–focused CHNRIs, indicating strong agreement that they are priority implementation challenges in global health [15,18,19].

Although the CHNRI method represents a systematic attempt to address the challenges inherent in the complex process of research investment priority setting, the approach is not without limitations. Yoshida and colleagues conducted an analysis of the CHNRI methodology [20], examining the concordance among top ranking research priorities as sample size increases from 15 to 90. They found that a high degree of reproducibility of top ranking research priorities was achieved with 45–55 experts, suggesting that our relatively small sample of 20 scorers may be a limitation. However, it should be noted that they still observed an appreciable degree of reproducibility with a sample size of only 15 persons. An additional potential limitation of the present study is the possibility that there were sound research options that were not included in the list of questions generated by experts. These options, therefore, could not have been scored and identified as priorities. It is also possible that the list of highly–ranked research priorities might have been different if there was greater representation of program experts or policy makers; for instance, they may have provided more detailed questions pertaining to their specific implementation challenges. Proposed research questions and their subsequent scores were limited to the opinions of the experts involved in the exercise. In an effort to minimize response bias, we employed a comprehensive process of identifying experts with relevant knowledge to participate in the study. The predetermined CHNRI criteria also ensured that questions were anonymously scored against a transparent and standardized set of values; thus, eliminating the advantage of more eloquent speakers advocating for their own research agenda. Lastly, experts might have scored questions about patient populations, interventions or health conditions outside of their area of expertise. To avoid inaccurate scores, experts were instructed to leave the cell blank when they did not feel sufficiently knowledgeable to answer a particular question.

The top–ranked research questions addressing capacity development and retention strategies reflect the current global shortage of skilled health workers with the ability to implement integrated ECD and MNCAH&N interventions. This finding is highly consistent with what others have previously identified as a top priority of scaling up integrated ECD programs [21,22], lending further credibility to our results. Collectively, the 75 countries with more than 95% of the current burden of maternal and child mortality have an estimated median of 10.2 physicians, nurses and midwives per 10 000 people, and three–quarters are below the World Health Organization benchmark of 22.8 per 10 000 [1]. Task–shifting has been successfully implemented in several countries to increase access to essential interventions. Community health workers are well positioned to respond to local cultural and societal norms and to foster the acceptability and uptake of integrated interventions. However, it has been argued that a potential disadvantage of integrating programs is the risk of overloading health services and reducing their effectiveness [23]. Hence, question eleven – “What is the feasibility of integrating ECD interventions into the responsibilities of community health workers, and what specific interventions should be prioritized?” – is especially critical to inform the integration debate.

Three highly–ranked questions related to the financial aspects of integrated implementation; in particular, assessing the cost–effectiveness of different delivery models, identifying financing gaps, and linking ECD programs with microcredit or conditional cash transfer programs. Support platforms that provide direct or indirect monetary incentives to households have been employed for decades in Latin America and Sub–Saharan Africa, and more recently, in South Asia [24]. Such financial incentive programs are widely implemented strategies to improve health inequities and have been shown to alleviate poverty, improve access to health services, and scale up intervention coverage. These programs, such as conditional cash transfers, also facilitate the uptake of specific interventions and behaviors such as immunizations, care seeking and nutrition interventions. They can also offer exceptional opportunities to help families partake in platforms and interventions to promote health, nutrition and ECD interventions.

Quality assessment of integrated programs was the central theme of two highly ranked research questions. A disproportionately high burden of mortality and morbidity is observed among poor, rural, and remote communities with limited access to quality health services [25]. Culturally–informed quality assessments are thus an important component of the monitoring and accountability agenda. Timely data on the quality and coverage of essential interventions is necessary for recognizing and reducing inequities, as well as understanding which programs are working and why. The identified research questions could feed into the Measuring Early Learning Quality and Outcomes project (MELQO) – convened by UNICEF, UNESCO, the World Bank and the Center for Universal Education at the Brookings Institute. This initiative is pulling together expertise on measurement from around the world to produce feasible, efficient and accurate approaches to the measurement of the quality of ECD programs and children’s learning environments.

In a recent CHNRI exercise focused solely on ECD, all top–ranked priorities related to the impact of implementation of interventions, with three priorities pertaining specifically to integration of ECD and maternal, newborn, and child health and nutrition services [26]. This finding underscores the importance of examining integrated implementation, and the present study expands on this broader ECD research agenda, indicating specific priority areas for accelerated research.

To our knowledge, there has been one other CHNRI that has explored implementation research priorities across the entire continuum of MNCAH&N and, like the present exercise, children were the most represented target population [18]. This focus on children could be because early childhood, particularly the first 1000 days, has the greatest potential for gains in health, growth and development [27]. However, what is novel in our research agenda compared with other implementation–focused CHNRIs is the mention of specific at–risk populations, such as refugees and migrant workers, children with nutritional and cognitive deficits, and mothers and children susceptible to violence. These populations are most vulnerable to adversity exposures and are therefore, most likely to benefit from increased access to ECD interventions via integrated delivery. Although high–risk populations were mentioned, we noted there were no questions targeting conflict and humanitarian settings – where disruptions in the health care infrastructure and exposure to stress, violence, food insecurity, and child neglect are greatest. The same applies to refugees and displaced populations, despite the latter now numbering in the millions, especially in the wake of the incessant conflict in the Middle East. This gap in the identified research priorities was potentially a result of limited expertise in this area among the respondents, and it must be acknowledged given the disproportionate burden of poor ECD in fragile states.

Investing in ECD is critical to achieving a number of SDGs [6], including SDG 3 [2], which aims to ensure health and well–being for all, and SDG 4, which aims to ensure inclusive and equitable quality education and promote life–long learning opportunities for all. The integration of delivery platforms presents an opportunity to maximize the impact of health and development interventions within sensitive windows across the life course, thereby reducing pervasive inequities that exist both within and across communities. The generated research agenda is expected to be a valuable tool that drives discussion on mainstreaming implementation research on integration of ECD interventions with the health and nutrition sectors. We call upon the global community of donors, researchers, policy–makers and program managers to advocate for the breakdown of siloes between health, nutrition, education, social protection and ECD initiatives; to support the translation of these recommendations into appropriate and transparent funding opportunities; and in doing so, to actively work toward enabling the sustainable and inclusive development of societies.