A total of 69 experts participated independently in the voluntary scoring exercise (a 19.8% response rate). Scoring took place over three months. The respondents consisted of 41 (59%) researchers/scientists, 5 programme managers, 3 in the field of public health (self-described), 2 working in policy, 7 working in other fields and 11 (16%) were left blank. Among the respondents, 54% were female, 39% were male (7% non-disclosed), 55% were from HICs and 43% were from LMICs (with one left undisclosed). The final results of the scoring process (top 20 and bottom 10) are shown in Table 1.

The final RPSs for the 54 research options ranged from 52.7 to 87.3, with a median score of 75.2. The average expert agreement scores ranged from 0.46 to 0.78, with a median score of 0.60. Consistent with previous CHNRI exercises, the average expert agreement was significantly and positively correlated (although weak to moderate) with the RPS (r = 0.386, P < 0.01). Mean scores for each criterion were highly inter-correlated (Table 2). Research options were organised by six thematic areas or goals:

The top three ranked research options were (Table 3):

A diverse range of global health sciences experts with knowledge and experience in the field engaged in the well-established CHNRI methodology to identify research priorities for improving ECD to 2025. The systematic ranking of research options against predetermined criteria exposes the strengths and weaknesses of the research options and offers greater replicability and transparency than other priority setting methods [33]. The results of this process highlight that experts’ priorities for future research focus on testing the best models of service and support to parents to enable them to provide nurturing care, as well as research on how to train health workers and non-specialists in the effective implementation of interventions to improve ECD. Sixty-nine experts eventually undertook scoring for this priority setting exercise. The more experts who agree to participate in the scoring, the more reliable the outcomes of the research priority setting exercise [30]. The range of scores (both RPS and AEA) were consistent with previous CHNRI exercises. The RPS range shows significant variation, indicating that the methodology discriminated well among the competing research options.

Three of the six thematic goals of the present priority setting centered on interventions (ie, improving impact, implementation of interventions and improving the integration of interventions). These three goals have the highest average RPSs. In order to achieve higher coverage through sustainable high quality interventions to improve ECD i with equitable reach, interventions should be integrated into health, nutrition, education and child protection sectors [7,34]. This is in line with the Nurturing Care Framework (NCF) guiding principles of a whole of-society approach including all sectors [14]. However, there is limited knowledge on how to best integrate programmes to improve ECD [35,36]. The NCF offers an important framework to help move this forward. In addition, the Lancet Commission on Child Health and Well-being aims to place the child at the centre of all country actions and will stress how attempts to cultivate healthy productive societies must begin with investment in children [37].

Another key finding of this exercise was that simply increasing access to programmes in the early years is not sufficient unless programme quality is also improved [38,39]. The theme with the highest rank (improving the impact of interventions) is specifically aimed at quality assurance though testing and implementing best models. This is in line with the current global focus on the importance of quality at the level of programmes as well as at the level of the health system [40,41].

The second highest scoring option “Can community health workers/paraprofessionals be trained to deliver ECD interventions effectively?” addresses human resource skill and capacity, and is a reflection of the global shortage (but especially in LMICs) of skilled health workers to implement interventions to improve ECD. This finding is consistent with the barriers to scale up in ECD research identified by Richter et al. [7] and also with the priority setting exercise of Sharma and colleagues [8], which lends further credibility to the results of the present priority setting exercise. Seven of the 54 research priorities focused on the financial aspects of programmes and interventions (most of these came from the sixth goal: increasing the understanding of health economics and social protection strategies) although these research priority options were not highly ranked [7], and given the range of experts it was somewhat surprising that these research options were not ranked higher. It should be noted however, that the difference in score between the highest scoring research option and the first finance-related question (score of 79.85 and a rank of 16) was only 7.5 percentage points. The aim of the CHNRI exercise is to produce a transparent account of the scoring (the collective thinking) of a range of experts, that then permits funding agencies to make decisions with some level of data and ability to understand where one option stands in comparison to others. An agency might then decide to fund options 5 and 16, for example, based on their priorities and having seen a small difference between the top 5 ranked options and those in places 10-16.

In order to compare responses by sub-groups, comparative analyses were undertaken by geographic location, sex and by occupation (ie, academic vs non-academic). There was almost no agreement between experts from LMIC (44%) and experts from HIC (56%). This is perhaps unsurprising given the vast variation in types of programmes to improve ECD, sectors, implementers and target populations globally [38]. Experts in the LMIC group gave significantly higher scores overall than experts from HIC. The top three research priority options from the LMIC experts came from the third thematic goal of improving the impact of interventions, whereas the top three research priority options from the HIC experts came from different goals: improving the integration of interventions and increasing the understanding of health economics and social protection strategies, as well as improving the impact of interventions. Experts from LMICs were in agreement with HIC respondents about the need to prioritise impact with high quality interventions [38].

Female (58%) experts gave slightly higher scores than their male (42%) counterparts. There was moderate agreement between female and male experts. The top three research priority options from the female and male experts were from the goals of improving ECD interventions. The academic expert group (ie, working in research; 59%) gave a slightly larger range of scores compared to the non-academic expert group (ie, programme management, policy, health practitioner, other or those left blank; 41%). In addition, there was little specific agreement between academic experts and non-academic experts. The highest rated research priority from the academic expert group came from the fifth thematic goal of expanding the integration and coordination of programmes targeting the early childhood period, and the second highest came from the fourth thematic goal of enhancing the implementation of interventions. The top three research priorities from the non-academic group of experts came from the third research priority thematic goal of improving the impact of interventions. In general, however, the top three thematic goals as rated by the academics and non-academics came from a set of goals that centred on improving interventions (ie, either improving impact, implementation or integration). Ensuring high quality implementation of interventions is a particular concern currently in global health [38,39].

In this priority setting exercise the questions that were proposed varied from the general (the highest ranked option: child development packages focusing on nurturing care to improve child cognitive development) to the specific (how sleep promotion can improve neurocognitive development, Rank 51). This may in part be due to the nature of the respondents, but could also be a function of the nature of the methodology – in particular the two criteria of ‘impact’ and ‘effect on equity.’ General questions, with the promise of greater impact and for reducing inequity are likely to be ranked higher.

The CHNRI methodology has a number of advantages over other priority setting approaches [30]. The systematic listing and scoring of research options limits the influence of personal bias, while there is transparency in the input and contribution of individuals involved in scoring research priorities. The method also prevents one or a small group of individuals from dominating the process, while also providing a quantitative measure. The CHNRI methodology also has the added advantage of including non-technical stakeholders, and different domains of research can be simultaneously evaluated using the same set of criteria [30]. Strengths of our priority setting exercise include using a proven methodology, a sufficient number of experts with an acceptable gender balance, geographical spread and occupational balance.

Limitations to the approach undertaken in this study relate to the validity (it is not yet possible to verify the validity of the rankings) [42] of the CHNRI approach overall, and the potential sampling biases and high non-response rate [32,42]. While 74 people participated in generating the research options that were ultimately rated, it is unattainable in such an exercise to create a set of research options that comprehensively covers all possible research options. As a result, both the research options and the ratings generated are likely to replicate biases of the participating experts. In addition, participants not fluent in English would have limited participation in such an exercise. It is conceivable that the scores of experts who responded could be systematically different from those who did not respond [32]. In addition, the experts who did respond were primarily from the health sciences. The utility of future priority setting exercises may be improved through research to explore how the variability of profession among the raters in a CHNRI exercise affects both the research options provided as well as the final scoring. Different strategies to recruit a more diverse panel of experts need to be considered and put in place in future exercises.

The low response rate likely was a result of the time consuming nature of the exercise, but our response rate, while lower, was not markedly different from other similar exercises [21,26,43]. Dean and colleagues [43] conducted a priority setting exercise on preconception care in LMICs to reduce maternal and child mortality and morbidity with a 37% response rate, while George et al. [44] had a response rate of 36% in a priority setting exercise on reducing preterm births and stillbirths. However, the CHNRI methodology attempts to account for this based on the theory of the wisdom of crowds that suggests that approximately 24 scorers are needed in order to cancel out personal biases and judgements and to arrive at the collective wisdom of the group [31].

Finally, in the light of current global priorities, a number of key areas of research are notable for their absence in this exercise. Despite early childhood care being a key global issue there was only one research option on childcare (Rank 13: approaches to improve early childhood care). Equally surprising is that there were no questions (to be scored) on the delivery or implementation of interventions in humanitarian settings or fragile states. Given the current global crises in terms of children in humanitarian settings and future concerns in this regard, this is a significant limitation of this exercise. Also, current initiatives to develop population-based, cross-country measurement tools of child development did not feature in the research priorities of this group.

Despite the above limitations, this study was successful in producing research options from 74 experts, and scores from 69 global experts. In addition, this study demonstrates a need to invest in research that focuses on improving the impact, implementation and integration of interventions to improve ECD.

This research priority setting exercise provides a significant contribution to establishing research priorities in the field of ECD to 2025. International organisations, national governments, research institutes, and donors are encouraged to consider the findings of this exercise in order to address key gaps in our knowledge and enhance the nurturing care for early childhood development agenda and the achievement of the SDGs. Future exercises should ensure a more diverse panel of experts.