The CHNRI method is the dominant method of health research priority setting [12]. It capitalises on experts’ wisdom to generate and score research priorities against pre-selected criteria, and is a transparent and systematic method to set research priorities [10]. Below, we have provided a succinct account of our methods; however, the detailed methodology the methods used in the ICMR-INCLEN national research priority setting exercise is published elsewhere [13].

A multidisciplinary and multi-stakeholder national steering group (NSG), comprised of policy makers, academics, program managers, civil society representatives, and donors was assembled to oversee the exercise. The NSG finalized 12 areas of concern (AOC) for the child health theme that together accounted for 75% of child mortality and morbidity. A child health research sub-committee (RSC) was set up and included 26 experts that further developed the technical details for this theme, including the iterative refinement and consolidation of the research options. Further information on their involvement is published elsewhere [13].

A nation-wide network of 186 experts was established [13]; they were located in the various states and union territories to capture diverse programmatic, socio-cultural, and geographic contexts within the country. These experts along with the 26 RSC members were invited to contribute to the first step of the CHNRI exercise which entailed crowd-sourcing research questions (RQs), refined into research ideas (RIs). These RIs were iteratively consolidated into research options (ROs). In the second step of crowd sourcing, ROs were scored using criteria pre-defined.

Five criteria were selected for scoring: (i) relevance; (ii) answerability; (iii) equity; (iv) innovation and investments in research resolving complex and refractory challenges; and, (v) investment in research. Experts were assigned to score two of the five criteria to reduce scorer fatigue [13]. Scorers were asked to score 0 if the research question does not satisfy the criterion, or 1, if the research question satisfied the criterion. Research Priority Score (RPS), which is the mean score across each criterion divided by 100, were calculated, and these scores could range from 0.00 to 1.00. Average Expert Agreement (AEA) were also computed for each research option. The AEA is calculated as follows:

In tandem with the scoring phase of the CHNRI exercise, a Larger Reference Group (LRG), which contained diverse stakeholder groups including politicians, policy makers, funding agencies, and senior researchers was created and tasked with weighting the criteria against one another. This exercise is described fully elsewhere [13]. The resulting weights were applied to each criterion prior to calculating the RPS, resulting in a weighted RPS (wRPS). The ROs were ranked in descending order of their wRPS. Finally, three of the authors (KW, AM, and HG) reviewed the ROs by their domains to classify them. Any disagreements were resolved by a senior researcher (NA).

India is a large country with diverse population and heterogeneous governance status. We, therefore presented national research priorities, as well as regional priorities. The states were grouped into the following regions: (i) EAG (Empowered Action Group) and North-Eastern (NE) states; (ii) Northern states and Union territories (UTs) (including West Bengal); and, (iii) Southern and Western states and UTs (Box 1).

Of the 212 experts requested to contribute RIs, 90 (participation rate: 42.5%; 75 from the nation-wide network and 15 from the RSC) contributed 596 RIs. These were refined onto 648 RQs and finally consolidated as 101 ROs. ROs spanned across the four domains of research (d⁴ domains: description, discovery, development, and delivery). The description domain included 32 ROs (31.7%), 27 ROs (26.7%) were within development and another 27 ROs (26.7%) within delivery and only 2 (2%) within the discovery domain. The remaining 16 (15.8%) were classified within multiple domains, the majority of which including delivery (n = 13) or development (n = 8).

The 101 ROs were scored by 233 experts; 69 belonged to the EAG & NE region, 68 to the Northern region and 96 to the Southern and Western region. The gender and age distribution of the scorers, nationally and by region, is provided in Table 1.

Table S1 in Online Supplementary Document shows the complete list of ranks, wRPS and AEA scores at the national and regional level for all 101 research priorities. Nationally, wRPS ranged from 0.923 to 0.512 and AEA ranged from 0.921 to 0.563.

The top 10 research priorities nationally, by domain, unweighted criterion scores, their wRPS and AEA are displayed in Table 2. All of the top 10 research priorities fell into either delivery (n = 5), development (n = 2) or a combination of delivery and development (n = 3). Nationally, the top research priorities were in the areas of cost-effectiveness, feasibility, barriers, and challenges in delivering interventions or programmes (numbers 1, 7, 9), monitoring and evaluation or surveillance (numbers 2, 8, 10), managing conditions through developing new technologies (numbers 3, 5, 6), and behaviour change (number 4).

Table 3, Table 4 and Table 5 display the top 10 research priorities for different regions of the country. Each table provides the RO list in descending order as per its priority score. The tables also provide the d⁴ domains, unweighted scores in each criterion, wRPS and AEA. The consistent feature was the emergence of implementation issues as the most important research options in the form of development and delivery with some variations across the regions. In North India and UTs, the top research priorities focused on better management of sickness, implementation of new point of care diagnostic tools (numbers 1, 2, 3, 4), monitoring and evaluation or surveillance (numbers 5, 9), program delivery and cost-effectiveness (numbers 7, 8, 10), and communications (number 6).

Among the top 10 research priorities for the Southern and Western States (Table 5) development of low cost, environment friendly, and innovative interventions for vector control (number 7) and community engagement (number 9) were the novel and contextually important priority research areas.

The scorers from EAG and North-Eastern states (Table 3) favoured policy analysis specific to the SDGs, improving administrative data quality for decision making and better accountability and issues related to antimicrobial resistance as the top research priorities.

Table 6 compares the rankings of the top 20 research priorities nationally with the regional rankings of those research priorities. There was divergence among priorities between the regions; 15 ROs each from North, and South & West regions appeared in the national list. EAG and NE states differed the most from the national list; the first priority of EAG and North-Eastern states did not appear in the national top 20 alongside 8 other such ROs. The Northern states and Union territories did not prioritise improving administrative data at the primary health centre (PHC) and district hospital level, examining barriers to achieving Indian Public Health Standards (IPHS) at primary and secondary level health facilities or community-based management of childhood morbidities generally, but rather prioritised research questions that were morbidity- or program-specific, with the exception of number 7. The Southern and Western states’ priorities were more aligned with the National priorities.

Although the research priorities themselves are quite divergent, the type of research being prioritised was similar between regions. Many of the top research priorities across regions and nationally prioritised research on program delivery, including impact evaluation and cost-effectiveness studies on existing or new programs, improving monitoring and evaluation of existing conditions and programs and improving the management of morbidities.

India is a vast and diverse country and so it might be inappropriate to consider single national-level research priorities, due to great levels of heterogeneity and inequity between states. Indeed, some authors have argued that reaching the MDGs was not possible due to this inequity [14]. Under-five mortality between states had an inverse relationship with economic transition status, and significant disparities existed in life expectancy [15]. The governance of health system also varied across the states [16]. Private health expenditure was high, and there have been disparities in States’ ability to finance health care. These were some of the factors that may have resulted in an imbalance on health spending between wealthier and poorer states, and management of the health system in general [15].

Because of this heterogeneity between health care access, governance, financing, and local socio-cultural practices, factoring context into this research priority setting exercise was of top importance in order to make this exercise as impactful as possible in achieving the SDGs. To account for the state-level heterogeneity, we aggregated research priorities for groups of states that were similar in economic transition level and governance of health systems. Similar to the recently published Million Deaths Study, Global Burden of Disease Study, and other analyses of disparities in service provision in India [2,4,5], the results of our research prioritisation exercise also demonstrated a need for regionally targeted programs.

The overall types of research being prioritised was similar across regions (ie, program delivery, monitoring and evaluation, and improved management of morbidities), but the content of the research options was different between regions. The EAG and North-Eastern states are among the least developed and was also the only region that prioritised a policy analysis reflecting the need for reviewing, refining and aligning programs and interventions to local requirements if they have to achieve SDGs in time. The other high priority RO from this region was the need for strengthening administrative data (number 2). Valid and high quality administrative data encourages timely decision making for rational and specific action in addition to bringing accountability across the health system [17,18]. The EAG-NE region and Northern states have high burden of malaria and hence prioritized ROs addressing point-of-care diagnostics, regimens for treating malaria, issues of resistance and its changing epidemiology. The Southern and Western states were relatively better governed and covered by health facilities and it appeared that they were now focussing on prevention; regional priorities related to vaccines introduction, safety monitoring and identifying strategies to reach the unreached segments of the populations [16].

Prioritising health research and aligning these with program interventions will be important to make progress to reach the SDGs. Indeed, the recent Global Burden of Disease Study for India found that not only there were differences between states at different levels of economic transition, states that were further along in economic transition had higher burden of non-communicable diseases in contrast to rest of the states with persistence of higher burden of communicable diseases. However, grouping states simply due to economic status might miss some nuances. For example, Odisha and Jharkhand had the highest DALYs due to diarrhoea but lowest due to LRTIs as compared to rest of India [4]. The results of this Global Burden of Disease study illustrated the importance of targeting health interventions and research priorities as much as possible.

In our exercise, researchers gave most ROs high scores against the ‘answerability’ and lower score to “innovation” criterion. The LRG on the other hand allocated greater weight to relevance and innovation. Thus, the LRG weights were impactful in discriminating between ROs and helped in deciding the final priority listing. The vast majority (>70%) of CHNRI exercises have not employed a LRG, which has the benefit of including wider stakeholder values in calculating research priorities [12,19]. Yoshida and colleagues posit that this is due to difficulty obtaining a representative sample of stakeholders [20]. Our LRG was composed of policy makers, representatives from funding agencies, senior researchers, and program managers from state and central government.

The EAG and North-Eastern states was the only region to prioritise conducting a policy analysis in-line with the SDGs in their top 10; this was their top priority. It was important to note that while this exercise was conducted thinking forward to the SDGs, the SDGs had not yet been released during the first round of the CHNRI (generating the ideas), and had been released in the middle of the second round (scoring the ideas). Therefore, while the exercise had the lens to look forward to the SDGs, it was not informed by the wider SDG agenda, which unlike the 8 MDGs, are much broader, and are comprised of 17 goals with 169 targets [21]. Many of the ideas prioritised in the current CHNRI exercise related to SDG 3.2, which asks to end preventable deaths children under 5 to at least 25 per 1000 live births [21]. This goal is similar to MDG 4, which focused on reducing child mortality and related interventions and monitoring, but misses the wider SDG agenda of child, economic, environmental, and societal well-being. SDG 3.2 is a crucial goal for India in obtaining the SDGs [22].

Moreover, one major cause of mortality was missing from the regional and national prioritised lists. The Million Deaths Study had demonstrated that India had shown accelerated declines in pneumonia and diarrhoea, though these were still major contributors to child deaths. However, deaths from injuries, including falls and drowning, have risen to the third most common cause of death in children, and is responsible for almost 70 000 deaths [2]. In an analysis including all ages, a separate paper showed that DALYs from injuries increased across all economic transition state groups in India [4]. Neither regions nor the overall country prioritised an intervention, program, or data collection mechanism for deaths due to injuries.

The d⁴ domains were independently assigned to every RO by three reviewers (KW, AM, and HG) and matched; disagreement was sorted by a senior researcher (NA) and through consensus with the reviewers. However, the domains overlapped and therefore, categorization of some of the research options was liable to subjective variation in interpretation of domains.

This research priority exercise mobilised hundreds of researchers, funders, program managers from central and state governments, and policy makers and was part of a larger research priority setting exercise for maternal, child, newborn health, and maternal and child nutrition. Simply having research priorities are not enough; they must be implemented. The ICMR launched a grants programme for Implementation Research on Maternal and Child Health in spring 2017 to pursue the research priorities identified by this umbrella exercise. Ultimately, we hope that the outcome of this exercise and the joint ICMR grant programme shall positively impact child health in India, and accelerate India’s journey towards achieving the SDGs.