Generating support for NEP among senior stakeholders and partner institutions

In the project planning phase, IIP-JHU identified key in-country partners to engage given the governance structures they represented and given their personal interest in program evaluation. Senior IIP-JHU staff met personally with senior in-country stakeholders – including leaders of the Malawi National Statistics Office; the Malian Center for Research, Studies and Documentation for Child Survival; the Mozambican National Institute of Health; and the Tanzanian National Bureau of Statistics – to assess their interest in the initiative, discuss the project vision, and identify evaluation priorities. These discussions confirmed a desire for capacity building in evaluation and allowed IIP-JHU to identify early champions of the NEP. They also allowed in-country partners to participate in establishing NEP structures locally by determining whether specific committees were needed and advising on their membership. However, individual-level support did not translate automatically into institutional support, and it was not always sustained over time. Maintaining the support of early champions required continued effort, eg, seeking their strategic input throughout project implementation.

Engaging senior decision makers in the NEP approach

After the planning phase, IIP-JHU sought to formalize senior stakeholder engagement in NEP by establishing a High-Level Advisory or Steering Committee (SC) in each country. The SC is led by government leaders, with participation by representatives of MNCH&N programs and institutions responsible for statistics, planning, and research. Beyond general oversight of NEP implementation, the SC is responsible for identifying priority evaluation questions relevant to the country’s MNCH&N needs; receiving and providing feedback on NEP analyses pertaining to those priority questions; and applying NEP findings to inform decision making. By the midterm evaluation, SCs in all four countries had met at least twice annually, bringing together stakeholders from a range of ministries that did not typically work together around important MNCH&N issues. However, the degree to which SCs had articulated clear evaluation priorities or decisions based on NEP findings varied across the four countries. One barrier was the logistical complexity of convening high-level stakeholders, which limited the frequency of SC meetings, making it difficult to keep members engaged over long periods. Local NEP teams adapted to this challenge by sending out routine project updates and in some cases by creating sub-groups of SC members who could meet more frequently to facilitate decision making of the broader committee.

Establishing local structures and recruiting government staff

Other components of the NEP structure were designed so that local public-sector institutions played a central role in project implementation. Each NEP has a Home Institution (HI) – a national statistics office or public health research institute – that is responsible for day-to-day project management. HIs receive funding for staff and project activities, including workshops and dissemination events. Having respected governmental institutions take on this leadership role lent credibility to the project and helped to frame the NEP as a national resource. However, by the midterm evaluation, some of the challenges often associated with partnering with government institutions had become apparent. Most of the HIs had limited personnel and frequent turnover, with existing staff stretched across multiple projects. Contributing salary support for HI staff mitigated the challenge of limited human resources to some extent, as did communicating regularly with HI directors about the importance of protecting project staff’s time.

To facilitate collaboration between the US-based team and the HIs, IIP-JHU recruited Resident Advisors (RAs) who were nationals of each country. RAs are members of the IIP-JHU team who work closely with HI staff (three of the four RAs work from HI offices). As mid-career professionals actively engaged in the MNCH&N sector in their respective countries, all four RAs had an advanced understanding of local stakeholders’ interests, which helped external project staff navigate local politics. However, the degree to which each RA was perceived as being affiliated with the HI rather than IIP-JHU varied across the countries, and this occasionally created tensions for the RAs. The RAs also had limited power to impact bureaucratic processes, as they were outside government. The midterm evaluation highlighted the importance of being realistic about the power RAs wielded and of making sure the US-based team provided them adequate support.

IIP-JHU also worked with local partners to establish a Technical Task Team (TTT) at the center of each NEP structure, composed of staff from the HIs and the M&E units of stakeholder institutions. These groups are the target participants of NEP capacity building activities. Through workshops, small-group working sessions, and intensive mentoring, TTT members complete multiple “cycles” of data mapping and quality assessment, statistical analysis, and communication of results to respond to evaluation questions approved by the SCs. Though capacity building constituted a distinct set of activities within the NEP TOC, this aspect of the project facilitated local project ownership in several ways. Allocating resources to strengthening capacity of local staff signaled a commitment from IIP-JHU to support in-country partners in making their own evidence-based programming and planning decisions. Furthermore, TTT members developed effective collaborative relationships with each other; over time, most country teams worked more autonomously of IIP-JHU to complete analyses. Relationships built between TTT members also facilitated access to data and areas of expertise outside those of IIP-JHU, further enriching NEP as a national M&E platform. Despite these successes, consistency in participation throughout the learning cycles varied across the four countries, in some cases limiting the effectiveness of capacity building efforts. Planning activities more in advance and with greater frequency, eg, through working sessions between formal workshops, helped increase TTT member participation, especially when the sessions were coordinated autonomously by local NEP team members.

.  Photo: Project implementers and members of the Mali Technical Task Team at an NEP workshop (from the NEP collection, used with permission)

Implementing NEP around the information needs of government stakeholders

Early in project implementation, senior stakeholders had difficulty grasping NEP’s value in concrete terms and did not initially express a demand for analyses. Therefore, for the first cycle, IIP-JHU proposed standard evaluation questions that would be useful to policymakers in the short term. NEP implementation began as MDGs were ending and as new planning cycles were beginning, so IIP-JHU proposed prospective program planning questions related to meeting national targets or retrospective questions about progress towards MDGs. After obtaining approval from each SC, all country teams analyzed coverage, mortality, and nutrition data from household surveys using the Lives Saved Tool to answer these questions [9]. Once SC members were presented with specific results, they understood more clearly what kinds of products could be generated through NEP. Results were used by government stakeholders in most settings, eg, to prioritize interventions in national health strategies, and dissemination of results was followed by requests for additional analyses. However, after this “quick win,” a tension emerged between satisfying government stakeholders’ needs and fulfilling project deliverables related to developing the NEP. For subsequent analyses, a less standardized approach was used across the four countries. In some cases, this allowed NEP analyses to be more responsive to stakeholder interests (as NEP is intended to function), but it also made it difficult to incorporate certain technical tools and analytical methods that IIP-JHU had committed to testing into country work.