Fifty-five of the 196 IHR states parties have completed JEE assessments that were publically available for review as of March 4, 2018. Twenty-three (41.8%) of the analyzed countries are from the WHO Africa region, 13 from the Eastern Mediterranean (23.6%), eight from Europe (14.5%), five from Southeast Asia (9.1%), five (9.1%) from the Western Pacific, and one from the Americas (1.8%). Together, the 55-nation sample represents 25.8% (n = 1.88 billion) of the world population, and 69.1% (n = 38) are classified as low or lower-middle income by the World Bank (by comparison, 42.8% or n = 83 of all WHO-member countries are classified as lower or lower-middle income). The distribution of demographic, economic and health variables among these regions are reported in Table 1. Among our study sample, significant differences were observed across regions, with countries in AFRO reporting the poorest levels of performance across these various factors. Within each region, the countries that have undergone the JEE were observed, with two exceptions, to not have significant differences across these baseline variables when compared to those countries who have yet to undergo the assessment (Table 2).

Forty-three of the 48 (89.6%) total indicators included in the JEE assessment had a median score of <4 across the 50 countries analyzed. Analyzed by region, 46 (95.8%) indicators had a median score <4 in AFRO, 7 (14.6%) in PAHO+EURO, 37 (77.1%) in EMRO, 40 (83.3%) in SEARO, and 39 (81.3%) in WPRO (Figure 1).

Figure 1.  Distribution of median performance on 48 JEE Indicators across the 6 WHO regions.

The following indicators had the lowest median scores across the sample: antimicrobial stewardship activities, surveillance of infections caused by antimicrobial resistance (AMR), and the presence of biosecurity systems. The highest median indicator values were the presence of national vaccine access and delivery protocols, laboratory testing capabilities for priority diseases, and syndromic surveillance systems.

Statistically significant correlations were observed in the full sample between the JEE index score and life expectancy (Spearman’s ρ = 0.84, 95%CI = 0.76 to 0.90), DALYs lost to communicable diseases (ρ = -0.83, 95% CI = -0.90 to -0.75), and under-5 mortality rate per 1000 live births (ρ = -0.87, 95% = CI -0.92 to -0.78). In AFRO, the only statistically significant relationship with health outcomes was with the under-5 mortality rate (ρ = -0.59, 95% CI = -0.81 to -0.26) (Table 3 and Figure 2). In EMRO, JEE performance was significantly correlated with under-five mortality (ρ = -0.80, 95% CI = 0.97 to -0.25) and lost DALYs to communicable diseases (ρ = -0.65, 95% CI = -0.92 to -0.01).

Figure 2.  Scatter plots of various demographic characteristics with composite Joint External Evaluation (JEE) performance across various country groupings.

Among enabling factors, statistically significant correlations were observed within the overall sample for GDP per capita (ρ = 0.81, 95% CI = 0.68 to 0.89), public health expenditures as a percentage of GDP (ρ = 0.36, 95% = CI 0.10 to 0.58), and density of skilled health professionals per 10000 persons (ρ = 0.81, 95% = CI 0.71 to 0.89) (Table 3 and Figure 3). No enabling factor was significantly correlated with JEE score in AFRO, but GDP per capita was correlated with JEE performance in EMRO (ρ = 0.85, 95% CI = 0.61 to 0.99).

Figure 3.  Scatter plots of various enabling factors with composite Joint External Evaluation (JEE) performance across various country groupings.

We examined performance on the Joint External Evaluation, the WHO’s voluntary tool for assessing IHR core capacity and preparedness to potential public health threats, among the initial 55 countries that have undergone the assessment. These evaluations reveal large gaps in core public health capacity and preparedness across a range of indicators with a large majority of countries scoring less than 4 (indicating non-sustainable or undeveloped capacities) on a majority of indicators. Antimicrobial stewardship and surveillance for highly resistant pathogens were areas of particular concern though there were other areas with substantial challenges as well. There were large variations in performance by geography, with countries from WHO AFRO and SEARO regions scoring lowest on the JEE and those in PAHO and EURO scoring highest. Taken together, these findings are a critical wake-up call that while JEE assessments are critical, they are only a first step towards making the kinds of financial and technical investments necessary to help countries improve their public health capacities.

Most of the capacities measured by the indicators in the JEE represent essential public health functions, such as disease surveillance, laboratory capacity, and sufficient human resources. These combine to create an infrastructure which protects the health of a country’s population from threats such as communicable zoonotic diseases, food poisoning and waterborne illnesses, allowing nations to contain public health emergencies at their source. The poor performance on many of these indicators (scores of less than 4) suggest that countries are not only ill-prepared for cross-border outbreaks but are struggling to provide key public health services that are critical to keeping their populations healthy and safe. Without these core capacities in place, future outbreaks may become large-scale pandemics. Reasons for poor capacity are clearly multifactorial, as many countries exhibit broad-based weaknesses across a range of indicators. One fact is clear from these analyses: further investments are needed that specifically target preparedness and core public health functions represented by the JEE.

Yet, success in building any capacity can only be achieved with leadership and investment of resources, prioritization, and commitment by the specific countries themselves. Few countries completely lack capacity but appear to have some existing infrastructure and competence on which to build. Not surprisingly, resource-constrained regions of the world performed less well on the JEE than relatively affluent regions like PAHO, EURO and parts of EMRO and WPRO. Further, poor health outcomes, as measured by DALYs due to communicable disease, are more frequent in AFRO and SEARO, which have high burdens of disease while also exhibiting lower life expectancy and higher mortality rates. The observation that JEE performance seems to correlate well with these baseline health, demographic and economic data points suggests that this tool is accurately measuring its intended core public health-related capabilities.

Indeed, the JEE does appear to correlate well with the gross overview of the strength and performance of the core public health components of health systems as defined by life expectancy, under 5-year old mortality, and disability due to communicable disease. Moreover, performance on the JEE positively correlated with the presence of enabling resources for health (high levels of GDP per capita and skilled health professionals). However, less consistent correlations in region-specific analyses highlight a need to further validate the JEE tool across contexts and suggests value for context-specific prioritization of functions. Overall, our findings suggest the JEE tool accurately measures relevant health security capabilities, though longitudinal studies will be needed to confirm this. A slightly revised update of the tool was recently published by the WHO, but changes are small and thus unlikely to affect our results. So far, no JEE reports using the revised tool have been published.

There are important limitations to our work. First, JEE reports have only been conducted on 28.2% (n = 55) of the WHO Member States; further, each evaluated country has had only one assessment. Also, for some specific activities, such as prevention of antimicrobial drug resistance, interventions have started only relatively recently. Thus, the limitations of a cross-sectional analysis apply here and diminish the generalizability of our findings, since we expect variation in JEE performance over time to be more informational in guiding improvement. Further, there is asymmetric representation from each WHO region, which may have led to an inaccurate estimation of regional trends in JEE performance; however, with our adjustments, we believe this initial analysis will provide helpful guidance to policymakers as an interim summary on JEE results to-date.

Moreover, our analysis is not intended to serve as a definitive validation exercise, nor are we aiming to assess cause and effect between JEE scores and outcomes. Our relatively small data set handicapped our ability to pursue more advanced statistical models that may better identify possible predictors of JEE performance. As more analyses are performed, and particularly as countries go through additional iterations of the assessment over time, we believe pursuing such analyses in the future will be both helpful and appropriate.

In conclusion, core public health functions and preparedness remains broadly weak across a range of indicators among a moderately large sample of WHO member countries, though substantial regional variations exist. Capabilities in AMR and emergency response are among the lowest scoring, and our results suggest that regions with greater health resources exhibit improved performance on the JEE. We do find, however, that the JEE tool itself and the process it is applied by are important and seemingly effective in measuring the strength of the core public health components of health systems. As global generic indicators for these functions remain sparse and underutilized, the JEE process provides an additional opportunity to measure the impact of investing in essential public health functions.