Literature search method

A scoping review of current practices for measuring coverage of malaria interventions was conducted in 2018. It included English language literature published between January 1, 2015–June 1, 2018, identified through systematic searches of Pubmed, OvidSP (EMBASE & Global Health), and the Cochrane database of systematic reviews. Search results were reviewed to identify articles according to predefined criteria, primarily the inclusion of malaria intervention coverage estimates with a defined numerator and denominator. Secondary analyses, simulation studies, and reports of a solely qualitative nature were excluded. Malaria prevention and control interventions included ITN distribution, IRS, IPTp, MDA, and SMC. Malaria case management was excluded because coverage estimation relates to broader issues around population access to health services, the effectiveness of, access to, and quality of confirmatory diagnostic testing, and the effectiveness of surveillance systems to record malaria testing and treatment.

While data were extracted for all studies meeting the criteria above, only studies presenting intervention coverage estimates using program data or post-campaign surveys are included in the current article. Additional publications describing new developments in the use of programmatic data or denominator estimation in public health were identified from multiple sources, including PubMed searches, articles previously known to the author, and reference lists. Characteristics of all studies included in the scoping review are listed in Table S1 in the Online Supplementary Document, with a full description of the approaches to estimate coverage provided in Table S2 in the Online Supplementary Document. There is no formal assessment of study quality in line with current scoping review guidelines [5,6]; however, the advantages and disadvantages of different methodological approaches, in general, are discussed.

Selection of studies

A total of 658 articles were identified, 183 of which were potentially relevant after screening titles and abstracts. Two other studies that met the inclusion criteria were identified through the included studies, and one study that was not yet published was included. After full-text review, 35 articles met the inclusion criteria and 151 articles were excluded from the review. See Figure 1 for the flow diagram for selected and excluded studies.

Figure 1.  PRISMA flow diagram for studies assessing malaria intervention coverage 2015-2018.

Seasonal malaria chemoprevention

Seasonal malaria chemoprevention is designed to reduce the incidence of severe malaria in children living in areas with highly seasonal malaria transmission [7]. The World Health Organization (WHO) recommended indicator for monitoring SMC programs is the “proportion of children aged 3-59 months who received the full number of courses of SMC per transmission season”, using routine reporting system data [8]. While the recommended indicator assumes measurement of completed courses of SMC drugs in each cycle, the WHO's 2013 SMC field guide suggests instead defining coverage according to the first dose within each cycle [9]. The focus on recording the first SMC dose is a result of directly observed administration by the health care provider, with doses two and three given at home by the caregiver. An alternative approach to estimate SMC coverage is through post-campaign surveys, where a random selection of households are visited shortly after completion of the SMC campaign (ideally within one week) to measure reported participation in SMC, including completion of the full three day course [9].

Five studies reporting SMC coverage estimates, described in seven articles, were identified in the review [10-16]. Two studies included estimates derived from program data [12,14-16]. All five studies primarily relied on post-campaign surveys to assess SMC coverage, which used caregiver report of a child's participation in SMC as the numerator and the number of surveyed children eligible (based on age and residence) for the intervention as the denominator. All five studies additionally reported using programmatic tools, such as SMC cards, distributed to track participation of children in each round; however, data from these tools (retention of cards, coverage estimated from cards) was rarely reported [11-13].

Studies incompletely or inconsistently described how children meeting more specific ineligibility criteria, such as contraindications, were handled in the denominator. Two studies excluded children ineligible for SMC due to severe illness, known allergies to SMC drugs, or receipt of co-trimoxazole treatment, from the denominator [10,11]. Another study specified that children who were febrile on the day of the study were excluded from SMC and referred to clinics for malaria testing; however, the authors noted that community health workers did not consistently refer febrile children and withhold SMC [14]. The authors suggested an alternative definition of administrative coverage that would capture instances where febrile children were referred to facilities but found not to have malaria and consequently received SMC drugs at the facility, requiring modification of SMC cards to capture SMC given at clinics following negative malaria tests [14]. An additional challenge for denominators raised by authors regarded the potential exclusion of mobile populations from coverage surveys, such as those temporarily residing close to farmland for some or all of the SMC period [11-13].

Two studies triangulated results of coverage estimates from program data to estimates derived from post-campaign surveys [12,14-16]. One study in Burkina Faso used adjusted census data to estimate the eligible population for SMC; however, the adjustment method was not reported, and coverage regularly exceeded 100%; in contrast, the post-campaign surveys estimated coverage of 95% or lower depending on the SMC cycle [12]. In this context, it is difficult to determine if coverage exceeding 100% is a result of inaccurate denominator estimation or inclusion of children from outside the target geographical area or target age group in the numerator. Inclusion of children older than 59 months in post-campaign coverage surveys is suggested to estimate coverage among non-target children [11], and may also allow age heaping (whereby reported ages are rounded to attractive numbers) to be identified and addressed. Conversely, the second study, in Senegal, used Demographic Surveillance System (DSS) population estimates for the program data denominator and found that post-campaign surveys provided consistently higher estimates of coverage than program data. The authors hypothesized that denominator estimates from DSS data might have overestimated the target population, by including families who had temporarily migrated and were not present during the SMC campaign. However, DSS data are generally considered an up-to-date and reliable source of population denominators, though limited to the specific surveillance sites.

Mass drug administration

Mass drug administration for malaria involves the administration of a full therapeutic course of anti-malarial medicine to a defined population within a specified time period and geographic region, regardless of the presence of symptoms or infection [17]. The WHO's 2018 Malaria surveillance, monitoring & evaluation reference manual does not include any MDA coverage indicators [8]; however, WHO technical and operational guidance is available for organizing a successful MDA campaign and suggests two methods to measure coverage [17]. 'Distribution coverage' is defined as the proportion of the targeted population who received the first dose of treatment in the specific MDA round and is calculated using MDA program data for each MDA round. The WHO suggests conducting a household census before the MDA campaign to generate a denominator of people in the target area. However, in settings where a pre-distribution census is not feasible, WHO recommends using a post-MDA coverage survey, which will generate both a numerator (number of surveyed individuals reported to have taken MDA drugs) and a denominator (number of surveyed individuals age-eligible for inclusion in MDA campaign) from a representative sample of the population [17]. Women in the first trimester of pregnancy are excluded from MDA campaigns, generating additional challenges in identifying and estimating the denominator.

The review identified ten studies with coverage estimates for MDA strategies targeting administrative areas or specific households [18-28]. The reviewed articles involved two to three rounds of two- or three-day courses of anti-malarials, and all but one study [24] involved direct observation of at least the first dose of treatment. Studies used program records [20,22,23,27], post-campaign surveys [26] or a combination of program records and post-campaign surveys [18,19,21,24,26,28] to estimate coverage. One study compared coverage estimates using capture-recapture methods to those from program and survey data [28].

Most studies included in the review provided estimates of programmatic coverage as the proportion of the target population who received MDA. However, coverage terminology was not used consistently, and denominator estimation methods were rarely described in detail. None of the included studies used the term 'distribution coverage' as defined by WHO [17]. Only two articles used the terms 'operational coverage' and 'effective coverage' [23,25], and only one defined the terms clearly [25]. One study used the term 'program coverage,' equivalent to 'operational coverage' [28]. It was common to include the full target population in the denominator, and separately note the percentage of non-participants who met exclusion criteria (eg, women in their first trimester of pregnancy, infants less than six months of age and individuals who are seriously ill or have known allergies to the MDA drugs) [18-20,22,24]. Two studies additionally tracked coverage among mobile populations, such as returning residents, migrants and visitors [20,22].

Studies relied on data collected at different points of the MDA campaign to estimate the denominator, including household enumeration conducted before [18,20,22-24] or during drug distribution [19,27], with the timing and completeness of enumeration affecting estimates. Despite using household registration data from a mass ITN campaign conducted approximately six months earlier, one study found that the targeted population was underestimated in urban areas, where the population is less stable [23]. Another study initially estimated MDA coverage needs using a denominator based on a census of the target population conducted by program staff, which was verified with local leaders in all operational districts [24]. This process of validation with local leaders yielded a markedly higher estimated population size than expected (184% increase). The authors suggest this may be due to overcrowding and multiple families sharing the same household. Despite the increase in the estimated population, the denominator was still underestimated due to the exclusion of several small minority communities that were not discovered until the second round of MDA. These communities were not initially identified as part of the enumeration area by the local leaders.

MDA in epidemic or complex emergency settings generally relies on existing population denominators, for example, from other recent health interventions [23,24]. In Sierra Leone, the target population was estimated from ITN mass campaign records from six months prior, with the numerator taken from program records (number of MDA courses distributed) to estimate coverage [23]. Estimates were triangulated with post-MDA campaign surveys conducted by an independent monitor [23]. In Liberia, where fixed-point distribution was used to give vouchers that could be exchanged for MDA drugs for two adults and five children, estimating coverage was challenging since neither the numerator (number of MDA treatments provided) nor denominator (target population at risk) could be readily estimated [24]. However, the authors used the program data to determine how many vouchers were distributed and how many were traded in for medication, by round, providing an MDA acceptance rate estimate.

Intermittent preventive treatment in pregnancy

Intermittent preventive treatment in pregnancy with sulfadoxine-pyrimethamine (SP) is recommended in areas of moderate to high malaria transmission in sub-Saharan Africa. The WHO recommends using routine health information systems or household surveys [8] to monitor the proportion of pregnant women receiving each dose of SP, including the updated guidance regarding third and fourth doses [29]. When surveys are used to estimate IPTp coverage, numerators and denominators are defined from the surveyed population, but questions relating to IPTp are usually restricted to women who have had a live birth. When coverage is estimated using routine program data, the numerator for IPTp must come from ANC records at health facilities (including any pregnancies that did not result in live birth), and the denominator is likely to be estimated from census projections. The denominator for the number of women eligible for IPTp is also influenced by human immunodeficiency virus (HIV) infection rates since SP is contraindicated in women receiving co-trimoxazole prophylaxis [29]. As a consequence of these differences, IPTp coverage estimates from health facility records are most useful for process evaluation purposes, to understand if uptake of IPTp at ANC is adequate since the denominator is restricted to those women attending ANC. Assessments of IPTp impact generally require an understanding of IPTp coverage among the eligible population, incorporating both ANC access and IPTp uptake among ANC attendees.

In the review, 16 studies assessed coverage of IPTp interventions [30-46], but only two relied wholly on program monitoring data for coverage estimates. One study used program monitoring data from a pilot intervention of community- and facility-based delivery of IPTp [36], while another relied on ANC logbooks and monthly district reports [33]. Nine studies collected data on IPTp coverage using surveys, mostly facility-based, of which six studies confirmed respondent self-report via the use of routine data, including hospital records, logbooks, or ANC cards [30,35,37,39-41].

Coverage terminology was a challenge for IPTp studies identified in the review, with a range of indicators used to reflect the different number of rounds of IPTp that could be received. While most IPTp coverage estimates used ANC attendance as a denominator, only one study used the number of pregnant women attending their first ANC visit as the denominator, with the number of women receiving their first, and second, dose of IPTp as the numerators [33]. In general, the use of ANC attendance as a denominator results in an over-estimate of coverage if some women do not attend ANC at all or seek ANC services outside the public health sector. An additional indicator describing the number of missed opportunities for IPTp could be useful in settings where barriers to IPTp are at the provider-level, rather than as a result of low ANC attendance [42]. While IPTp with SP is not recommended for women who are taking co-trimoxazole prophylaxis, few studies reported whether these women had been excluded from the denominator used to estimate IPTp coverage [30,35]. Survey-based IPTp coverage estimates use women's recall of taking SP during an ANC visit; however, women may not be told the name of drugs prescribed during ANC visits, and consequently may not be aware if they receive IPTp [43]. Coverage indicators derived from both survey and program data should be sensitive to the local variations in service delivery across Africa, including the expansion of IPTp provision through community health workers [43].

Two studies used community and facility-based designs to estimate IPTp coverage. A trial of IPTp delivery methods in Nigeria used two different coverage estimation methods [36]. The numerator was derived from study-specific outcome forms completed by community and facility-based staff to capture SP doses, then aggregated by facility within District Health Information Software 2 (DHIS2). A census of the intervention area was conducted for the intervention arm denominator, including the number of eligible and ineligible pregnant women, though operational issues prevented complete enumeration of all intervention areas. In contrast, control arm areas estimated the denominator using official population estimates (Sokoto State Government), assuming 5% of the population was pregnant, and that 94% of pregnant women were eligible for the intervention, prorated over the eight months of the project. The authors noted discrepancies between official census and intervention area enumeration estimates that suggested the official census data may under-estimate the number of eligible women. A second study, which identified eligible survey respondents from a health and demographic surveillance site, suggested that IPTp coverage may have been overestimated due to underrepresentation of the rural population in the study district, and that rural women are less likely to receive IPTp [30].

Insecticide-treated net

Ensuring universal vector control coverage for all people at risk of malaria is a pillar of the Global Technical Strategy (GTS) for malaria; and includes 100% access to, and use of, either IRS or ITNs by populations at risk of malaria [47]. The GTS minimal recommended indicator for vector control intervention outcomes is the proportion of the population at risk of malaria sleeping under an ITN or living in a house sprayed by IRS in the previous 12 months [8]. This indicator can be measured independently with a household survey, but is preferably assessed in combination with routine monitoring data [48]. Other indicators recommended to be sourced from routine monitoring data include the proportion of the population at risk potentially covered by distributed ITNs; and the proportion of targeted risk group receiving ITNs.

Household surveys continue to be the most common method for estimating ITN coverage, and there are a range of recommended ITN ownership and use indicators [4]. However, recent evidence suggests that population access to ITNs–where the denominator is people, not households–is a better indicator of universal coverage than the proportion of households owning at least one ITN for every two people [49].

The scoping review identified eighty studies with defined coverage estimates for ITN interventions, mostly derived from cross-sectional household surveys. A few studies used surveys administered through other venues, including health facilities [50], plantations [51], and schools [52]. Only eight studies relied on program monitoring or routine surveillance data [33,53-59].

Most studies using program data reported on the proportion of the population potentially protected by ITNs using NMP reports on net distribution to calculate the numerator and population projections for the denominator [53,56,58,59], though few studies explicitly described the assumptions used for estimating either the numerator or denominator. Among those that provided full details, one study in Ghana calculated the proportion of the population (all ages) potentially protected by long lasting insecticidal nets (LLIN) each year from 2005 to 2015 using district records and assuming that each LLIN distributed covered 1.8 persons and lasted three years. The denominator was calculated using the 2010 Ghanaian census and the United Nations growth rate for Ghana [53].

One study in Kenya conducted school-based post-campaign surveys in order to quickly obtain ITN ownership and use coverage estimates, with the limitation that responses could not be verified by observation [52]. Lot quality assurance sampling (LQAS) was employed by a handful of post-campaign surveys [60-63]. A small study validating a rapid assessment tool for malaria prevention used LQAS to identify areas that were not reaching targets of intervention coverage and use [61]. Another applied LQAS methods to a secondary analysis of a nationally representative household survey data set to estimate community-level coverage by considering each cluster as a lot, and assigning a pass-fail threshold for different ITN coverage targets, using a 20%-point margin between target and minimally acceptable result [63]. LQAS was also used to determine whether net ownership and use thresholds were met after a national ITN distribution campaign in Mozambique, with findings similar to estimates generated by the NetCalc tool [60]. NetCalc uses programmatic data describing the number of nets distributed, estimated durability of nets, pre-distribution net coverage, and estimates of population and household size to generate predictions of expected net coverage.

Alternative approaches to generate rapid and low-cost malaria intervention coverage estimates include surveys among easy to access groups (EAGs) and mobile phone surveys. EAGs are defined as representative subsets of the population or at-risk groups that assemble at easily accessible locations such as schools or health facilities, or during public health intervention activities such as catch-up vaccination campaigns [64]. The utility of EAGs for monitoring and evaluation has not been fully explored, though a review found estimates from EAGs commonly over-estimated population coverage values, but with varying degrees of accuracy [64]. Concerns relating to the representativeness of EAG samples could be alleviated by inclusion of a small calibration survey, to generate a correction value that can be applied to EAG sites. Additional hybrid sampling approaches and strategies to minimize bias from the use of EAG data are presented by Sesay et al. [64]. Mobile phone surveys offer another survey approach that is faster and lower cost than household-based surveys. A pilot of phone-based surveys in Tanzania to assess ITN coverage suggests that by use of non-response adjustments such as raking or post-stratification, representative estimates of ITN coverage can be achieved (Worges et al, in preparation).

Indoor residual spraying

The leading coverage indicator reported by IRS programs is defined as the proportion of targeted or found households or structures which were successfully sprayed. This a clear example of a coverage indicator that may be useful for monitoring program implementation but cannot be easily extrapolated to understand IRS among the population at risk and consequently to evaluate the impact of IRS. Both the numerator and denominator for this indicator are generated from spray operations reporting [65]. Additional strategies described by the WHO IRS operational manual include conducting baseline geographic reconnaissance and census to identify households in the targeted areas, and use of house spray cards to track participation over multiple rounds [65]. The manual also recommends a separate estimate of IRS coverage through post-campaign surveys, where both the numerator and denominator are generated by the survey [48].

Eighteen studies reporting IRS coverage were identified in the review [27,58,59,66-80]. Coverage estimates were more commonly obtained through post-campaign household surveys [27,66,69-71,74,76,79,80] than routine IRS program records [58,59,68,72,73,77] or study records [75]. Two studies reported coverage from both household surveys and program records [67,78]. Three studies piloted innovative methods for improving IRS implementation, including mobile phone-based household sensitization techniques [73], the use of satellite enumeration and spatial aids to assist spray teams [66], and integrating IRS within a community-based rural health services program [77]. One study assessed the impact of multiple targeted interventions, including IRS, in transmission foci [27].

The most commonly reported indicator using program data was the proportion of enumerated structures sprayed (per spray cycle) using spray campaign data [59,67,68,72,73,77,78]. None of the reviewed studies reported the proportion of population at risk sleeping under an ITN or living in a house sprayed by IRS in the previous 12 months, potentially due to the difficulties in translating program data to household-level ownership measures [81]. Two studies reported estimates for the proportion of the population at risk protected by IRS, using routine monitoring data [59,77].

Few of these studies discussed the representativeness of estimates or inclusion of migrant and mobile populations. Survey response rates were rarely presented or discussed, although one study reported that almost one-third of households were not available to interview, and of those reached, only 62%-80% were willing to be interviewed [66]. Nwe et al. (2017) reported IRS coverage in Myanmar based on NMP district estimates for 2010-2014 and discussed the difficulties of accessing relevant data on migrant and mobile populations [58]. Another study noted that households were unable to be sprayed due to residents working in their fields during the spray campaign, resulting in particularly low coverage [73].

Two studies used multiple data sources to triangulate coverage estimates obtained during routine program implementation. A study in Uganda triangulated data from cross-sectional household surveys, routine cohort assessments, and monitoring data from the NMP, finding high IRS coverage by administrative and cohort data, but lower coverage from household survey data [78]. However, the household survey was conducted concurrently with IRS implementation, so surveyed households may not have yet been reached by IRS spray teams, emphasizing the importance of timeliness for drawing estimates. Citing concerns that administrative coverage estimates may underestimate the actual coverage, a study in Namibia compared IRS coverage reported using routine NMP data against household survey data [67]. However, the estimates used were not directly comparable as the program data considered structures as the sprayed unit, while the survey considered households as the sprayed unit.

IRS is often targeted based on administrative areas or may even be targeted according to environmental or ecological zones, the boundaries of which may not be apparent to spray teams in the field. A field-based enumeration approach has traditionally been used to guide indoor residual spray operations, which is often completed without validation that the entire targeted area was indeed enumerated [82]. Bridges et al. (2018) assessed the accuracy of satellite-based enumeration to identify sprayable structures in Zambia [66]. The sensitivity of satellite enumeration was assessed by dividing the total houses enumerated from satellite imagery by the total houses found during field-based enumeration. The study found that satellite enumeration underestimated the number of structures in the sampled areas. However, with an overall sensitivity of 94%, the authors concluded that satellite enumeration is an accurate and more cost-effective and scalable alternative to field-based enumeration methods for planning and monitoring IRS campaigns, with potential application to other interventions (ITNs, MDA, vaccination campaigns).

Innovations in estimation of population denominators

Increased availability of satellite-derived imagery, use of global-positioning technology in field activities, and developments in statistical methodology and computing power have contributed to innovations in estimation of population denominators for settings where reliable and contemporary population census data are not available [83].

Top-down statistical methods allow census data to be disaggregated to a higher resolution through approaches such as dasymetric mapping [83,84], and small area estimation [85]. Model-based approaches to re-weight census data across census units have been used to generate high-resolution maps of estimated population distribution, with resulting gridded population data freely available from WorldPop [86-88]. Data from large-scale surveys and other sources have also been used by the Malaria Atlas Project (MAP) to generate high-resolution predicted surfaces for indicators such as accessibility, household construction, and relative vector abundance [89-91], which can be useful in defining the population in need of various malaria interventions. However, the reliability of top-down estimation methods is dependent on the availability of recent and high-quality input data and may not be appropriate in areas with high population mobility.

Bottom-up methods describe approaches using high-resolution satellite data to define settlement patterns or individual households. Both manual and automated methods to enumerate households have been piloted [66,82,92,93]. These methods require obtaining high or very-high resolution satellite imagery and estimates of mean household occupancy in order to generate population estimates [94]. The use of satellite imagery to estimate denominators has particular utility for IRS campaigns, but is limited in utility in urban settings with multi-level buildings or complex roof patterns, or where houses are obscured by tree canopy or cloud cover in satellite imagery. Bottom-up population estimates have been recommend to complement census and other enumeration activities, emphasizing that methods used to generate estimates should be transparent, and the importance of engaging relevant stakeholders to minimize political sensitivities related to population estimates [83].

Capture-recapture methods, initially developed by ecologists to estimate population sizes, have potential utility in estimating denominators where repeat samples of the same population are available. Finn et al. used a capture-recapture approach to estimate the total number of households that should have been visited by intervention teams during MDA activities in Zambia [28]. The authors matched two independent lists of people, one list from the MDA program data and a second list from a cross-sectional survey conducted in the intervention area, and estimated the total households in the targeted area using the Schnabel estimation method [28]. Using the estimated total households as the denominator yielded similar epidemiologic and household coverage estimates to the household post-campaign survey and a satellite enumeration method, but all were lower than estimates derived solely from MDA program data. Where individual-level data are captured in registers during successive rounds of intervention delivery, these capture-recapture estimation techniques offer another alternative to estimating the true denominator population.

Researchers have begun to produce dynamic estimates of population denominators, incorporating information about internal and international migration patterns, or seasonal movements of populations for agricultural and other reasons [95,96]. The use of anonymous and aggregated mobile data can assist in describing population distribution and movement [96,97]. Seasonally-appropriate denominators may be particularly useful for planning and estimating coverage of interventions such as SMC, vaccination campaigns and net distributions in settings with temporally-dynamic populations.

Gravity models have potential utility in estimating health facility catchment populations by incorporating information about estimated population distribution, distance to facilities, and facilities' relative attractiveness [98,99]. Modified gravity models have been used to estimate facility catchment populations in Haiti, Botswana, and Mozambique (E. Cameron, personal communication).