Study design

We undertook a phased scoping review including a landscape scan of implementation activities and a systematic keyword search of academic databases, guided by interviews with experts and practitioners in the field and an emergent theoretical framework. The scoping review methodology is increasingly used for mapping areas that are nascent or widely scattered [24], where conventional searches of academic databases are less likely to be fruitful. This approach can be used to understand key concepts, theories and sources of evidence as a means of guiding new innovations, empirical research or systematic reviews, and informing policymakers. Scoping reviews typically do not involve critical appraisal of study methodology or detailed extraction of outcomes data, since they are chiefly concerned with mapping the landscape of evidence rather than establishing the effectiveness of particular interventions [24,25]. Table 2 summarizes the differences between scoping reviews and comprehensive systematic reviews.

As outlined in Box 1 , we performed all of the activities recommended in Arksey and O’Malley’s widely cited scoping review framework [25], as well as a landscape scan of implementation activities and consultation with experts, which are typically regarded as optional. We summarize each of these phases below and discuss our methodology more exhaustively in the complete version of the report on which this article is based [26].

Mapping concepts and refining the research questions

The project began with a broad remit to review the evidence on innovative uses of mobile technology for strengthening “leadership, management and governance” in the health sectors of low– and middle–income countries in line with the topic areas of the funding scheme. In order to better refine the scope and focus, and avoid duplication, we began by examining existing reviews and commentary in the field, to differentiate the above three sub–topics and determine where the important knowledge gaps lie. This revealed an important gap in the literature concerning uses of digital technology for health sector governance, in contrast to a more extensive literature related to health care management and leadership issues, specifically the “good governance” agenda described in our introduction and in Table 1 .

Landscape scan of implementation activity

Based on the above, and informal discussions with experts known to our team, we determined that ICT for health governance is an active area of applied activity, although somewhat under–researched. For this reason, we began by seeking case reports to better understand the nature of projects in this area, beginning with those we were familiar with and snowballing via web links, tracing the work of key organizations and funding streams, and undertaking targeted keyword searches in Google and Google Scholar. Case reports included “grey literature” such as project reports, compendia of mHealth/eHealth initiatives, and websites and blog posts describing active or completed projects. Searches at this early stage were conducted in English, Spanish and Portuguese; since members of the team are fluent in these languages. From an initially large and diffuse set of results we identified 22 case reports that reflected the review’s iteratively refined focus on good governance, rather than management or leadership. We also developed a list of key actors who surfaced repeatedly in relevant case reports, including funders (eg, US government, Swedish government), research organizations (eg, the Anti–Corruption Resource Centre, Transparency International) and technology organizations (eg, Ushahidi) [26]. In a spreadsheet we summarized each report’s “use case” and key themes. Based on this spreadsheet we continued to iteratively refine our search terms and initiate the search for peer–reviewed literature. Our approach reflects similar landscape scans undertaken as part of other scoping review exercises [27,28].

Systematically searching academic databases and applying inclusion criteria

Based on the initial concept mapping exercise and landscape scan, we defined a strategy for systematically searching for articles published in English and indexed in PubMed (for medical literature) or Web of Science (for interdisciplinary literature). Searches included combinations of the following terms: “governance,” “transparency,” “accountability,” “participation,” “participatory,” “stakeholder engagement,” “corruption,” “absenteeism,” “mHealth,” “eHealth”, “mobile phone”, “social media” and “digital.” Further articles were identified by examining reference lists and through key informant interviews.

To be eligible for inclusion articles had to describe digital technology for good governance purposes in the health sector of a low or middle–income country. Those that did not encompass all four features, or were purely concerned with information governance or project management in the context of an mHealth or eHealth project, were excluded. Figure 1 shows a PRISMA diagram representing the formal literature review and sifting process.

Figure 1.  PRISMA flowchart illustrating the search process.

Charting and analyzing the data

Due to resource constraints, articles that could not be accessed through the University of Cambridge or University of Edinburgh e–libraries were excluded. The remaining articles were downloaded for full review. In keeping with standard scoping review frameworks, we charted these studies according to key themes rather than performing full data extraction. We also followed Levac et al.’s [29] recommendation to make charting an iterative process by continually updating the data–charting spread sheet to fit the study data being extracted. The fields used for data compilation and analysis were as follows:

Consultation with expert practitioners and researchers

To validate and develop our emerging insights, we posted questions to relevant ICT and global health–oriented email lists and online forums, including GHDonline, the mHealth Working Group listserv, and several LinkedIn groups. Through these posts we identified a number of additional gray literature reports and peer–reviewed articles. Key respondent interviews were also undertaken as a means of identifying additional unpublished work, testing emergent themes, informing iterative improvements to the analysis, and supporting interpretation with reference to “real world” challenges. Interview participants were 10 purposively sampled practitioners and researchers affiliated with key organizations or technology projects that emerged repeatedly in the searches, including men and women with work experience in Africa, Asia and Latin America. Interviews were informal and unstructured, lasting for approximately 45 minutes each.

Composition of the evidence–base

Peer–reviewed evaluative research was sparse relative to other article types. The majority of included articles were identified through iterative and adaptive online searches (n = 25) rather than using keywords to systematically search academic databases (n = 9). This reflects the fact that academic articles used different terminologies and came from disparate communities of practice, including political science, sociology and medicine, confirming the appropriateness of our iterative scoping methodology. The technical reports came from WHO, the World Bank, or non–governmental organizations. Most of the peer–reviewed articles and technical reports included conceptual frameworks or descriptive case examples, rather than evaluative research.

Common uses of ICT for Good Governance Interventions in the health sector

Our analysis revealed 15 distinct ways of using ICTs as components of health governance interventions, or use cases. We grouped these into four conceptual categories: 1) gathering and verifying information on health services to improve transparency and auditability, 2) aggregating and visualizing data to aid communication and decision making, 3) mobilizing citizens in reporting poor practices to improve accountability and quality, and 4) automating and auditing processes to address fraud or similar inappropriate practices. Figure 2 illustrates how ICTs within these four broad categories have been used to support particular good–governance initiatives.

Figure 2.  Information and communication technologies (ICTs) used for health governance interventions.

Gathering and verifying information

Routine data collection is one of the more widely discussed use cases in the mHealth literature; it is well established that using mobile devices can improve data timeliness and quality [30]. Some governance initiatives have used the same or similar technologies to collect data for transparency or governance purposes. These initiatives include organization–oriented approaches, such as using mobile phones to collect data on governance practices or to independently verify routine government health statistics. For example, government health workers can use the USAID–funded GovScore app to report on institutional governance practices, such as in a project evaluating the formation of local health advisory committees [31]. The Performance Monitoring and Accountability 2020 project, sponsored by the Gates Foundation, uses the mobile app Open Data Kit to undertake public surveys about family planning and sanitation services in LMICs [32]. While this does not explicitly target governance challenges such as corruption, using data captured directly from civil society to verify routine health service statistics may enable external accountability in ways that are not possible for mHealth initiatives reliant on self–reporting by government health care providers.

Other information gathering approaches involve engaging citizens to crowdsource intelligence on health system performance, such as stock–outs, absenteeism, bribery or corruption. Rather than providing equipment (eg, smartphones) to a relatively small group of data gatherers, projects that solicit the participation of large groups of people are more likely to rely on technologies that people can access via the Internet or, particularly in LMIC settings, via the least expensive and most widely available mobile phones. In this vein, the social enterprise “Open Health Networks” enlisted members of the public to submit SMS reports of drug stock–outs and health worker discrimination against indigenous patients in Guatemala [33]. According to a press release by the Canadian Government’s International Development Research Centre, this data and subsequent community meetings resulted in seven municipalities increasing the amount of funding allotted to keep medicines in stock or provide fuel for ambulances [34]. One of the larger crowdsourcing for global health initiatives is UNICEF’s social platform U–Report, which originated in Uganda and now operates in several countries, with over a million registered users in Nigeria alone [35]. The large database of registered users enables U–Report to conduct massive polls. For example, data from 16 117 U–Report respondents was used in conjunction with traditional survey methods in a project assessing client satisfaction with services in Uganda’s public health facilities [36].

Data aggregation and visualization

While data gathered digitally may simply be summarized in written reports and discussed in face–to–face meetings, we identified a second category of use cases related to data aggregation and visualization. Digital analysis tools can become necessary as data sets grow extremely large or when information management teams are understaffed. While digital analytics and visualization tools, such as dashboards and maps, are increasingly common throughout the health sector, their use in good governance interventions involves distinctive data sets (eg, on absenteeism) or civil society verification of government statistics. Thus such tools might enable comparison of drug stock–outs across multiple catchment areas or draw attention to “hot spots” of corruption that would be less obvious when viewing massive spreadsheets. A number of digital technologies integrate data gathering, analysis and visualization tools, either for organization–oriented governance or for citizen–centered approaches such as crowdsourced maps that can be used to negotiate change [37].

Mobilization

A number of the articles and interventions we identified involved a mobilization component. Projects of this kind use ICTs to raise public awareness of corruption in order to generate political pressure for change or otherwise spur collective action aimed at reforming unethical or negligent health care practices. Such efforts reflect an important tenet of the contemporary transparency and accountability movement; that transparency alone is insufficient to drive greater government performance or accountability [16].

Some digital mobilization efforts unfold primarily online; for example through social media and blogging, eg, [4]. Others blend digital and “offline” approaches, such as pairing community meetings and poster campaigns with the information gathering and analytics tools discussed previously eg, [33]. Another common approach involves digital diary methods, where people experiencing a health issue are invited to document their own perspective and share photographs, audio or video with others eg, [38]. In some cases dialogue and mobilization are proactively cultivated by governments or through public private partnerships aiming to improve responsiveness or “feedback loops” among citizens and government actors [39,40]. In other cases, however, mobilization efforts are prioritized because government actors were initially unresponsive to citizen concerns. For example, in the 1990s over 300 000 indigenous Peruvian women and 20 000 indigenous men underwent forced sterilization through a state government “poverty reduction” campaign that was funded by international donors and initially supported by women’s rights organizations. With the aim of pressuring the government to acknowledge that the health policy was harmful, the Quipu Project uses mobile phones, radio, and an interactive documentary to communicate testimonies of those affected [41].

Automation and auditing

Finally, automation and auditing may help to address inappropriate practices by taking processes or decisions out of the hands of individual health care personnel and intermediaries. For example, new algorithms can automatically detect “outlier” data sets that show signs of having been faked by an absentee worker rather than having emerged from a genuine patient encounter [42]. Electronic billing and e–cash registers may address informal payments or bribes [43] and disbursing money via mobile phones can address a major source of corruption, enabling ministries to ensure that full salaries go to the intended workers in a timely manner [32,44].

Digitizing processes can also increase auditability. For example, doubts regarding whether community health workers actually visit the homes of remote patients in their care may be addressed using biometric fingerprint technology to verify each patient visit [45]. Auditable databases of dispensed drugs are also being used to tackle the widespread problem of counterfeit medicines. This typically involves labeling all medicines with serial numbers so that purchasers who text message a unique code to an SMS hotline can verify that their product is registered eg, [46,47].

Discussion

While a growing number of anecdotal reports suggest that digital interventions for good health sector governance hold promise, the relevant evidence is undeniably mixed. For every success, there have been outright failures, as is the case with conventional good governance interventions (for which there are more randomized trials) [16]. Numerous reports and expert interviews stressed that ICT for health governance interventions hinge on nuanced contextual factors and the challenge of linking transparency and action.

Our analysis revealed fifteen unique use cases of ICT for good governance, clustering into four conceptual categories associated with better information for transparency, usable data for decision making, citizen mobilization for accountability and process automation for fraud prevention. While most of these use cases targeted government–sponsored services, some extended to the private sector, such as those aimed at combatting drug counterfeiting. Since the private health care sector is dominant in many low– and middle–income countries we anticipate seeing more ICT for good–governance focused on these settings in the future, mindful of the role of government in ensuring that these are effectively regulated.

It should be noted that, while we organized our findings around a collection of generic digital tools, in practice there is a tendency to mix and match two or more of these as components of integrated interventions that enable governance processes. Such integrated governance interventions aim to strengthen citizen–government “feedback loops” [42,48], or complete an “action cycle” [16], using data to drive performance improvement or responsiveness in government services through particular mechanisms. For example, the My Voice project in Nigeria enabled citizens visiting health facilities to send feedback via SMS; the feedback was visualized for government leaders through an online dashboard, and these leaders then completed a feedback loop by using citizen–generated data in their routine review meetings regarding the performance of specific health facilities [42]. This reflects a growing acknowledgment that transparency alone (eg, having an Open Government data portal) is insufficient to influence governance [16,49]. To put this in other terms, it should not be assumed that digital technologies or eGovernment platforms will deliver good governance–related benefits unless they explicitly address specific and measurable concerns with performance or facilitate concrete mechanisms of responsive governance.

Factors limiting the effectiveness of digital good governance inventions in developing countries include lower rates of Internet access and mobile phone ownership among women and vulnerable groups [42,50,51]. Poor local network access or smartphone penetration could further marginalize people who already have less influence in governance. Understanding local patterns of technology use is therefore vital when determining which components of good governance interventions should be digitized and which are better left offline. The extent to which governments are responsive to public concerns also has implications for which approaches are likely to succeed [16]; indeed complaining about the government can be dangerous for citizens in some countries.

Finally, the preponderance of reports and expert interviews indicated that digital good governance interventions in health care are deeply complex. Their outcomes hinge on distinctive political factors in addition to the myriad organizational and sociotechnical dynamics that shape digital health innovation generally [52]. The value of human–centered design (HCD) in addressing such complexities is increasingly recognized in reports on ICT for health governance [31,33,42] as well as broader consensus statements, such as the widely ratified Principles for Digital Development [53]. HCD principles and practices include grounding the project in insights from fieldwork undertaken in the context of use, involving end–users in the design process, and iteratively adapting to feedback and experiential insights. Such approaches share important conceptual and philosophical links with transparency and accountability interventions, the former influenced by Scandinavia’s participatory design movement [54], and the latter by the broader participatory global development community [16]. In both communities, participatory approaches reflect practical priorities as well as the democratic view that ordinary people should have a say in matters that affect them. While participatory co–design will not guarantee the effectiveness of digital good–governance interventions, it offers practical resources for dealing with complex design situations and merits further attention, given its links with participatory approaches to governance.

Limitations

In keeping with our research aims and with methodological guidance for scoping reviews [24,25], our search and study inclusion process was broader than would be typical for a fully systematic review but also more limited, in terms of databases, keywords and the absence of critical appraisal. As already noted, scoping reviews are best understood as hypothesis generating activities rather than hypothesis testing endeavors, and are useful for mapping emerging areas with scattered literatures. In this case, the scoping process proved extremely valuable in helping us to refine our research focus, align with appropriate theoretical literature and converge different types of evidence to better understand this evolving interdisciplinary area. In addition to undertaking all of the research stages recommended for scoping reviews, we conducted two stages that are typically considered optional: a landscape scan of implementation activities and consultation with experts and practitioners. We recommend that future systematic reviewers wishing to build on this work use a wider range of databases, including those specializing in research from LMIC, as well as more exhaustive search methods.