The extent of the experience in 25 countries with the QA/QI MN tool and the specific attention paid by the tool to case management provide an unprecedentedly comprehensive and detailed insight into the existing quality gaps in a wide variety of settings, thus contributing to our understanding of the key issues to be addressed by health providers, hospital managers, and decision makers to ensure effective, respectful and dignified health care to all mothers and babies [6,8].

Although quality gaps in MNHC have been documented for specific components of care in many health system contexts, to our knowledge this is the first description of quality gaps along the continuum of care from admission to discharge in a wide array of health system contexts by using a homogeneous approach. A similar approach was used to assess the quality of neonatal and paediatric care in several LMIC countries [17-20], while a study documented gaps in MN care in three countries in Eastern Europe and Central Asia using the first version of the tool [21].

The review allowed to the identification of macro trends in QoC across countries. Serious quality gaps related to all WHO standards [7] were observed. Areas of care which emerged as particularly critical across facilities and countries included: insufficient or incomplete adherence to recommended evidence-based procedures for normal childbirth and at risk maternal and neonatal conditions and complications; excess of inappropriate or unnecessary interventions; insufficient attention to ensure early contact and initiation of breastfeeding; failure to provide respectful care, adequate communication and emotional support to mothers and babies; insufficient infection control; poor use of information generated locally to analyse processes and outcomes. Insufficient provision of life-saving procedures (too little-too late) and excessive interventions of unproven efficacy (too much-too soon) [22] and lack of respectful, dignified care and of adequate information to patients and their family members were observed in all countries, irrespectively of health systems structure and capacity, thus indicating that there is some convergence in quality gaps even in greatly diverse systems [23]. At the same time, significant differences were observed among facilities belonging to the same health systems, ie, with very similar staffing, infrastructure, equipment and commodities, thus confirming previous observations [17,20,24] and indicating that responsibility for quality gaps cannot be attributed only to system components and that action at system level must be accompanied by action at facility level, where key staff and hospital mangers can make the difference.

The describe tool has four distinctive features, each producing an added value to the QA/QI process.

First, it explores in depth into medical, nursing and organizational procedures from admission to discharge, looking at actual case management while most of currently available quality assessment tools are focused on basic requirements such as staffing, equipment, and protocols [12]. Second, in line with WHO recommendations [6] it includes women’s experience of care by collecting their views and incorporating them in the feedback to hospital managers and staff, who can in this way acquire a new perspective into the QoC they provide [25,26]. Third, it builds local capacity at both facility level, by refreshing and updating health professionals’ knowledge about evidence-based interventions, and at national level, by promoting the creation of a national team of assessors and a sensitization to quality improvement among lead professionals and policy makers. Finally, and most important, the tool is conceived to prompt a move from the first step, ie, analysis, to the second, ie, planning, thus providing the basis for complete quality improvement cycles, a step which is often missing in current maternal and neonatal quality measurements at facility level [12]. It also allows, through the scoring system, a semiquantitative, and easily visualized when the scoring is converted in a color-code, description of the severity of the gap, which is used for prioritizing actions and to provide a baseline to evaluate change over time.

Although none of these features of the tool is innovative per se, yet their combination is not encountered in current QA/QI approaches in MNHC. Among the tools which have been developed and used to assess quality along the continuum of MNH care, most are focusing on the availability of infrastructure, staff, equipment and on their potential capacity to deliver basic or more comprehensive procedures [9,12,27,28]. They are not designed to build awareness among managers and staff at facility level about what can be done better with the available resources, let alone to promote willingness to change, both of which are hardly achieved without participation in the assessment process and findings. UNICEF has recently proposed a comprehensive approach which by many ways is similar to what we have described in that it includes assessment of case management and collects users’ views along the whole continuum of MNHC [29]. This approach, however, is very resource-intensive and does not seem suitable for use at scale, since it requires 7 days of training for the assessment team and 14 days of observation and analysis in each facility, as opposed to 2 days of training and 2 to 3 days for the assessment required by the WHO tool.

The multifaceted potential of the tool needs to be fully understood as well as its limitations, among which the fact that the key component of the assessment, focusing on case-management, remains necessarily based on assessors’ judgment. However, the granularity of the assessment, which is articulated in several observation items for each recommended standard of care, the fact that the assessment is grounded on information derived from multiple sources, such as observed cases, records and interviews, and the cross-referencing between two or more assessors on the same items reduce significantly the subjectivity of the assessment [14,15]. The tool adopts a quantitative scoring scale indicating whether quality gaps represent moderate, serious or very serious threats to the health and dignity of the recipients of care [14,15]. The scoring system is designed to allow prioritization of issues and to appraise and quantify the results of quality cycles within the same facility, not to make comparisons among hospitals within the same country, let alone among countries. It should be stressed that the primary aim is to foster change in each facility rather than to rank facilities. Use for accreditation, however, is among the potentials of the tool, as country-specific minimum score requirements for each aspect of care could be set for certification or for stepwise progression to excellence.

Our review had to face several limitations. We are not able to claim that we included all the existing experiences done with the WHO QA/QI MN tool. In at least one case where we know the assessment was made we could not find the corresponding report. The reports themselves were heterogeneous in the amount of detail they provided so that we had to work on the maximum amount of information that was available in all reports. The scrutiny for identification and reporting of the quality gaps, based on frequency and severity, had to be based on the judgment of two of the Authors, those who could ensure the best knowledge of the process based on direct participation in many of the assessments, although the findings were confirmed by all Authors.

The experience made with the WHO QA/QI MN tool provides compelling evidence about the importance of a competent quality assessment, based on information collected through a variety of sources and performed by a multidisciplinary team of professionals to ensure specific expertise and insights in the process of care. The professional competence provided by the multidisciplinary assessment team provides authoritative technical and scientific support when needed and has the vital role of ensuring a supportive, peer-to-peer attitude throughout the process. This competent professional insight cannot be substituted by checklists, which provide a guide to ensure standard-based scrutiny but do not offer a peer-to-peer exchange focused mainly on case-management. Similarly to internal auditing facilitated by external professionals [30], this insight is key to build awareness about quality gaps, identify causes and actions to remove obstacles to quality of care and build local capacity.

While recognizing that action at all level of the system is needed to ensure quality of care [31], we argue that the experience made with the tools supports the view that QI cycles at facility level should be primarily based on competent assessments performed by multidisciplinary team of professionals and aimed at identifying in great detail the parts of the care pathway which require improvement.