You are here

  • Home
  • Archive
  • Volume 8, Issue 4
  • What is the quality of the maternal near-miss case reviews in WHO European Region? Cross-sectional study in Armenia, Georgia, Latvia, Republic of Moldova and Uzbekistan

Article Text

Article menu

Download PDFPDF

XML
Global health
Research
What is the quality of the maternal near-miss case reviews in WHO European Region? Cross-sectional study in Armenia, Georgia, Latvia, Republic of Moldova and Uzbekistan
  1. Alberta Bacci1,
  2. Stelian Hodorogea2,
  3. Henrik Khachatryan3,
  4. Shohida Babojonova4,
  5. Signe Irsa5,
  6. Maira Jansone6,
  7. Iurie Dondiuc7,
  8. George Matarazde8,
  9. Gunta Lazdane9,
  10. Marzia Lazzerini1
  1. 1 WHO Collaborating Centre for Maternal and Child Health, Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste, Italy
  2. 2 State Medical and Pharmaceutical University "N. Testemitanu", Chisinau, Moldova
  3. 3 WHO Country Office in Armenia, Yerevan, Armenia
  4. 4 Republican Perinatal Center, Tashkent, Uzbekistan
  5. 5 Pauls Stradins Clinical University Hospital, Riga, Latvia
  6. 6 Riga Stradins University, Riga, Latvia
  7. 7 Municipal Clinical Hospital Nr 1, Chisinau, Moldova
  8. 8 UNFPA Country Office, Tbilisi, Georgia
  9. 9 Division of Noncommunicable Diseases and Promoting Health through the Life-course, WHO Regional Office for Europe, Copenhagen, Denmark
  1. Correspondence to Dr Marzia Lazzerini; marzia.lazzerini{at}burlo.trieste.it

Abstract

Objectives The maternal near-miss case review (NMCR) cycle is a type of clinical audit aiming at improving quality of maternal healthcare by discussing near-miss cases. In several countries this approach has been introduced and supported by WHO and partners since 2004, but information on the quality of its implementation is missing. This study aimed at evaluating the quality of the NMCR implementation in selected countries within WHO European Region.

Design Cross-sectional study.

Settings Twenty-three maternity units in Armenia, Georgia, Latvia, Moldova and Uzbekistan.

Assessment tools A predefined checklist including 50 items, according to WHO methodology. Quality in the NMCR implementation was defined by summary scores ranging from 0 (totally inappropriate) to 3 (appropriate).

Results Quality of the NMCR implementation was heterogeneous among different countries, and within the same country. Overall, the first part of the audit cycle (from case identification to case analysis) was fairly well performed (mean score 2.00, 95% CI 1.94 to 2.06), with the exception of the ‘inclusion of users’ views’ (mean score 0.66, 95% CI 0.11 to 1.22), while the second part (developing recommendations, implementing them and ensuring quality) was poorly performed (mean score 0.66, 95% CI 0.11 to 1.22). Each country had at least one champion facility, where quality of the NMCR cycle was acceptable. Quality of the implementation was not associated with its duration. Gaps in implementation were of technical, organisational and attitudinal nature.

Conclusions Ensuring quality in the NMCR may be difficult but achievable. The high heterogeneity in results within the same country suggests that quality of the NMCR implementation depends, to a large extent, from hospital factors, including staff’s commitment, managerial support and local coordination. Efforts should be put in preventing and mitigating common barriers that hamper successful NMCR implementation.

  • maternal health
  • near-miss case review
  • standard-based assessment
  • quality of care
  • middle-income countries

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-017696

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This is the first study reporting on the quality of the hospital-based near-miss case review (NMCR) in Central Asia and Eastern Europe.

    • The assessment included five countries within WHO European Region and was based on a predefined checklist, providing the opportunity to evaluate the implementation of the NMCR approach in a standardised manner.

    • In three countries facilities included in the evaluation accounted for all facilities implementing the NMCR within in the country. In the remaining two countries, where the NMCR was implemented in more hospitals, facilities were chosen in dialogue with local authorities (non-probability sampling), and not at random; however, criteria used to select facilities included also geographical distribution (ie, so that different regions were represented) and hospital type (ie, different types of hospitals were selected).

    Introduction 

    Ensuring adequate quality of healthcare is a primary objective of WHO Global Strategy for Women’s, Children’s and Adolescent’s Health 2016–20301 and of Health 2020, the European strategic framework setting the policy directions for the 53 member states in WHO European Region.2 Quality in healthcare is recognised as essential for the health and well-being of the population, and as a basic aspect of human rights.3–5

    Among the different strategies aiming at improving quality of care at maternity services, the facility-based maternal near-miss case review (NMCR) cycle was proposed by WHO in 2004 as a type of clinical audit.6–8 With respect to mortality audit, the NMCR has the advantage to imply less legal issues, and is therefore perceived as more acceptable by staff. Near-miss cases are defined as a woman who nearly died but survived a complication that occurred during pregnancy, childbirth or within 6 weeks after pregnancy.9 In the facility-based NMCR all hospital staff involved in the management of the chosen near-miss case—including obstetricians, midwives, nurses and ancillary staff—get together to discuss and evaluate the care provided against national evidence-based guidelines, local protocols and standards of care. The aim of the case review is to critically discuss local management, procedures and attitudes, and to identify areas that can be further improved.9 Actions to improve quality of maternal healthcare are proposed and agreed by hospital staff, and subsequently monitored to check their implementation, as for a continuous quality improvement process.9 One of the key characteristics of this method is the bottom-up approach, aiming at facilitating local ownership of the process, commitment in implementing the proposed recommendations and team building. Currently, the review of severe maternal morbidity cases (‘near-miss’ events) is recommended by WHO as a key action to eliminate avoidable maternal and perinatal mortality and morbidity and improve the quality of care.10

    While in some countries within WHO European Region (such as UK, Norway and the Netherlands) the practice of reviewing maternal near-miss cases was introduced by the government or by professional associations, in several other countries (most often middle-income countries) its implementation was assisted by WHO and/or United Nations Population Fund (UNFPA). In the latter scenario, coverage and quality of the NMCR implementation were usually discussed during workshops,11–13 but so far they have not been evaluated using a systematic methodology.

    In 2015, WHO developed a checklist for assessing the quality of the implementation of the NMCR cycle at hospital level through a systematic methodology.9 This study aimed at evaluating the quality of the NMCR implementation in five countries of Eastern Europe and Central Asia, using WHO checklist, to identify common strengths and weaknesses among different settings.

    Material and methods

    Population and setting

    The assessment was conducted in Armenia, Georgia, Latvia, Republic of Moldova and Uzbekistan between June 2015 and October 2016. Countries were chosen based on the following criteria: (1) activities planned by the Ministry of Health (MoH) included a quality assessment of the NMCR; (2) there was a request for technical assistance from WHO or UNFPA.

    In all of the countries the NMCR approach was introduced following WHO methodology.9 The year of NMCR introduction differed among countries (table 1).

    View this table:
    Table 1

    Characteristics of the countries and of the maternity units assessed

    The number of facilities visited in each country depended on the total number of hospitals implementing the NMCR cycle: in Armenia, Georgia and Latvia all facilities implementing the NMCR were visited; in Moldova and Uzbekistan, where a large number of maternity units are implementing the NMCR, a sample was selected in agreement with the MoH and the national NMCR coordinator/s, following a geographical criteria (ie, so that different regions were represented) and including different types of hospitals. Overall, 23 maternity units were visited in the five selected countries (table 1).

    Data collection

    Each facility was visited for at least the duration of a whole day by two independent external experts with long-term experience in NMCR implementation. The international team was joined by the national assessors, experienced in NMCR implementation at local level. The team was under the leadership of one international assessor (AB), who participated in all hospital visits, with the objective of ensuring standards procedures in all assessments.

    The assessment was carried out using a checklist developed by WHO to evaluate the quality of the NMCR cycle at hospital level (online supplementary table S1). The checklist was developed by WHO in 2014, field tested and optimised for use in early 2015.9 The methodology for the quality assessment is fully described in a WHO manual.9 Briefly, the checklist includes 50 items, grouped in 11 domains. The sources of information for the assessment include: direct observation and evaluation of one or more NMCR sessions; discussion with participants, coordinators and managers; documents from the NMCR sessions (templates and notes from the sessions); local documents (regional/local policies and guidance documents; protocols and standards for care; documents related to quality assurance, monitoring and supervision; reports on NMCR activities); national documents (national policies and guidance documents, guidelines and reports on NMCR implementation). According to WHO methodology, using WHO manual9 as source of standards, each of the 50 items was scored from 0 (totally inappropriate) to 3 (appropriate) (online supplementary table S1). For each of the 11 domains the arithmetic mean and 95% CIs among all the items in that domain were calculated. The median and the range between the first and third quartile (IQR range) were also calculated.

    Supplementary file 1

    In each facility, immediately after the assessment, feedbacks were discussed with the local staff and plans for improvement of the NMCR implementation were developed, using a simple matrix (online supplementary table S1).

    After completing the visits to all maternity units in the country, a national restitution workshop was organised involving representatives from the hospitals, health authorities, professional organisations and partners. During the workshop, achievements and constraints were presented and underlying reasons were discussed. Recommendations for improvement were developed and synthesised in a standard predefined simple matrix (online supplementary table S1).

    Ethical considerations

    Activities of this observational study were initiated on request of the MoHs and carried out in close collaboration with the health authorities; ethical approval was not required. Information to hospital staff was provided by MoH representatives and local authorities. All people involved in the NMCR sessions were informed about the purpose of the visit and oral consent from the hospital staff and local coordinators and facilitators participating in the observed sessions was obtained. The review of near-miss cases was carried forward anonymously, that is, information that may have disclosed the identity of the patient, or providers of care, was not reported.9 This study did not aim at directly comparing countries or single facilities with different background, context and timelines of implementation, therefore, results of the assessment are reported in an anonymous way, according to WHO methodology.9 Detailed finding of the assessment together with feedback on how to improve quality of the NMCR implementation were provided to each facility and to each country individually.

    Results

    The assessment pointed out that quality of the implementation of the NMCR cycle was heterogeneous among different countries, as well as among different hospitals within the same country. Table 2 reports the results of the summary scores, for each of the 11 domains of WHO assessment checklist.

    View this table:
    Table 2

    Summary scores

    Overall, the first part of the audit cycle (steps 1–6 in table 2, ie, from case identification to case analysis) was on average fairly well performed in all countries (mean score 2.00, 95% CI 1.94 to 2.06), with the exception of the domain ‘inclusion of users’ views’ which was poorly implemented in most facilities (mean score 1.06, 95% CI 0.12 to 2.00). The second part of the audit cycle (steps 7–10), which involves developing appropriate recommendations, implementation of the recommendations, follow-up, documentation and dissemination of results within the facility and the country, was on average poorly performed in all countries (mean score 1.20, 95% CI 0.93 to 1.46). In particular, the domain 11 ‘ensuring quality in the NMCR cycle’, which implies a process of periodical quality assessment, development of recommendation for quality improvement and related actions, was overall substandard (mean score 0.66, 95% CI 0.05 to 1.28), with the exception of country E, where regular monitoring and supervision was carried out by a team that included national and international members.

    In each country it was possible to identify at least one ‘champion’ facility, where quality of the NMCR cycle had only minor deficiencies (A-H3, B-H4, C-H1, D-H3, EH1 and H2). On the other hand, in a few facilities (A-H2; B-H1 and H3; DH6) most of the areas assessed were judged as ‘totally inappropriate’.

    In some facilities examples of good practices were also observed for domains that were on average implemented on a substandard level at a country level. For example, despite inclusion of users’ views being substandard in most facilities in countries B and D (mean scores 1.11, 95% CI 0 to 2.22 and 0.61, 95% CI 0 to 1.48, respectively) single facilities reached good scores (B-H4 had a score of 3 and D-H3 had a score of 2), being able to regularly interview women and incorporating their views in the development of recommendations to improve hospital care (table 2).

    On average, quality of the implementation of NMCR was on a higher level in country E, where evaluation scores pointed out that there were only few weaknesses in implementation compared with other countries (mean score 2.12, 95% CI 1.84 to 2.39).

    Table 3 summarises main common strengths and weaknesses in the quality of the NMCR implementation, as divided in three categories: (1) those mostly related to technical aspects, (2) those predominantly of organisational nature and (3) those related to the attitude towards the NMCR. The main technical strength was that, beside the existence of appropriate technical skills in the methodology, most facilities developed several recommendations that were achievable, realistic, time-bound and with a potential impact on the quality of care. Although recommendations were not always well documented (thus resulting in low scores under domain 10), gaps in reporting results did not always indicated actual gaps in implementation, and in many cases several recommendations were actually implemented. This was a common observation in country B, where recommendations were poorly recorded, but several actions to improve quality of care—such as setting up emergency kits and related protocols, and introducing the Modified Obstetric Early Warning Score chart14—were actually implemented. Among strengths in organisational aspects, the most important was that NMCR was regularly held, and staffing at all levels, including midwives, participated. Main strengths in attitude included the endorsement and application of the basic principles of the NMCR (confidentiality, openness, respecting diverting opinions and avoiding blame).

    View this table:
    Table 3

    Strengths and weaknesses observed in the quality of the NMCR implementation

    Main gaps in technical aspects were: inappropriate case reconstruction, case analysis not getting to the ‘real point’ and not using a ‘why-but-why’ approach (ie, discussion of underlying causes) and recommendations not being fully SMART (specific, measurable, achievable, realistic, and time-bound).15 Main gaps of organisational nature were: lack of continuity in the role of facilitator/coordinator, lack of proper dissemination of the results (ie, circulation of information within the facility level and at national level on how many and what type of recommendations were developed) and lack of follow-up on previous recommendations. Major gaps in adopting the background philosophy and principles of the NMCR were observed in some facilities such as: lack of respect for other people’s opinion, persistence of blaming and judging others rather than using the NMCR cycle to discuss and improve ways of working and insufficient involvement of mid-level staff. Lack of inclusion of the users’ view, which was a frequent observation, was reported to be due to the lack of trained interviewers, and this was interpreted as not merely an organisational gap, but also as a problem in attitude of the health providers, that is, lack of understanding the importance of taking into account the women’s point of view. Finally, common to most facilities, there was insufficient monitoring and evaluation, and lack of a quality assurance mechanism. In most cases this was due to deficiencies in establishing and efficiently running a NMCR coordination system at national level.

    Recommendations developed by local stakeholders during the national restitution workshops were setting specific. Nevertheless, there were several similarities. The most frequent/relevant recommendations developed for implementation at different levels—hospital level, national level, WHO and development partners—are reported in table 4.

    View this table:
    Table 4

    Recommendations made by local stakeholders on how to improve NMCR quality

    Examples of the observed impact of the NMCR on quality of care at facility level are reported in online supplementary table S2. Despite progress was often poorly reported both in the hospital and in national reports, several achievements could be observed. These included improved use of national clinical guidelines, development and use of local protocols and standards of care, better availability and organisation of emergency services, improved autonomy of midwives and positive dynamics such as improved team working.

    Discussion

    This study aimed at evaluating the quality of the NMCR at hospital level in selected countries within WHO European Region using a standardised checklist and methodology. Overall the assessment pointed out that the practise of reviewing near-miss cases at hospital level is currently ongoing in all countries included in this study; however, both coverage and quality of the implementation of the NMCR cycle are heterogeneous. Overall, while the first part of the audit cycle (from case identification to case analysis) was fairly well performed, with the exception of the ‘inclusion of users’ views’, the second part of the audit cycle (developing recommendations, implementing them and ensuring quality) was in general poorly performed. Gaps in implementation were both of technical, organisational and attitudinal nature.

    These findings are not entirely surprising. Previous, although less systematic, evaluations in the same geographical area pointed a series of challenges7 8 11 12 in effectively implementing the review of near-miss cases at facility level. Beside technical and organisational challenges, the successful implementation of clinical audits such as the NMCR often calls for a major change in staff’s attitude.7 8 11 12 In the countries assessed, especially in the ex-Soviet countries, the successful implementation of the NMCR aims at moving away from a ‘traditional’ system of carrying forward clinical audits, where blame and punishment were the routine, subjective judgement were the rule and audit involved only doctors, while midwives, other mid-level staff and service users had no voice.7 8 11 12 The ‘traditional’ audit system mainly resulted in punishing single individuals, rather than at looking to the health system failures and finding solutions at organisational level.7 8 11 12 Changing practices involved building knowledge and skills together with a drastic shift in attitude. Given these substantial constraints, the successful implementation of the NMCR at least in one country (country E) and in several champion maternity units in other countries, must be seen as a positive achievement, proving that NMCR can be successfully implemented in different settings.

    This paper reports the quality of the NMCR implementation in middle-income countries (Armenia, Moldova, Uzbekistan are lower middle income countries and Georgia is an upper middle income country), where the NMCR was carried forward with relatively limited resources. Findings of this assessment cannot be generalised to other high-income countries of WHO European Region, such as UK, Norway and the Netherlands, where the practice of reviewing maternal near-miss cases has been institutionalised, with major efforts on creating coordinating mechanisms.16–18 However, it must be acknowledged that the review of near-miss cases at facility level is still not a routine practice in many European countries. We were unable to identify any study reporting on a standard-based assessment of the quality of the NMCR from any country of WHO European Region.

    Interestingly, findings of this study suggest that quality of the implementation of the NMCR cycle is not strictly associated to the duration of the implementation. However, it is also true that adequate time is needed for implementation, and completing a pilot phase in a country cannot take <18–24 months from the first technical workshop. In this regard, it must be acknowledged that country B started piloting just 6 months before the quality assessment; therefore, observed results in this country can be interpreted as satisfactory given the short time frame.

    The high heterogeneity in results within the same country (such as in the case of countries A, B and D) suggests that quality of the NMCR implementation depends, to a large extent, from hospital factors, including staff’s commitment, managerial support and local coordination. These results are in line with a systematic review on facilitators and barriers to effective implementation of NMCR cycle, pointing out that hospital factors (good leadership), together with a system of coordination (which often includes external support), are key enablers for effective NMCR implementation (Lazzerini, Ciuch, Covi, et al. Submitted, 2017).19

    This assessment pointed out that, despite WHO recommends conducting an interview with the women/her family for each near-miss case, inclusion of women’s view was still substandard in many of the assessed facilities. However, some facilities (B-H4 and D-H3) reached good scores even when this domain was problematic at a country level (table 2). In WHO framework, ‘experience of care’ is one of the two key components of quality of maternal and newborn healthcare, along with ‘provision of care’.1 2 The views of women and their families can provide relevant information on aspects related to case management, including important details on what happened, such as organisational issues, communication issues and respectful care. In a study in Moldova, it was observed that the implementation of NMCR improved attitude towards patients,20 while in Kazakhstan it successfully improved patients’ satisfaction.21 22

    This study points out that quality in the reporting on the NMCR activities was overall low. WHO manual now provides a series of templates to facilitate a uniform reporting.9 Sustained monitoring and evaluation based on appropriate reporting, as well as periodical quality assessments should be part of a strategy to achieve quality in the NMCR implementation.

    This paper has the merit of reporting the actual state of implementation of NMCR in a real setting and not in a study setting (where usually a limited number of facilities is involved for a limited period of time, with dedicated human and financial resources). Another strength of the study is that the evaluation was carried out in a systematic way using a predefined standardised tool and methodology, aiming at evaluating all key aspects that contribute to overall NMCR quality (online supplementary table S1).9 To our knowledge, no other previous similar systematic evaluations have been performed.

    We acknowledge that the scoring system used by the checklist may be open to some subjectivity. However, this scoring system is similar to others extensively used by WHO in the last 15 years for systematic, standard based, quality assessments and it proved to be able to capture key elements of quality of the implementation in both pragmatic and research settings.23–27 No other validated tool or scoring system exists to assess quality of the NMCR. The checklist and its score system were field tested before use, until when they were considered satisfactory covering all key aspects of quality of NMCR.9 The score is attributed by a team of experts, thus reducing subjectivity of the single individual in the evaluation.9

    As a second limitation, we acknowledge that in two out of the total five countries (Moldova and Uzbekistan), the sample was selected based on MoH indications (non-probability sampling), and one cannot exclude a selection bias towards the better performing institutions. However, we emphasise that the main purpose of the assessment was to create an opportunity at national level, to discuss quality of the NMCR and to develop recommendations for improvement. Subsequent assessments could extend the evaluation to other facilities and monitor progress in specific areas.

    Based on the results of this study, in the future, more efforts should be put in evaluating the quality of the implementation of NMCR on a regular basis. More implementation studies should explore interventions aiming at improving quality of the NMCR implementation in different settings.

    The objective of this study was not evaluating the impact of the implementation of the NMCR, but rather the quality of the process. Nevertheless, several achievements could be observed (online supplementary table S2), despite this type of information was not consistently available. These results are in line with other studies28–41 and a systematic review reporting that NMCR is an effective strategy in improving quality of care when measured against predefined standards and it may even significantly reduce maternal mortality in high burden countries (Lazzerini, Richardson, Ciardelli, et al. Submitted, 2017).

    Conclusions

    Ensuring high quality in the implementation of the NMCR may be difficult in countries of Eastern Europe and Central Asia, but achievable. In the future, more efforts should be put in evaluating the quality of the implementation of NMCR on a regular basis, capitalising from these lessons and preventing and mitigating common barriers that hamper successful implementation. The availability of a new manual on how to implement and to monitor the NMCR at facility level, and of a standard methodology for assessing quality of the NMCR, as well as templates for reporting,9 may facilitate this process.

    Acknowledgments

    We would like to thank the local NMCR country coordinators, the hospitals staff, WHO regional and country staff and development partners who collaborated in the organisation of the assessment missions. We thank Sonia Richardson for language revision in this paper.

    References

    1. 1.↵
      World Health Organization. Global Strategy for Women’s, Children’s and Adolescent’s Health 2016-2030. http://www.who.int/life-course/partners/global-strategy/global-strategy-2016-2030/en/ (accessed 15 Dec 2016).
    2. 2.↵
      World Health Organization (WHO), Regional Office for Europe. Health 2020: the European policy for health and well-being. Copenhagen: WHO Regional Office for Europe, 2013.
    3. 3.↵
      World Health Organization (WHO). The prevention and elimination of disrespect and abuse during facility-based childbirth. Geneva: World Health Organization, 2014.
    4. 4.↵
      2. Tunçalp Ӧ ,
      3. Were WM ,
      4. MacLennan C , et al
      . Quality of care for pregnant women and newborns-the WHO vision. BJOG 2015;122:1045–9.doi:10.1111/1471-0528.13451
      OpenUrlCrossRefPubMed
    5. 5.↵
      World Health Organization. Standards for improving quality of maternal and newborn care in health facilities. Geneva: World Health Organization, 2016.
    6. 6.↵
      World Health Organization. Beyond the numbers: Reviewing maternal deaths and complications to make pregnancy safer. Geneva: World Health Organization, 2004.
    7. 7.↵
      2. Bacci A ,
      3. Lewis G ,
      4. Baltag V , et al
      . The introduction of confidential enquiries into maternal deaths and near-miss case reviews in the WHO European Region. Reprod Health Matters 2007;15:145–52.doi:10.1016/S0968-8080(07)30334-0
      OpenUrlCrossRefPubMedWeb of Science
    8. 8.↵
      2. Bacci A
      . Implementing “Beyond The Numbers” across the WHO European Region: steps adopted, challenges, successes and current status. Entre Nous 2010;70:6–7.
      OpenUrl
    9. 9.↵
      World Health Organization. Regional Office for Europe. Conducting a maternal near-miss case review cycle at the hospital level manual with practical tools. http://www.euro.who.int/en/health-topics/Life-stages/maternal-and-newborn-health/publications/2016/conducting-a-maternal-near-miss-case-review-cycle-at-hospital-level-2016 (accessed 29 Nov 2016).
    10. 10.↵
      World Health Organization Regional Office for Europe. Action plan for sexual and reproductive health: towards achieving the 2030 Agenda for Sustainable Development in Europe – leaving no one behind. Copenhagen: World Health Organization Regional Office for Europe, 2016.
    11. 11.↵
      World Health Organization. Regional Office for Europe. Multi-Country review meeting on maternal mortality and morbidity audit “Beyond the Numbers”, Report of a WHO meeting, Charvak, Uzbekistan 14–17 June 2010. Copenhagen: WHO Regional Office for Europe, 2010.
    12. 12.↵
      World Health Organization. Regional Office for Europe. The impact of implementation of ’Beyond the numbers' approach in improving maternal and perinatal health. 29-30 April 2014, Bishkek, Kyrgyzstan. Copenhagen: WHO Regional Office for Europe, 2014.
    13. 13.↵
      WHO Regional Office for Europe Making Pregnancy Safer in Uzbekistan. Maternal mortality and morbidity audit Activities Report 2002-2008. http://www.euro.who.int/__data/assets/pdf_file/0004/98797/MPS_UZB.pdf (accessed september 8, 2016).
    14. 14.↵
      The Royal Free Hospital Nhs Trust Maternity Clinical Guidelines. MEOWS Guidance in Maternity. http://www.oaa-anaes.ac.uk/assets/_managed/editor/file/guidelines/meows/royal%20free%20meows%20guideline%20-%20mcglennan_.pdf (accessed 29 Nov 2016).
    15. 15.↵
      2. Doran GT
      . There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives.  Management Review 1981;70:35–6.
      OpenUrl
    16. 16.↵
      2. Knight M ,
      3. Lewis G ,
      4. Acosta CD , et al
      . Maternal near-miss case reviews: the UK approach. BJOG 2014;121(Suppl 4):112–6.doi:10.1111/1471-0528.12802
      OpenUrl
    17. 17.↵
      2. Marr L ,
      3. Lennox C ,
      4. McFadyen AK
      . Quantifying severe maternal morbidity in Scotland: a continuous audit since 2003. Curr Opin Anaesthesiol 2014;27:275–81.doi:10.1097/ACO.0000000000000079
      OpenUrlCrossRefPubMed
    18. 18.↵
      2. Knight M
      . INOSS. The International Network of Obstetric Survey Systems (INOSS): benefits of multi-country studies of severe and uncommon maternal morbidities. Acta Obstet Gynecol Scand 2014;93:127–31.doi:10.1111/aogs.12316
      OpenUrl
    19. 19.↵
      2. Lazzerini M ,
      3. Richardson S ,
      4. Ciardelli V , et al
      . Effectiveness of the facility- based maternal near-miss case reviews in improving maternal and newborn quality of care in low-income and middle-income countries: a systematic review. BMJ Open 2018:e019787.
    20. 20.↵
      2. Baltag V ,
      3. Filippi V ,
      4. Bacci A
      . Putting theory into practice: the introduction of obstetric near-miss case reviews in the Republic of Moldova. Int J Qual Health Care 2012;24:182–8.doi:10.1093/intqhc/mzr079
      OpenUrlCrossRefPubMed
    21. 21.↵
      2. Sukhanberdiyev K ,
      3. Ayazbekov A ,
      4. Issina A , et al
      . Initial experience of Near Miss Case Review: improving the management of haemorrhage. Entre Nous 2011;74:18–19.
      OpenUrl
    22. 22.↵
      2. Hodorogea S
      . Piloting near miss case reviews in Kazakhstan: improving quality of maternal care. Entre Nous 2010;70:28–9.
      OpenUrl
    23. 23.↵
      2. Duke T ,
      3. Keshishiyan E ,
      4. Kuttumuratova A , et al
      . Quality of hospital care for children in Kazakhstan, Republic of Moldova, and Russia: systematic observational assessment. Lancet 2006;367:919–25.doi:10.1016/S0140-6736(06)68382-7
      OpenUrlCrossRefPubMedWeb of Science
    24. 24.↵
      2. Lazzerini M ,
      3. Shukurova V ,
      4. Davletbaeva M , et al
      . Improving the quality of hospital care for children by supportive supervision: a cluster randomized trial, Kyrgyzstan. Bull World Health Organ 2017;95:397–407.doi:10.2471/BLT.16.176982
      OpenUrl
    25. 25.↵
      2. Tamburlini G ,
      3. Yadgarova K ,
      4. Kamilov A , et al
      . Improving the quality of maternal and neonatal care: the role of standard based participatory assessments. PLoS One 2013;8:e78282.doi:10.1371/journal.pone.0078282
    26. 26.↵
      2. Tamburlini G ,
      3. Siupsinskas G ,
      4. Bacci A
      . Quality of maternal and neonatal care in Albania, Turkmenistan and Kazakhstan: a systematic, standard-based, participatory assessment. PLoS One 2011;6:e28763.doi:10.1371/journal.pone.0028763
    27. 27.↵
      2. Campbell H ,
      3. Duke T ,
      4. Weber M , et al
      . Pediatric Hospital Improvement Group. Global initiatives for improving hospital care for children: state of the art and future prospects. Pediatrics 2008;121:e984–92.
      OpenUrlAbstract/FREE Full Text
    28. 28.↵
      2. Kayiga H ,
      3. Ajeani J ,
      4. Kiondo P , et al
      . Improving the quality of obstetric care for women with obstructed labour in the national referral hospital in Uganda: lessons learnt from criteria based audit. BMC Pregnancy Childbirth 2016;16:152.doi:10.1186/s12884-016-0949-1
      OpenUrl
    29. 29.↵
      2. Mohd Azri MS ,
      3. Edahayati AT ,
      4. Kunasegaran K
      . Audit on management of eclampsia at Sultan Abdul Halim Hospital. Med J Malaysia 2015;70:142–7.
      OpenUrl
    30. 30.↵
      2. Gebrehiwot Y ,
      3. Tewolde BT
      . Improving maternity care in Ethiopia through facility based review of maternal deaths and near misses. Int J Gynaecol Obstet 2014;127 Suppl 1:S29–34.doi:10.1016/j.ijgo.2014.08.003
      OpenUrl
    31. 31.↵
      2. Luz AG ,
      3. Osis MJ ,
      4. Ribeiro M , et al
      . Impact of a nationwide study for surveillance of maternal near-miss on the quality of care provided by participating centers: a quantitative and qualitative approach. BMC Pregnancy Childbirth 2014;14:122.doi:10.1186/1471-2393-14-122
      OpenUrl
    32. 32.↵
      2. Kidanto HL ,
      3. Wangwe P ,
      4. Kilewo CD , et al
      . Improved quality of management of eclampsia patients through criteria based audit at Muhimbili National Hospital, Dar es Salaam, Tanzania. Bridging the quality gap. BMC Pregnancy Childbirth 2012;12:134.doi:10.1186/1471-2393-12-134
      OpenUrlCrossRefPubMed
    33. 33.↵
      2. van den Akker T ,
      3. van Rhenen J ,
      4. Mwagomba B , et al
      . Reduction of severe acute maternal morbidity and maternal mortality in Thyolo District, Malawi: the impact of obstetric audit. PLoS One 2011;6:e20776.doi:10.1371/journal.pone.0020776
    34. 34.↵
      2. Bailey PE ,
      3. Binh HT ,
      4. Bang HT
      . Promoting accountability in obstetric care: use of criteria-based audit in Viet Nam. Glob Public Health 2010;5:62–74.doi:10.1080/17441690802190776
      OpenUrlPubMed
    35. 35.↵
      2. van den Akker T ,
      3. Mwagomba B ,
      4. Irlam J , et al
      . Using audits to reduce the incidence of uterine rupture in a Malawian district hospital. Int J Gynaecol Obstet 2009;107:289–94.doi:10.1016/j.ijgo.2009.09.005
      OpenUrlCrossRefPubMed
    36. 36.↵
      2. Hunyinbo KI ,
      3. Fawole AO ,
      4. Sotiloye OS , et al
      . Evaluation of criteria-based clinical audit in improving quality of obstetric care in a developing country hospital. Afr J Reprod Health 2008;12:59–70.
      OpenUrlPubMed
    37. 37.↵
      2. Kongnyuy EJ ,
      3. Leigh B ,
      4. van den Broek N
      . Effect of audit and feedback on the availability, utilisation and quality of emergency obstetric care in three districts in Malawi. Women Birth 2008;21:149–55.doi:10.1016/j.wombi.2008.08.002
      OpenUrlCrossRefPubMed
    38. 38.↵
      2. Kongnyuy EJ ,
      3. Mlava G ,
      4. van den Broek N
      . Criteria-based audit to improve a district referral system in Malawi: a pilot study. BMC Health Serv Res 2008;8:190.doi:10.1186/1472-6963-8-190
      OpenUrlCrossRefPubMed
    39. 39.↵
      2. Muffler N ,
      3. Trabelssi MH ,
      4. De Brouwere V
      . Scaling up clinical audits of obstetric cases in Morocco. Trop Med Int Health 2007;12:1248–57.doi:10.1111/j.1365-3156.2007.01911.x
      OpenUrlCrossRefPubMed
    40. 40.↵
      2. Weeks AD ,
      3. Alia G ,
      4. Ononge S , et al
      . A criteria-based audit of the management of severe pre-eclampsia in Kampala, Uganda. Int J Gynaecol Obstet 2005;91:292–7.doi:10.1016/j.ijgo.2005.07.022
      OpenUrlCrossRefPubMed
    41. 41.↵
      2. Wagaarachchi PT ,
      3. Graham WJ ,
      4. Penney GC , et al
      . Holding up a mirror: changing obstetric practice through criterion-based clinical audit in developing countries. Int J Gynaecol Obstet 2001;74:119–30.doi:10.1016/S0020-7292(01)00427-1
      OpenUrlCrossRefPubMed

    Footnotes

    • Contributors AB and ML conceived the study, analysed the data and wrote the first draft of the paper. AB, SH, HK, SB, SI, MJ, ID, GM and GL collected data and contributed to the final draft of the paper. GL and GM contributed by procuring funds. All authors contributed to the final version of the paper.

    • Funding The assessment was supported by WHO Regional Office for Europe and UNFPA Regional Office for Eastern Europe and Central Asia.

    • Competing interests None declared.

    • Patient consent Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Additional details on the country assessments can be obtained from the first author.