Article Text

Download PDFPDF

Protocol
Assessment of the quality of routine ambulatory healthcare for common disorders in children and adolescents in Germany: study protocol for a retrospective medical record review (QualiPäd)
  1. Viktor Tischlik1,
  2. Claudia Mehl2,
  3. Dominik Ewald3,
  4. Monika Heinzel-Gutenbrunner4,
  5. Max Geraedts2,
  6. Christian J Bachmann1
  1. 1Department of Child and Adolescent Psychiatry, Universitätsklinikum Ulm, Ulm, Germany
  2. 2Institute for Health Services Research and Clinical Epidemiology, Philipps-Universität Marburg, Marburg, Germany
  3. 3Paediatric Primary Care Centre, Regensburg, Germany
  4. 4MH Statistical Consulting, Marburg, Germany
  1. Correspondence to Professor Christian J Bachmann; christian.bachmann{at}uniklinik-ulm.de

Abstract

Introduction The quality of healthcare in childhood and adolescence is of key importance, in order to foster a healthy development and to avoid chronic health problems. Yet, data for Germany regarding the quality of healthcare for this patient group are lacking. The QualiPäd research project aims to estimate the quality of outpatient healthcare for children and adolescents in Germany, focusing on common psychiatric and physical disorders.

Methods and analysis Quality indicators for seven common physical and mental childhood and adolescent clinical conditions (attention deficit/hyperactivity disorder, asthma, atopic dermatitis, depression, otitis media, conduct disorder/oppositional defiant disorder, tonsillitis) will be developed and ratified by experts, using the RAND/UCLA Appropriateness Method.

Initially, 1400 medical records of children and adolescents with one of the aforementioned clinical conditions will then be randomly drawn from 40 outpatient practices in the German federal state of Hessen. The records will then be assessed regarding their adherence to the respective quality indicators. Based on this, the percentage of appropriate and inappropriate (eg, wasteful) healthcare of all clinical conditions (primary endpoint) will be estimated. Additionally, possible factors influencing the quality of care (eg, patient characteristics, type of condition, type of practice) will be identified using generalised estimation equation models.

Ethics and dissemination This study will show for which of the studied clinical conditions and/or patients improvement of quality of care is necessary within the German health system. Also, the quality indicators designed for the study can afterwards be implemented in regular care and thus enable regular reporting of the outpatient care of this target group. The authors plan to disseminate their findings through international, peer-reviewed scientific publications, and through presentations at national and international paediatric and child psychiatric conferences.

Trial registration number DRKS00022408.

  • public health
  • paediatrics
  • child & adolescent psychiatry
  • community child health
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Strengths and limitations of this study

  • A mix of somatic and psychiatric disorders as well as acute and chronic conditions will be evaluated, in order to cover a broad clinical spectrum.

  • The age of the target group was deliberately chosen from the neonatal phase to adolescence in order to be able to include different manifestations and courses of disease in the evaluation.

  • The selected outpatient practice specialties are those most frequently involved in the care of children and adolescents with physical or mental health problems and are thus most suitable for depicting the breadth of outpatient care.

  • The federal state of Hessen was selected as the project region because it is close to the German average in terms of birth rate, per capita income and population density.

  • Data from hospital (emergency) outpatient departments were not included for this study, as these belong to a different healthcare sector in Germany.

Introduction

There are around 13 million children and adolescents (aged 0–18 years) living in Germany, which corresponds to about 16% of the population. The vast majority of medical care for this age group in Germany is provided in an outpatient setting, usually by paediatricians or general practitioners.1 More than 90% of children and adolescents in this age group receive outpatient care at least once a year.2

While the national KIGGS (German Health Interview and Examination Survey for Children and Adolescents (age: 0–17 years); www.kiggs-studie.de) health survey provides information on the health situation in general and on the use of health services, there is hardly any data on the quality of routine care for children and adolescents in Germany. The same holds true internationally: apart from two studies from the USA and Australia,3 4 the few existing studies are either limited to a single health service or limited to individual clinical conditions.5–7 This scarcity of data is regrettable, since the quality of care, that is, the extent to which diagnosis and therapy adheres to recommendations in clinical practice guidelines, is a key component of care and has a significant influence on patient outcome. Accordingly, since the report of the Advisory Council on the Assessment of Developments in the Health Care System 2000/2001,8 the focus has repeatedly been on the topic of needs-based care, for example, in the 2014 report.9 Conversely, underuse, overuse or misuse is associated with considerable disadvantages for the patient and the healthcare system (eg, in the form of increasing antibiotic resistance due to non-adherence to guidelines for antibiotic prescribing in otitis media,10 and poorer outcome if psychotherapy is not prescribed for attention deficit/hyperactivity disorder (ADHD)11).

The high importance of the quality of care gains even more importance in the age group of children and adolescents, since for many clinical conditions in this patient group suboptimal diagnosis and therapy significantly increase the risk of consequential damage (eg, hearing loss due to an incorrectly treated otitis media) or development of chronic disease (eg, in the case of untreated depression).

As far as the authors are aware, there are currently only two studies worldwide that have investigated the quality of care in a multicentre design for several clinical conditions in childhood and adolescence in different care settings: a study from the USA examined the quality of care in the period 1996–2000 in 12 US-American metropolitan areas with regard to 12 frequent paediatric conditions (n=1536, age: 0–17 years) which found an average quality of care of 46.5%.4 In a study from Australia, Braithwaite et al3 used quality indicators to examine the quality of care for 17 common childhood and adolescent clinical conditions in 97 primary care facilities and 34 hospitals (n=6689 children, age: 0–15 years). In this study, the adherence to the quality indicators derived from treatment guidelines averaged 59.8%, which is to be interpreted as requiring considerable improvement.

Both studies used medical records in order to examine the quality of care. While there are alternative approaches for studying the quality of care, for example, using secondary data from health insurance funds, or self-report data from practices, for our study we chose to also use medical records. This was done for two purposes: first, this approach enables comparability with the existing studies, and offers a more objective perspective than self-reports can provide. Second, in contrast to secondary data, medical records usually offer more detailed information, for example, regarding duration of pharmacotherapy, severity of illness and so on. Of course, compared with secondary data, there are also disadvantages to our approach in, especially a possible selection bias, as practices with lower quality of care might be reluctant to participate.

The aforementioned findings thus confirm recommendations which, due to considerable deficits in the quality of care for sick children and adolescents, had already called for the development of quality indicator sets years ago.12 This urgent need for research into the development and application of suitable measuring instruments for assessing the quality of care was also confirmed in the recently completed EU-funded project ‘Models of Child Health Appraised’ (MOCHA; http://www.childhealthservicemodels.eu/), and is also named in the Third European Health Program (https://ec.europa.eu/health/funding/programme/2014-2020_en).

The present project thus addresses a highly topical health policy issue. While other countries, such as the USA, now give high priority to measuring the quality of outpatient care in the area of child and adolescent health and promote it through centres of excellence,13 no data of this kind are available for Germany, so there is a clear research gap. Against this background, the aim of the present research project is to measure for the first time the quality of outpatient care for various common physical and psychiatric clinical conditions of children and adolescents in Germany.

Objectives

The study aims to contribute to an improvement of care within the framework of the German statutory health insurance (SHI) system on several levels: first, it will demonstrate for which of the studied clinical conditions and/or patients an improvement of quality of care is necessary. Second, the research project will point out starting points for interventions to improve the quality of care for children and adolescents within the framework of the SHI system, and thus contribute to the goal of improving the medical outcome on the patient side. Third, the quality indicators designed for the study can afterwards be implemented in regular care thereby enabling regular reporting of the outpatient care of this target group.

Methods and analysis

Overview

The project will initially develop and ratify quality indicators for seven common physical and mental childhood and adolescent clinical conditions with acute and chronic clinical condition characteristics (ADHD, asthma, atopic dermatitis, depression, otitis media, conduct disorder/oppositional defiant disorder and tonsillitis) using the RAND/UCLA Appropriateness Method (RAM).14

Subsequently, 1400 patient records (200 per clinical condition) of children and adolescents, who were diagnosed and/or treated for one of the abovementioned clinical conditions in one of 40 practices to be recruited (general practitioners, paediatricians, child and adolescent psychiatrists, paediatric out-of-hours services) in the German federal state of Hessen (with its 6.3 million inhabitants) during the time span from January 2018 to December 2019 will be drawn randomly and compared with the adherence to the quality indicators for the respective clinical condition. The percentage of adherence to quality indicators over all cases examined represents the primary outcome.

Inclusion criteria

Children and adolescents (0–18 years) with one of the abovementioned clinical conditions who received outpatient care at an SHI practice (specialties: general practitioner, paediatrics, child and adolescent psychiatry, paediatric out-of-hours services) in the federal state of Hessen during the years 2018 and 2019.

Study sample

A total of 1400 patient records (200 for each of the clinical conditions studied in this project) will be drawn from 40 randomly drawn outpatient practices (see above). The sample size planning is based on calculating the width of exact Clopper-Pearson CIs for a number of children and adolescents that can realistically be included in the study population in the planned period.15 16

On the basis of existing studies, it is assumed that children and adolescents with the studied clinical conditions have an average of about seven doctor contacts within 2 years (ie, in the studied period).17 The estimate of the percentage of quality indicators that will be assessed is thus based on 1400 (7×200) doctor contacts per clinical condition of interest. On the basis of the existing literature, it is assumed that the quality of care for the majority of the clinical conditions investigated ranges between 30% and 80%.4 Accordingly, 95% CI for possible percentages of quality indicators, assuming adherence will be between 30% and 80%, were calculated using the PASS 14 software (Power Analysis and Sample Size Software; NCSS, Kaysville, Utah, USA). Under the aforementioned assumptions, the width of the CI lies between 4.3% and 5.3%, which means that the high precision of the estimate required for the most accurate representation of the quality of care and the analysis of possible influencing factors can be achieved.

Primary endpoint

Percentage of quality indicators that are adhered to across all cases studied.

Secondary endpoints

Percentage of quality indicators that are adhered to, stratified by clinical condition, patient characteristics, specialty of outpatient practice and region (urban vs rural).

Proposed workflow

Development of disease-specific quality indicators

The development of own disease-specific quality indicators within the study is necessary, since no other quality indicators suitable for this study setting are available (existing indicators are kept very short to allow international comparability,18 are based on guidelines that have become obsolete5 19 or have been developed for specialised settings19). The development of independent quality indicators within the project also enables the validation of these indicators with regard to their intended use after the end of the project in routine quality measurement.

The RAM used for the development is state-of-the-art and was also used in the work of Braithwaite et al,3 Mangione-Smith et al4 and Ewald et al.18 20 First, a comprehensive literature analysis of existing quality indicators on the one hand and guideline recommendations for the diagnosis and therapy of the respective clinical conditions on the other hand will be carried out. On this basis, a preliminary set of indicators will be formulated, which is then reduced to a feasible and scientifically sound set in a two-stage consensus procedure. With regard to the project’s contribution to improving care, the RAM’s categorisation into appropriate care versus underuse or overuse is particularly noteworthy.

The development of quality indicators is divided into the following steps:21

  1. Identification of existing relevant guidelines/quality indicators through systematic literature analysis for each of the clinical conditions of interest.

  2. Derivation and formulation of specific recommendations from the results.

  3. Evaluation of the recommendations with regard to quality, feasibility/applicability (eg, ‘How likely is it that this recommendation is documented in a patient record’?) and importance by external reviewers (seven expert reviewers per clinical condition) within the framework of a modified two-stage Delphi process.

  4. Reduction of the recommended quality indicators based on the results of the Delphi process, reformulation into feasible quality indicators, classification of the quality indicators into phase of care categories (ie, diagnosis, acute therapy and ongoing management) as well as into quality type categories (adequate care, underuse or overuse).

  5. In a pilot phase, the quality indicators will then be tested for feasibility on 30 patient records from a single outpatient practice, and the interrater reliability will be determined. Depending on the results of the pilot phase, the selection of the quality indicators and their formulation will be adjusted.

Recruitment of healthcare providers

Starting in April 2021, a postal invitation, designed to elicit an optimum response,22 will be sent to randomly selected outpatient practices in the federal state of Hessen (first wave of invitations: 200 general practitioners, 100 paediatricians, 30 paediatric and adolescent psychiatrists, 5 paediatric out-of-hours services). After 2 weeks, a postal reminder is sent out, followed after another 2 weeks by a follow-up phone call. If the overall response is not sufficient 6 weeks after the start of the first wave, a second wave of invitations will be sent to previously uninvited practices, using the same methodology and the same number of practices per specialty.

Sampling

Fourty outpatient practices (ten outpatient practices per specialty) will be randomly selected from all practices interested in participating. During an initiation visit, a list of eligible patients will be drawn up by each practice for each of the seven clinical conditions to be examined. From the total number of eligible patients of all practices, 200 patients per clinical condition who have been treated in the years 2018 and/or 2019 will then be selected by stratified randomisation. In case that a substantial proportion of patient records (>10%) are not eligible even though the information systems indicate otherwise, up to two additional practices per specialty can be recruited.

The practices will extract the patient data of interest from the patient record and print or copy (in case of paper records) them. In these print-outs/copies, the practice then will make all information that will enable the patient to be identified unrecognisable.

This anonymised documentation of the respective patient records, including a basic data set (practice type, age and sex of the patient, postal code of the place of residence, socioeconomic status (approximated by insurance status), migration background (approximated by place of birth)) will then be transferred by the respective practice to the study nurse, who will check the data for eligibility and prepare them for the measurement of the quality of care. Study nurses’ background will be comparable to a nurse’s qualification, or to that of a medical assistant, respectively. Study nurses will be familiar with the documentation and remuneration system in paediatric primary care in Germany. The planned end date for the sampling is November 2022.

Review of the quality of outpatient care

Surveyors will be provided with a detailed manual that outlines the clinical conditions, quality indicators, definitions, criteria and processes for conducting patient record reviews. The prepared documentation from the patient records will be reviewed independently by two members of the study team for eligibility of the recorded quality indicators (yes vs no) and for adherence (yes vs no) to the quality indicators for the respective clinical condition. In the case of disagreement between the two surveyors, one of them will double-check diverging documentation, and will bring about consensus. The results will be then entered into a database. Additionally, kappa scores will be calculated for both eligibility and adherence. These will undergo a regular assessment in order to provide an ongoing quality assurance.

Data analysis

Subsequently, the data on the quality of care will be analysed, with the primary endpoint being the percentage of quality indicators adhered to across all cases studied. Secondary endpoints are the respective percentage of quality indicators adhered to, stratified by clinical condition, patient characteristics, type of care provider and region (urban vs rural). To estimate the percentage of quality indicators that are adhered to, exact Clopper-Pearson CIs will be calculated. The exploratory evaluations of possible factors influencing the quality of care (patient characteristics, type of illness, type of outpatient care provider, degree of urbanisation of the region being cared for) are carried out within the framework of multilevel models (superordinate level: care provider, middle level: patients, subordinate level: patients’ contacts with physicians). Depending on the respective outcome variable, generalised estimation equation models or linear mixed models will be used.

Data management and data protection

The data from the patient records will be made available to study nurses in anonymised form on the premises of the participating healthcare providers, and will be then processed in accordance with national data protection regulations. All data analyses will include plausibility checks, and will be carried out exclusively on anonymised data sets without linking and access to primary data. Both the data from the patient records and from the focus groups will be entered into a database on password-protected devices and stored on the device with password protection. Three years after the final data analysis, the data will be deleted.

Patient and public involvement statement

Due to the nature of the study, it was not appropriate or possible to involve patients or the public in the design, or conduct, or reporting, or dissemination plans of our research.

Discussion

Study design considerations

In order to ensure comparability, the study largely leans on the study by Braithwaite et al3 21 23 in terms of quality indicator development and outcomes; deviations from their study design are due to peculiarities of the German health system. The primary criterion for the selection of the clinical conditions of interest was a high relevance to healthcare in terms of a high prevalence in outpatient care, a high burden of disease or a considerable risk of chronicity.3 17 24 Relevant conditions in the field of child and adolescent psychiatry include ADHD, anxiety disorders, conduct disorder, depression, autism and anorexia, and in the field of paediatrics asthma, atopic dermatitis, fever, gastrointestinal complaints, headache, otitis media, tonsillitis and upper respiratory tract infections.

Within this selection, a mix of somatic and psychiatric disorders as well as acute (eg, asthma, otitis media) and chronic conditions (eg, atopic dermatitis, depression) was aimed for in order to cover the broadest possible clinical spectrum.

The age of the target group was deliberately chosen from the neonatal phase to adolescence in order to be able to include different manifestations and courses of disease in the evaluation.

The study setting in the field of ambulatory care is of particular interest for the project: outpatient primary care providers are the first point of contact for the majority of children and adolescents with the clinical conditions relevant for this project. In view of the intended strengthening of general practitioners and paediatricians and adolescents’ primary care in Germany, this role will become increasingly important in the future.9 25

The federal state of Hessen was selected as the project region because Hessen represents a balanced mixture of regional metropolis (n=10), regional centres (n=98) and rural regions (HE: 15%; all data for the year 2017 (inkar.de)). In addition, Hessen (HE) is close to the German (DE) average in terms of fertility rate (HE: 1.58, DE: 1.57), per capita income (HE: 3130 EUR, DE: 2845 EUR) and population density (HE: 296/km², DE: 232/km²), and its above-average proportion of non-German inhabitants (HE: 15.7%, DE: 11.7%) guarantees an appropriate representation of this patient group in the sample.

As of 2021, there are about 3200 general practitioners, 470 paediatricians, 90 child and adolescent psychiatrists and 11 paediatric out-of-hours outpatient practices in Hessen.

The selected specialties of the outpatient care providers are those most frequently involved in the care of children and adolescents with physical or mental health problems and are thus most suitable for depicting the breadth of outpatient care.

Limitations

Data from hospital (emergency) outpatient departments were not included for this study, as these belong to a different healthcare sector, which would exceed the scope and horizon of this study.

Also, due to economic and time constraints, we only evaluate quality of care for a limited number of clinical conditions and for a limited number of outpatient practices, which limit the generalisability of our findings. Finally, as our study only relies on medical records, the perspective of patients is lacking.

Ethics and dissemination

The study was approved by the ethics committee at the Landesärztekammer Hessen, Frankfurt, Germany (14 January 2021; reference number 2020–2026-evBO), including a waiver of consent for use of the anonymised patient data. In the case that the analysis of anonymised patient record data reveals serious diagnostic or therapeutic errors, the medical study lead will inform the respective practice lead. In the case of gross misconduct or harm evident from the anonymised data, law enforcement agencies will be involved.

The quality indicators designed for the study can afterwards be implemented in regular healthcare monitoring, thus enabling regular reporting of the outpatient care of children and adolescents with conditions analysed in this study.

The authors plan to disseminate their findings through scientific publications in national and international peer-reviewed journals, and through presentations at national and international conferences in the field of paediatrics, child and adolescent psychiatry, and health services research.

Ethics statements

Patient consent for publication

Acknowledgments

The authors thank Tim Bluthardt, MSc, for his contribution to the inception and management of this research project, and Professor Peter Hibbert, Macquarie University, for helpful advice regarding the study design and study protocol.

References

Footnotes

  • Contributors VT wrote the draft of the manuscript and made substantial contributions to the study design. CJB designed the study, and was a major contributor in writing the manuscript. MG made significant contributions to the study design and the writing of the manuscript. MH-G designed the analysis plan and carried out the sample size calculation. DE and CM contributed to the study design. All authors read and approved the final manuscript. MG and CJB are shared last authorship.

  • Funding The work is supported by Gemeinsamer Bundesausschuss (G-BA) (grant number: 01VSF19035; recipient: CJB).

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.