Article Text

Download PDFPDF

Study protocol for a quasi-experimental claims-based study evaluating 10-year results of the population-based integrated healthcare model ‘Gesundes Kinzigtal’ (Healthy Kinzigtal): the INTEGRAL study
  1. Ingrid Schubert1,
  2. Achim Siegel2,
  3. Erika Graf3,
  4. Erik Farin-Glattacker4,
  5. Peter Ihle1,
  6. Ingrid Köster1,
  7. Dominikus Stelzer3,
  8. Claudia Mehl3,5,
  9. Jutta Schmitz5,
  10. Patrik Dröge6,
  11. Christian Günster6,
  12. Andreas Klöss6,
  13. Werner Vach3,7,
  14. Max Geraedts5
  1. 1 PMV Research group at the Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University of Cologne, Cologne, Germany
  2. 2 Institute of Occupational and Social Medicine and Health Services Research, University Hospital Tübingen, Tübingen, Germany
  3. 3 Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center - University of Freiburg, Freiburg, Germany
  4. 4 Institute of Medical Biometry and Statistics, Section of Health Care Research and Rehabilitation Research, Faculty of Medicine and Medical Center - University of Freiburg, Freiburg, Germany
  5. 5 Institute for Health Services Research and Clinical Epidemiology, University of Marburg, Marburg, Germany
  6. 6 Health Services and Quality Research, AOK Research Institute (WIdO), Berlin, Germany
  7. 7 Department of Orthopaedics and Traumatology, University Hospital Basel, Basel, Switzerland
  1. Correspondence to Dr Erika Graf; egr{at}


Introduction Patients often experience interface problems when treated by different specialists and in different healthcare sectors. Integrated care concepts aim to reduce these problems. While most integrated healthcare models focus on individual diseases, the integrated care model ‘Gesundes Kinzigtal’ applies a population-based approach and addresses the full spectrum of morbidities for a population defined by area of residence—the Kinzigtal. A special feature of the model is the joint savings contract between the regional management company and the statutory health insurers. The INTEGRAL study aims at assessing the effectiveness of ‘Gesundes Kinzigtal’ under routine conditions in comparison to conventional care over a period of 10 years in order to understand the benefits but also the potential for (unintended) harms.

Methods and analysis Database

Claims data from statutory health insurance funds 2005–2015. The evaluation consists of a quasi-experimental study, with Kinzigtal as intervention region, at least 10 further regions with a similar population and healthcare infrastructure as primary controls and an additional random sample of insurees from the federal state of Baden-Württemberg as secondary controls. Model-specific and ‘non-specific’ indicators adopted from the literature and enriched by focus group interviews will be used to evaluate the model’s effectiveness and potential unintended consequences by analysing healthcare utilisation in general. Temporal trends per indicator in the intervention region will be compared with those in each control region. The overall variation in trends for the indicators across all regions provides information about the potential to modify an indicator due to local differences in the healthcare system.

Ethics and dissemination Ethic Commission of the Faculty of Medicine, Philipps-University Marburg (ek_mr_geraedts_131117). Results will be discussed in workshops, submitted for publication in peer-review journals and presented at conferences.

Trial registration number DRKS00012804.

  • integrated health care
  • evaluation
  • indicators
  • claims data
  • regional variation

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Strengths of the present evaluation study include its long observation period and comparisons of the intervention region with regions similar in population and healthcare infrastructure, which allows to estimate regional variation as well as the effect of the integrated care model.

  • The indicators relevant for this assessment will be developed in a structured process independent of the evaluation.

  • Another positive feature is the use of ‘non-specific indicators’ to reveal unintended consequences of the integrated care model and joint savings contract.

  • Limitations are those usually associated with collecting claims data, namely, the occurrence of diseases can only be documented and validated internally using the routine data-collecting tools available, patient-reported outcomes (ie, regarding lifestyle, quality of life, the perception of patient-centred care), data on medical examinations and laboratory findings are not accessible.

  • Moreover, only those services covered by statutory health insurance providers were documented, so that (few) services paid for by the patients themselves were not considered.


Healthcare provision in Germany today is mainly divided into outpatient care (general practitioners (GPs) and specialists), hospital care and rehabilitative care. These so-called ‘sectoral silos’ can be problematic due to their lack of exchange between stakeholders and even lead to poor health outcomes. ‘Integrated care’ has the potential to address these deficits using new structural approaches beyond the current way of service provision. Close cooperation between GPs, specialists, hospitals and other healthcare stakeholders is intended to lead to more patient-oriented care and cross-sectoral communication. Integrated care aims to improve the quality and cost-effectiveness of healthcare compared with today’s situation.1

The integrated care model ‘Gesundes Kinzigtal’ (ICM-GK) is considered a best practice example in Germany2 and internationally3 4 not least due to its population-oriented approach. Compared with other existing models based on the same contractual approach (so-called selective contracting) which focus on integrated care for selected diseases, ICM-GK addresses the full spectrum of morbidities and health issues for a population defined by residential area (with the only exception of dental care). The contract was concluded in early 2006 between two partners: the Gesundes Kinzigtal GmbH management company (a joint company founded by the ‘Medizinisches Qualitätsnetz Ärzteinitiative Kinzigtal e.V.’, a regional physicians’ network and OptiMedis AG, a management and holding company specialised in integrated care) and the AOK Baden-Württemberg (BW), the largest statutory health insurance fund in the federal state of BW. It is a population-based integrated care contract according to § 140—SGB V (Sozialgesetzbuch V, Book V of the German Social Security Code) as of 1 November 2005. Several months after conclusion, the Landwirtschaftliche Krankenkasse, a further health insurance, joined the contract. The contract covers the Kinzigtal region, which is located in the Black Forest in Southwest Germany and home of about 70 000 people, about 33 000 of whom are insured with the two statutory health insurers that are contract partners. The insurees, doctors and other providers can choose whether they want to join the contract. Even those insurees who decide to enrol into ICM-GK retain the option to visit doctors and other providers who are not part of the contract. Within ICM-GK, patients are entitled to individual targeted, integrated care which focusses on prevention and quality of life for people with chronic diseases. There are no direct financial incentives for insurees to join the contract. A key goal of the ICM is the participation (on different levels) and activation of patients: A patient advisory board consisting of several members elected from (and by) the insurees is part of many decision-making processes of the ‘GK’ management.5 The patient advisory board elects a patient ombudsman who represents patient interests and mediates in case of conflicts.6–8 ICM-GK aims to improve quality and efficiency in healthcare by dedicated investments in new activities which improve public health or patient care in the long run but simultaneously reduce costs. This is achieved by means of two strategies:

  1. Employing both target group-specific and general prevention and healthcare programmes to reduce incidence and prevalence of morbidities or to delay disease progression.

  2. Managing intersectoral interfaces (in particular between outpatient and inpatient care) in order to improve patient management.

The contract between the management company and the statutory health insurers includes a so-called joint savings contract, that is, the healthcare cost savings achieved are distributed between the contractual partners.2 6 Savings are calculated as the difference between the actual healthcare costs and the funds provided to the statutory health insurers to ensure service coverage, which in turn is based on the morbidities prevailing in the region (‘morbidity-oriented risk structure compensation scheme’). The calculation of the savings is based on all insurees of both statutory health insurers located in the Kinzigtal, not only those who have enrolled into ICM-GK. Inter alia, this serves to avoid a selection bias in favour of insurees of greater health.9 Since a joint savings contract can potentially incentivise lower levels of care, that is, an underutilisation of health services,10–12 an evaluation of the healthcare quality of the model is of high relevance.5

The model was built up in several steps between 2006 and 2010. An initial milestone of 8000 insurees of AOK BW joining was reached in 2011. The start-up phase was accompanied by an evaluation comprising several modules.13–18 Another external evaluation study of the model also had its primary focus on the start-up phase.17 Generating knowledge about the effectiveness of an integrated care project under routine conditions (ie, after the completion of the start-up phase in which the commitment of the stakeholders is extraordinary) is of high relevance for all population-based integrated care programmes and physicians’ networks in order to understand the true benefits, but also the potential for (unintentional) harm.

Research aims

This study protocol describes the evaluation of both the start-up and consolidation phase of the ‘GK’ model, with special focus on the latter: In order to assess differences to conventional routine care, the ICM should be analysed in its routine practice after the completion of the start-up phase, which is now possible for the first time.

The evaluation uses claims data from the statutory health insurer AOK BW covering the period 2005–2015. The evaluation aims to answer the following questions:

  1. Which indicators can be calculated from claims data in Germany in order to measure differences in patients’ treatments and outcomes between intervention group and control groups, with regard to ICM-GK prevention and treatment programmes (ICM-GK programme-specific indicators) as well as healthcare utilisation and healthcare of the populations under study in general (ICM-GK programme-unrelated non-specific indicators).

  2. Has the quality of healthcare provided on the basis of the joint savings contract remained stable or improved compared with its baseline level in 2005?

  3. How does the development of healthcare quality during the start-up phase (2006–2010) compare to the development during the consolidation phase (2011–2015)?

  4. Does ICM-GK succeed in avoiding underutilisation of health services although ICM-GK’s revenues are based on a joint savings contract?

We expect that the comprehensive set of study indicators based on claims data can also be used for healthcare monitoring in other ICMs.

Methods and design

Study design

The evaluation consists of a quasi-experimental study, with the Kinzigtal region as the intervention region and at least 10 other regions with a similar population and healthcare infrastructure as primary controls and an additional random sample of insurees from the federal state of BW as secondary controls. Figure 1 depicts the study concept.

Figure 1

Study concept for the evaluation of the integrated care model ‘Gesundes Kinzigtal’. IC, integrated care. 

Work package A: indicator development

A set of quality indicators will be developed to assess the development of quality of care within the study region, and between the study region and the control regions, during the observation period. Based on a mixed-methods approach, focus group interviews with stakeholders are combined with a systematic literature review and consensus decision-making.

We will develop ICM-GK programme-specific quality indicators to assess ICM-GK goal attainment, and ICM-GK programme-unrelated, non-specific quality indicators to capture potentially unintended consequences of the ICM. The indicator development uses, among others, Kessner’s tracer concept,19 the Health Care Quality Indicator criteria of the Organisation for Economic Co-operation and Development (OECD-HCQI)20 to assess the performance of the health system, and as amended by Fung et al 21, criteria for public reporting initiatives. Thus, above all, indicators should capture effectiveness, safety, patient orientation and unintended consequences of healthcare interventions.

Analysing all ICM-GK programmes executed during the observation period will be the starting point for the development process. We will then conduct focus group interviews (cf. below, module A1) and a systematic review of published specific and non-specific indicators (module A2–A3). Finally, we will decide on the set of indicators for evaluation through consensus decision-making (module A4).

Module A1: Focus group interviews: In order to shed light on stakeholders’ views on ideal concepts of integrated care and potentially unintended consequences, we will perform semistructured guided interviews. Six stakeholder groups with 4–6 participants each will be interviewed, consisting of patients from the study region (1)/non-study region patients (2), healthcare providers from the study region who are (3)/are not (4) members of the ICM-GK provider network/non-study region providers (5) and programme managers and participating sickness funds (6). The interviews will be recorded and transcribed. Potential indicators will be extracted.

Module A2: Development of ICM-GK programme-specific indicators: First, we will analyse programme goals and recommended treatment processes of all ICM-GK programmes carried out during the observation period. Second, appropriate indicators to evaluate these programmes will be developed using (1) clinical practice guidelines focusing on the diseases addressed by the ICM-GK programmes; (2) quality indicator databases (eg, National Quality Measures Clearinghouse,22 The Ambulatory Care Quality Alliance,23 RAND,24 National Quality Forum Quality and Outcomes Framework,25 District Health Board New Zealand,26 OECD Healthcare Quality Indicators27 and the German databases AQUIK,28 QUINTH,29 and QISA30; (3) a review of articles published in PubMed, Cochrane Library, Embase, Web of Science containing the search terms ‘quality indicator*’ and ‘integrated care’; (4) programme-specific indicators mentioned in the focus group interviews. Two independent reviewers will screen abstracts and full texts of articles, guidelines and QI databases for indicators suitable to measure the quality of programme-specific processes and outcomes of care. We will search the mentioned databases for English and German articles without time limit. Our focus will be on indicators assessing integrated care, health promotion and prevention. We will exclude indicators focusing on practice management and in-hospital care. All potential indicators will be extracted and entered into a database using the scheme developed by the Joint Commission on Accreditation of Healthcare Organizations (1990)31 to describe the indicators. We will eliminate duplicates and check whether the respective indicator could be calculated using routine claims data of German sickness funds. The final list of suitable indicators will be assessed by the consensus panel (see module A4).

Module A3: Development of ICM-GK programme-unrelated non-specific indicators: Indicators for the assessment of healthcare utilisation and the health status of the regional intervention and control populations will serve to identify potential underuse or overuse of services that are not in the focus of ICM-GK programmes in the Kinzigtal region. Therefore, we will use OECD indicators (, the frequency of disease groups classified according to the International Classification of Diseases (ICD, German modification), prescription drug categories (Anatomical Therapeutic Chemical (ATC) groups), and outpatient or inpatient procedures conducted in the intervention and control regions. We will differentiate between age groups, gender and people with/without multimorbidity.

Module A4: Consensus decision-making: A panel of 10 participants will be invited to finally assess the validity and feasibility of the indicators. As participants we will choose two healthcare providers of the ICM-GK network, two patient representatives of ICM-GK, one sickness fund representative, three members of the evaluation team who were not involved in indicator development and two quality indicator experts. We will use a modified RAND/UCLA Appropriateness Method28 and provide participants with information on the indicator development methodology and the consensus process at a first meeting. Participants will receive an online or print version of the indicator set to rate the validity and feasibility of the indicators on a 9-point Likert scale. In a subsequent face-to-face meeting, the participants will be invited to discuss the summary ratings of indicators and comments on which participants did not agree. Participants will then rate the remaining indicators a second time. Appropriate indicators without disagreement will constitute the final indicator set.

The indicators prespecified in work package A will be later supplemented by a data-driven statistical search considering a wide range of non-prespecified indicators based on diagnoses, prescriptions and procedures.

Work package B: claims data analysis of indicators for evaluation

Database and observation period

The evaluation is based on retrospective claims data of the major health insurer involved, the AOK BW. Anonymised data will be provided by the AOK Research Institute (Wissenschaftliches Institut der AOK—WIdO). In addition to master data (with information, eg, concerning age, gender, insurance status and period of insurance), information on the use of all sectors of healthcare (outpatient care, hospital care, drug prescriptions, benefits in kind, long-term care) is available and can be linked using a non-identifiable study number. For the analysis, ICD-10 coded diagnoses—available since the year 2000—from outpatient and inpatient care are provided, further medical services according to the ‘Einheitlicher Bewertungsmaßstab’ code (EBM, physician fee schedule), drug prescription with pharmaceutical registration number and linkage to ATC classification and defined daily dose, hospital stays with, for example, ICD-10 coded diagnoses, procedures according to the Operations and Procedure Code (‘OPS’ codes) and length of stay, benefits in kinds, information concerning inability to work (diagnosis, duration) and utilisation of long-term care.

Data are provided for the years 2005–2015 with 2005 as reference year, that is, the year before the start of the integrated care programme. The ICM-GK project defined a number of 8000 enrolled patients as a precondition to open the programme for other sickness funds. This number was reached in 2011, therefore, we will take the years 2006–2010 as the start-up phase, 2011–2015 as the consolidation phase. Furthermore, the increase of enrolments has remarkably slowed down from 2011, showing that enrolment dynamics is another feature suggesting that we may differentiate those two development phases.

Target population and control populations

The target population consists of all AOK insurees living in the intervention region irrespective of any enrolment—this results from the conception of the ICM-GK as a regional population-based healthcare system covering virtually all healthcare sectors and health conditions. AOK insurees are assigned to the intervention population (‘Kinzigtal population’) if the postal code of their place of residence encodes a place within the Kinzigtal region (German postal codes 77709–77797 and 78132). The target population consists of about 30 000 AOK insurees in 2005 and of nearly 32 000 insurees in 2015, and they will be surveyed completely.

Control populations (from regions characterised by ‘conventional’ or ‚‘usual care’) are necessary to check whether the developments observed in the intervention region are actually specific for the latter and may thus be attributed to ICM-GK activities or whether they correspond to a general trend or a small-scale variation pattern which may also occur under other circumstances. For this purpose, we will analyse two types of control populations: first, we will consider all AOK insurees from at least 10 control regions (identified by distinct postal codes); these regions shall be structurally similar to the Kinzigtal region. A structurally similar control region should meet the following requirements: it should be a geographically contiguous area (1) containing only rural communities, small towns or—at most—small medium-sized towns with less than 50 000 inhabitants each; (2) it should preferably be characterised by a river valley, comparable to the Kinzigtal region; (3) important socioeconomic and health service indicators of a control region should not deviate too much from its counterpart in the Kinzigtal region. As important socioeconomic and health service indicators we will consider (1) unemployment rate in 2005–2007, (2) income tax per inhabitant in 2005–2007, (3) commuter flow in 2014, (4) proportion of foreign residents in 2015, (5) proportion of employees with academic education in 2015, (6) proportion of employees without training qualifications in 2015, (7) average distance to the closest hospital (in minutes by car), (8) number of inhabitants per office-based GP in 2015 and (9) number of inhabitants per office-based physician or psychotherapist in 2015. Additionally, we will check whether there is an active network of physicians in a given region—a network comparable to the one which was active in the intervention region at the time when the integrated healthcare system was launched. (We assume that such a network might contribute to an above average quality of healthcare.) Our goal is to select 50% of control regions with such a network and 50% without. Apart from this control population, we will draw a random sample of about 500 000 AOK insurees residing in the German federal state of BW, but outside of the Kinzigtal region. The comparison with the first type of control groups will be our primary comparison, the one with the random sample a secondary comparison.

Before operationalising and analysing the study populations, we will investigate how many AOK insurees have not been insured continuously throughout a given year or whether insurees were not resident in the region concerned all year round. Depending on this investigation’s results, we will settle the final criteria for the inclusion in the study population.

Health services for AOK insurees in the federal state of BW outside of the Kinzigtal region are currently largely characterised by family doctor-centred healthcare—an AOK programme which also aims at a higher quality of healthcare and requires insurees to enrol, thereby choosing their family doctor. Unfortunately, for some health services, this implies a lack of data regarding these patients, as they are covered by a general fee for the GP. Therefore, we have to exclude these patients from the analysis of some indicators.

Operationalisation of indicators for data analysis

In a first step, we operationalise the indicators consented in work package A for the routine data analysis. For indicators expressed as a percentage of a target population, the nominator and denominator have to be defined with the information available in claims data. For each target population, the inclusion and exclusion criteria have to be determined (eg, ICD-10 codes, validation criteria, insurance period). Indicators will address processes of care as well as outcomes. For the process-related indicators, for example, billing codes for outpatient care services (EBM codes) and inpatient services (OPS codes) will be used for assessment. Outcomes are mostly clinical events which can be mapped with diagnoses. Other outcomes refer to the utilisation of special services such as long-term care, palliative care or rehabilitation. All-cause mortality will be assessed using variable ‘death’ as the reason for leaving the sickness fund.

Statistical considerations

In the descriptive part of the study, we present the percentage of the insured persons or the specific target population fulfilling the respective indicator. For the inferential part, our basic approach is to determine for each indicator and each region the temporal trend in the indicator and to compare the trend observed in the intervention region ‘Kinzigtal’ with the trends in the control regions. The overall variation in trends across all regions provides information about the potential to modify an indicator due to local differences in the healthcare system. If there is some variation and if the trend in the intervention region is smaller (or larger) than in all or in the vast majority of control regions, we will regard this as an indication of a specific situation in the intervention region, which is likely to be causally related to the ICM-GK.

The estimation of the temporal trends will be based on combining regression models for the individual patient data with a standardisation for the population of BW based on the full random sample from the latter. The choice of the regression models will depend on the type of indicator. For binary indicators, we use logistic regression models, for other types, we select the model accordingly.

In our analytical approach, the following issues will be taken into account:

  • Changes in population over time: The populations in the regions will change over time due to migration, fertility and mortality. We aim at including at each time point all patients living in the respective region in order to avoid any selection effects. The fact that the same patients will contribute to the same indicator at different time points will be taken into account when assessing statistical significance.

  • Baseline differences across regions: We avoid any assumptions about similarity across regions at baseline by using regions as fixed effects in all analyses. In spite of the structural similarity of all regions, we have to expect differences in the distribution of age, gender, comorbidity and social status. Consequently, we will adjust all analyses for the first three factors at the individual level and for the fourth factor at the postal code level.

  • Global time trends and structural changes: Due to using several control regions as well as a sample from the whole BW population, we are able to detect global time trends as well as global structural changes over time, for example, due to administrative changes in the healthcare system. We can account for this by using the calendar year as a categorical covariate in all analyses, and hence require the linearity of region specific trends only in addition to a potentially non-linear global trend.

  • Time window for trends and stability of trends: For each indicator, we will a priori define a starting point at which we expect a specific change in the time trend (eg, due to the start of a specific programme) as well as the period over which we expect the change to continue. If the starting point lies within the first 3 years of our observation period, we will also analyse the stability of the trend over time, in particular whether we can find evidence for an attenuation of the trend.

  • Assessing a specific role of the intervention region: In order to assess a potential specific role of the intervention region, we will visualise trend estimates of all regions in a forest plot as well as in a dot plot. A formal assessment will be based on assessing the statistical significance of a deviation of the intervention region from the mean of all control regions and from the BW region and on the representation of the deviation of the intervention region from the mean of the control regions as a z-score, that is, in the unit of the SD of the variation across the control regions.

  • Floor and ceiling effects: Some regions may for some indicators be already close to the maximum or minimum we can expect. Such circumstances can distort the interpretation of trends. We will take this into account by performing additional analyses which will give more weight to the control regions which are initially similar to the intervention region.

  • Provider effects: Indicators reflecting an action by a healthcare provider may be prone to provider effects, that is, the main source of variation may be differences between providers. We will take this into account when assessing statistical significance whenever the action can be assigned to a healthcare provider (typically the GP) in our data. In addition, provider variation within regions and their temporal trends will be described and the intervention region will be compared with the control regions.

  • Reducing trends to a single number: Such a reduction is necessary in order to allow the necessary comparisons between regions. However, this may fail to take a more complex development into account. We will address this issue by always additionally visualising the raw data behind each estimated trend.

  • Multiplicity: By comparing the trend estimate in the intervention region simultaneously with all trends in the control regions, we avoid multiple testing when analysing a single indicator. It remains to be borne in mind that we analyse a large number of indicators for a possible specific role of the intervention region. We approach this by assessing global measures such as the number of indicators hinting towards such a specific role or the average difference from the control regions. We will do this in a hierarchical manner, taking into account prespecified groupings of the indicators, reflecting the suspicion that some indicators might reflect the same signal.

  • Ranking of non-prespecified indicators: A huge number of non-prespecified indicators will be investigated with the aim to identify the most relevant potential signals. Here, we will make use of statistical methods, which have been successfully applied in analysing signals for unknown side effects of drugs based on routine data.32

Details of the analytical approach will be fixed in a statistical analysis plan to be finalised prior to starting the analyses.

Data will be stored on MS-SQL Server 2014 under Windows Server 2012. The analysis will be performed with SQL, SAS for Windows Release V.9.3 (SAS Institute) and Stata V.15.1 (StataCorp. Stata Statistical Software: Release V.14). The use of claims data follows the Guideline for Good Practice of secondary data analysis.33 Essential parts are contracts with data owner and the regulations for data privacy.

Patient and public involvement

Patients and public were not involved in the study design nor will they be involved in the recruitment and conduct of the study. Patient will be involved in focus groups addressing aspects of integrated care and in the consensus panel assessing validity and feasibility of the indicators for evaluation.

Ethics and dissemination

All participants in the focus groups (work package A, cf. figure 1) will give their informed consent, which can be withdrawn at any time during the study. The analysis of the meetings and the presentation of the results (work package C; cf. figure 1) will be anonymous. Participants will not be identified in any publication.

Access to the claims data is regulated by a contract between the AOK Research Institute (WIdO) and the researchers who analyse the data. Claims data for the evaluation analysis are provided in an anonymised manner, therefore, no informed consent is necessary. The internal project study number does not allow any reidentification of the insurees.

After completion of the project, a workshop with relevant stakeholders and participants of the focus groups is planned in order to discuss the results (work package C, cf. figure 1) and to start a process for disseminating results and transferring the methodology used to evaluate an ICM. A study report with an executive summary will be produced and will be made available for those contributing to the study and other interested parties.

Besides, results of the study will be presented at scientific conferences and submitted for publication in peer-reviewed journals. The indicators and transfer to routine data analysis will be provided (eg, via an electronic platform) for those interested in the evaluation of the service quality of population-based integrated care.


A solid and thorough assessment of ICMs is essential to evidence-based healthcare. A particular feature of this project is the long observation period, which is made possible by using routine data from the statutory health insurance funds. Although investigating the effects of complex interventions by relying on routine data entails certain limitations, it remains a reasonable and acceptable procedure. For example, the pay-for-performance programme,34 35 preferred provider organisation settings36 and patient-centred medical home37 38 were assessed with administrative data. In Germany, data from health insurance funds were used to evaluate disease management programmes39 40 and family doctor-centred care (Hausarztzentrierte Versorgung)41–44 as well as models of integrated care,45 46 not least because of their advantages such as availability for long periods of time without additional data collection, no selection, interviewer or recall bias—thus reflecting everyday practice of healthcare.

Our study aims to show whether the ICM-GK’s standard of care is at least equivalent, better or worse than that of ‘conventional’ or ‘usual care’ during the consolidation phase of ICM-GK. This project reveals evidence for the design of the population-based integrated care contract for the ICM-GK, and for health insurers and other stakeholders of healthcare structures in Germany. Should this evaluation reveal weaknesses in certain areas (such as underuse or inadequate care), similarly structured types of care involving selective contracts could make it possible to take countermeasures (ie, committing to continuous and prompt monitoring of care by employing specific codes or the obligatory publication of results as well as the redrafting of certain contractual regulations such as joint savings contracts).

The indicators developed here can also be employed to control quality and managed healthcare in other types of integrated care, and for monitoring the provision of standard care. The development process of the indicators, involving relevant stakeholders, ensures their relevance for the practice and for healthcare provision itself.


The authors would like to thank the AOK Baden-Württemberg for data access and Susanne Sollmann (WIdO) for her help with translating the protocol. The article processing charge was funded by the German Research Foundation (DFG) and the University of Freiburg in the funding programme Open Access Publishing.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.


  • WV and MG contributed equally.

  • IS and AS contributed equally.

  • Contributors MG, EF-G, EG, IS, AS and WV are principal investigators and responsible for the study design and project management. CG, PD, AK, IK and PI are responsible for data provision, data collection and data management. IK, PI, DS, WV and EG are responsible for the statistical analysis. AS and WV for the concept of defining control regions. MG, JS and CM are responsible for the indicator development. IK and IS for the operationalisation of indicators with routine data. All authors reviewed and approved the final version of the study protocol.

  • Funding The project is funded by the Innovation Committee of the Joint Federal Committee after peer review by the scientific advisory board under the grant number 01VSF16002.

  • Competing interests AS declares involvement in former studies on Gesundes Kinzigtal GmbH (2006–2015) and an employment at Gesundes Kinzigtal GmbH (1 June 2015 until 31 December 2015). IS, IK and PI declare that they were involved in one former study evaluating the start-up phase (2006–2011) of the integrated care model ‘Gesundes Kinzigtal’. All authors report grants from the Innovation Committee of the Joint Federal Committee, during the conduct of the study.

  • Patient consent Not required.

  • Ethics approval Ethical approval has been obtained from the Ethic Commission of the Faculty of Medicine, Philipps University Marburg (ek_mr_geraedts_131117).

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Technical appendix, details on statistics, and data tabulations will be made available from