Objective To explore the methodologies employed in studies assessing transition of care interventions, with the aim of defining goals for the improvement of future studies.
Design Systematic review of comparative studies assessing transition to adult care interventions for young people with chronic conditions.
Data sources MEDLINE, EMBASE, ClinicalTrial.gov.
Eligibility criteria for selecting studies 2 reviewers screened comparative studies with experimental and quasi-experimental designs, published or registered before July 2015. Eligible studies evaluate transition interventions at least in part after transfer to adult care of young people with chronic conditions with at least one outcome assessed quantitatively.
Results 39 studies were reviewed, 26/39 (67%) published their final results and 13/39 (33%) were in progress. In 9 studies (9/39, 23%) comparisons were made between preintervention and postintervention in a single group. Randomised control groups were used in 9/39 (23%) studies. 2 (2/39, 5%) reported blinding strategies. Use of validated questionnaires was reported in 28% (11/39) of studies. In terms of reporting in published studies 15/26 (58%) did not report age at transfer, and 6/26 (23%) did not report the time of collection of each outcome.
Conclusions Few evaluative studies exist and their level of methodological quality is variable. The complexity of interventions, multiplicity of outcomes, difficulty of blinding and the small groups of patients have consequences on concluding on the effectiveness of interventions. The evaluation of the transition interventions requires an appropriate and common methodology which will provide access to a better level of evidence. We identified areas for improvement in terms of randomisation, recruitment and external validity, blinding, measurement validity, standardised assessment and reporting. Improvements will increase our capacity to determine effective interventions for transition care.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study focusing on the methodological issues of studies assessing transition of care interventions.
An extensive review of the published literature and new research was achieved.
Original methodological recommendations were proposed to advance the quality of future research designs.
Qualitative studies were excluded; studies in progress were included but may not represent the full range of such studies
A specific risk of bias assessment tool is missing to evaluate the level of bias of non-randomised studies in the field of complex intervention. We discussed the various sources of bias without using a specific tool.
In the course of their care, young people with chronic conditions undergo a transition from paediatric to adult care services. The goal of transition is to ‘maximise lifelong functioning and potential through the provision of high-quality, developmentally appropriate healthcare services that continue uninterrupted as the individual moves from adolescence to adulthood’.1 It is recommended to plan a longitudinal process to prepare young people to become autonomous, enabling them to assume responsibility for their chronic care management after transfer to adult services.2 Dysfunctions in adaptation to adult care have been identified,3 ,4 which lead to significant adverse health consequences through lack of continuity of care or resort to emergency services.5 ,6 Many studies have highlighted these consequences of transition failure for different chronic conditions, such as increased organ rejection,7 ,8 the rise of glycated haemoglobin in young people with type 1 diabetes mellitus (T1DM)9 or the increase of disease activity in chronic rheumatic disease.10 From the perspective of psychosocial impact, it has been reported that poor metabolic control at transition is associated with higher risk of drop-out and psychosocial morbidity in T1DM,11 and that satisfaction with transitional care is related to social and emotional quality of life (QOL) over time.12
In light of these issues, the past 20 years have seen many programmes that attempted to improve the transfer of young people to adult medicine. However, in the absence of a reliable methodology, evidence-based data are rare.13 ,14 Recently, systematic reviews have shown the diversity of transition interventions15 and the heterogeneity of the criteria16 used to evaluate them. These are complex interventions, defined as those which include several interacting components, have variable outcomes and present problems for evaluators, in addition to practical and methodological difficulties.17 In 2011, significant international groups were formed to work on the problems posed by transition, and to make recommendations for interventions,2 ,18–20 while other groups argued for the need for improved assessment.21 ,22 In view of the increasing number of publications on transition interventions we decided to review methodological problems in transition research. Our aim was to look at the methodology used in these studies, and to highlight achievements and areas of focus for efforts to build up a body of literature based on a high level of evidence in the field of healthcare transition.
The review is reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement.23 The systematic review followed the PRISMA guidelines (see online supplementary appendix 1: Completed PRISMA 2009 Checklist).
Literature search: MEDLINE, EMBASE and ClinicalTrial.gov were searched without any language restriction to identify studies published up to the 1 July 2015. The electronic search strategies are described in online supplementary appendix 2. The last update was on 1 July 2015.
Inclusion and non-inclusion criteria: Studies were included if they evaluated a health service intervention involving transition from paediatric to adult care, collected data at least in part after the transfer to adult care and compared outcomes either between an intervention and control group or between preintervention and postintervention in a single group. We did not include studies with exclusively qualitative assessments.
Study selection: Reports were assessed by two reviewers (ELR and HM), who screened the titles and abstracts to identify relevant studies. Full texts were read when abstracts met inclusion criteria, and when abstracts were not clear enough to ensure eligibility. When full text was not available online, authors were contacted by email; all the contacted authors responded favourably and shared their articles with us.
Data extraction: A standardised data collection form was developed. The two reviewers independently extracted data from studies and disagreements were resolved by discussion. For each study the following were noted: general information, population, characteristics of study methodology (blind, sample size, comparison group and intervention), data collection, analysis characteristics and results as reported by the authors. Our extraction grid has been developed using the PICOTS (populations, interventions, comparators, outcomes, timing and setting) elements. Studies were classified as ‘completed and published’ or ‘in progress, no final results published’ for those registered on ClinicalTrial.gov but without publication yet on PubMed (or solely a protocol publication).
Data definition and analysis: The transition outcomes of each study were categorised into three groups: patients' perspectives (satisfaction with care or with intervention, psychological well-being, health knowledge, self-management, social or relational issues, QOL/perceived health status); healthcare providers’ perspectives (clinical and biological data, adherence to medical treatment); policymakers' perspectives (health service usage, cost).
Most components of transition interventions were reported as recorded in articles, but we collected under the generic terms ‘AYAC’: adolescents and/or young adults clinics; ‘transition file’: all written summaries or files comprising a set of data about the paediatric history of the young patient (therapeutic, ±educational, ±environmental); ‘paramedical transition’: a transfer of files between paramedical professionals of paediatric and adult teams; ‘joint consultation’: a consultation with at least one member of the paediatric team, one member of the adult care team and the young patient. Parents were considered as involved if special attention was addressed to them in the description of the intervention; their mere opinion on decisions about transfer timing was not treated as an involvement in the intervention. Depending on the data reported, we calculated ages of participants at recruitment and age at transfer. If not reported (NR), characteristics of the care structure in which intervention took place (paediatric care only, adult care only or both in the same structure) were checked on the internet. We classified tools used for outcome assessment as ‘validated’ when an evaluation of their properties was previously published. We considered that there was a statistical evaluation when a difference in outcomes of the interventional group and the comparative group was reported using a statistical test; otherwise it was considered as a qualitative difference. All analyses were carried out with SAS V.9.4 software. Qualitative data were expressed as numbers (percentage) and quantitative data as median (minimum–maximum).
Literature search: The electronic search strategies used identified a total of 2592 references: 106 were selected based on the titles and abstracts. Thirty-nine studies met the inclusion criteria and were selected for the review. The flow diagram is presented in figure 1. Full description of studies included is provided in online supplementary appendix 3.
Study characteristics: Among the 39 included studies: 26/39 (67%) published their final results and 13/39 (33%) were in progress (no final results published yet). Among these 13 studies, 5 (38%) published their protocols or first data in scientific journals, the others were found on the registry database.
Interventions were carried out between 1980 and 2015 (median 2007, NR=6). They mostly began before the last paediatric consultation (26/39, 67%) and were mainly conducted with young people with a single specific chronic disease (32/39, 82%). Evaluation of intervention effect was measured either in the same group before and after the intervention (9/39, 23%) or with one (25/39, 64%) or several comparison groups (5/39, 13%; table 1).
Evaluation methodology and reporting
Comparison and blinding: In nine studies (23%), a real comparative group was lacking: comparison between preintervention and postintervention in a single group of young people was used. Over time, historical data were most commonly used as a comparison (11/39, 37%). In studies in progress, control groups constituted by individual randomisation of patients are the most used (8/13, 67%). There is no blinding strategy described in complete studies but 2/13 (15%) studies are in progress24 ,25 (table 1).
Outcomes: Over all periods, 87% (34/39) of studies used several outcomes for intervention assessment. Before 2011 there was no distinction between primary and secondary outcomes.
Criteria reflecting the clinical condition were heavily used to evaluate the effect of transition interventions (27/39, 69%), in one-third of the cases as secondary criteria (9/24, 37%).
Health services usage is a criterion consistently and frequently used (26/39, 67%). Only one study used an assessment of costs as a primary criterion.26
Patients' perspectives outcomes are increasingly considered. Autonomy (6/13, 46%) and QOL (7/13, 64%) particularly emerged in studies in progress. New evaluation criteria relating to social and familial relationships appear in recent studies (table 2).
Assessment: Among the 23 studies which used questionnaires, 11 (48%) employed at least one validated questionnaire. Four studies (4/39, 10%) reported exclusively qualitative analysis of outcomes (table 2).
Quality of reporting: In terms of reporting quality, when judged against reporting guidelines27 ,28 there are some gaps in published studies. Fifteen (15/26, 58%) did not report the ages of young people at transfer, and 6/26 (23%) did not mention the time of collection of at least one outcome. Of the 12 studies which included a transition coordinator, 3 (25%) did not specify his or her educational background or profession.
Intervention characteristics: In all studies completed or in progress, the most common components of interventions are education (20/39, 51%) and presence of a transition coordinator (16/39, 41%).
Study population: Most studies focused on a unique chronic disease (32/39, 82%) which was mainly T1DM (15/39, 38%). Several studies in progress concerned a set of pathologies (4/13, 31%), and one study was evaluating an intervention dedicated to all types of chronic illness or cognitive disabilities.29
Regarding non-inclusion criteria, 7 studies (7/39, 18%) did not include young people with intellectual disability30–36 and 2/39 (5%) did not include those with comorbidities.33 ,37 One study did not include patients known to be non-adherent33 and one excluded those with condition(s) which in the opinion of the investigator might interfere with the ability to participate in the study.24
Persons involved: The involvement of families is reported in 11/26 (42%) of published studies and in 2/13 (15%) of ongoing studies (table 3). The participation in interventions by adult teams is reported in 23/39 (59%).
This review shows that there has been relatively little assessment of transition interventions, with only 39 comparative studies in the whole review period. Overall, there is room for improvement in methodological quality and reporting, to enhance the reliability and value of studies and to determine which interventions are really effective. However, evidence-based assessment in the field of transition interventions faces a number of difficulties which go some way towards explaining the methodological weaknesses of these studies. All complex interventions pose substantial challenges to investigators,38 and only a small number of academic research teams work on transition issues. Consequently, methodological progress in evaluation over the review period (including the use of validated measurement tools, the choice of an unbiased comparator or the choice of an adapted evaluation design) has been relatively slow when compared with advances in transition care approaches.39 From the results of this review we have identified six areas for possible future improvement and related proposals (table 4).
The possible actions which we propose are general actions: using randomisation, calculation of a realistic sample size, the use of blind evaluation, validated questionnaires, standardised outcomes, an adapted reporting grid.
Some of the problems about transition evaluation are common to all paediatric research such as the recruitment problems due to the small number of eligible patients. But some are more specific to transition intervention and other complex intervention: the lack of standardisation of usual care; the absence of consensus on the main outcomes that we have to evaluate, the absence of consensus on how we have to evaluate these outcomes (which tool, at what point in time…), the choice of an adapted evaluation design, the difficulty to perform the blind trial.
Methodological implications for future studies
Users' involvement: Actively involving young people and their perspectives in transition's research can help in the ethical design and conduct of research. Thus, the research and intervention are more relevant, useful and beneficial to the patients. It may also help to define what is of high priority and acceptable to participants; improve the process of informed consent; improve the experience of participating in research; and improve the communication of findings to participants.45 ,46
Recruitment: Conducting studies in the field of paediatric clinical research is challenging due to the small number of patients in the area of transition studies. Participant recruitment is well known to be a limiting factor in the realisation of high-quality randomised controlled trials (RCTs).47 The inclusion of a sufficient number of patients is not always possible, particularly in the field of transition. Thus, a better adapted methodological approach is needed for intervention assessment that includes a basis on reasonable and relevant hypothesis that allow a realistic sample size to conclude. The choice of a quantitative criterion, ideally one proposed by the consensus study,48 reduces the number of participants required.
For example, 264 patients are required if the calculation is made on the assumption of a decrease in the proportion lost to follow-up (binary criterion) from 15% to 5% (with α=5%, 1−β=80%) while 72 patients are required if the calculation is made on the assumption of an increase in the transition satisfaction score (quantitative criterion) from 7/10 to 9/10 (with SD=3).
An alternative solution for investigators could be to broaden the recruitment to achieve the required number. A possible solution to insufficient recruitment—due to the low numbers of young people in transition in each medical specialty—could be to provide interventions targeting groups of young people suffering from various chronic diseases. Our review found that intervention programmes are in fact generally non-specific, which leads us to suggest that a pooling of resources and interventions could be beneficial. Similarly, about one in five studies excluded young people with intellectual disabilities: restrictive inclusion criteria tending to retain the ‘good performers’ should be avoided to ensure the external validity of the study.49 Last but not least, the possibility to carry out multicentre RCTs should be considered.
Control groups: Only a quarter of the studies reviewed were RCTs. The use of historical comparison groups was the most common method. It may introduce bias, and the results must be interpreted with caution.50 There is no guarantee that the historical group is comparable to present patients: service recruitment and medical conditions may have changed over time. Moreover, care may have evolved, particularly in the transition field where healthcare teams are increasingly aware and informed.39 Among RCTs, two-thirds involved were non-active groups. In regard to recommendations published over the two last decades, the control group should benefit from the following usual care for transition: ongoing information throughout the care pathway, inclusion of the family, consideration of developmental aspects, proposal of specific education and resources, coordination with primary care, time of transfer adapted to the individual level.51
Blinding: Only two studies reported the use of blinding strategies. One reported the blinding of analysts and outcome assessors,24 and one planned the blinding of participants in the context of a phone and computer-based intervention.25 Other studies were open-label design. The nature of the interventions often makes it impossible to blind participants or personnel; however, assessors of at least some outcomes may be blinded. In the absence of a blinding strategy for patients, patient-reported outcomes will inevitably be collected with knowledge of the intervention received, but outcomes like clinical or physiological aspects may be measured by an independent outcome assessor to fulfil quality requirements27 and avoid detection bias.52
Measurement validity: Among studies using questionnaires, half of them used validated questionnaires. The remaining studies used non-validated questionnaires, whose content is generally not described. In designing new studies, we would advise research of existing validated tools for assessment, and that the use of existing, pertinent and valid tools should be preferred. Design and use of new measurement tools need to be accompanied by validation studies to ensure valid, precise and reproducible measures.53 Furthermore, to allow interstudies comparison the use of non-disease-specific questionnaires will be relevant. The Transition Readiness Assessment Questionnaire (TRAQ), best-validated transition-readiness tool,42 ,54 which measures the independence in transition-relevant skills and the 36-item Short Form Health Survey (SF-36) which was found to be useful in differentiating the global health benefits produced by a wide range of different treatments41 can be considered to assess the independence and QOL criteria. Unfortunately, there is currently no generic validated questionnaire that measures transfer to adult healthcare satisfaction.43
Standardised assessment: Transition interventions are complex with various components at different levels. Here a median number of four outcomes are used to assess the effects of interventions. Cost is used as an end point in only one study. However, as it is a complex intervention, investigators should systematically include economic measures, in addition to outcomes relevant to patients and to the health system.55 Cost assessment is crucial to gauge effectiveness and feasibility on a large scale.
Among the studies we reviewed, outcomes were multiple and varied thus, comparison between studies was not conducive. The international consensus study identified one indicator that was considered as essential and a very important indicator to assess transition success.48 There should be a shared reflection on standardised methodologies of assessment, including the best timing of data collection. In the studies reviewed, the time of data collection after transfer of patients, varies between 3 and 96 months. This illustrates that standardised assessment criteria alone will not solve the problem of comparison if methods of data collection are not the same. Attendance at an adult care clinic should be a starting point given that it has been identified as consensual,48 but the timing or frequency of data collection and the exact nature of what is being measured (at least one adult consultation, any missed consultation, number of months elapsed since the last visit in specialised care or any structure of care) remains to be determined. Among the historic recommendations on transition assessment, Blum stated that the research should follow patients for sufficient time (3–5 years) to account for the variation in natural history among their conditions. For reasons of feasibility and costs, assessments at 3 years appear to be relevant. Finally the use of administrative databases for transition assessments must be considered since it enables the access of data regularly updated, with no additional costs of collection, in addition to the capture of individuals who may not respond to surveys and the potential linkage with other data sets.56
Effects interpretation: The transition period between childhood and adulthood involves many rapid changes in physical, emotional, mental, psychological, social and cognitive terms. Emerging adults with chronic disease also have to assume increasingly full responsibility for disease management. The transitioning youth's developmental and functional abilities may influence the transition's success.2 These may complicate the evaluation of transition programme. These changes may act as a third factor linked both to intervention and outcome and can skew the cause–effect relation. This is why the use of a randomised controlled group is particularly relevant. Random allocation ensures no systematic differences between intervention groups in factors that may affect outcome. Thus, only the effect of intervention explains the differences observed.57 As complex interventions, transition programmes' effects have to be interpreted in regard to some specific factors as it has been mainly described in the guidance of the Medical Research Council (MRC).44 The observed effects should be interpreted taking into account the theoretical efficacy of the intervention itself as well as contextual factors that may affect implementation, intervention mechanisms and outcomes. In fact, transition interventions are not only complex interventions they take place in a period of developmental changes as well as structural care-related changes that are important contextual factors. Lack of effect may reflect implementation failure rather than genuine ineffectiveness; a thorough process evaluation is needed to identify implementation problems. The MRC recommends the realisation of a pilot study to examine the key uncertainties that have been identified during development that may impact on the effects obtained in the following efficacy study (including questions of acceptability, compliance, delivery of the intervention, recruitment and retention, expected effect sizes).17 When interpreting the results of the transition programme it is necessary to take into account possible implementation failure, variability in individual level that may reflect higher level processes, the choice of the primary outcome that may not make best use of the data, the strict standardisation of the intervention that may be not adapted to each local settings.
Reporting: As noted in the Template for Intervention Description and Replication (TIDieR) checklist,58 description of an intervention involves more than providing a label or an ‘ingredients list’; all the key features can influence efficacy and replicability. For complex interventions, providing details for each component of the intervention is a minimum requirement to enable other researchers to replicate the intervention or to build on research findings. For this purpose, a specific set of criteria has been developed to ensure high-quality reporting of studies for the development and evaluation of complex interventions.28 In contrast it has also been reported that in non-pharmacological interventions59 and complex interventions,38 the completeness of intervention information is often insufficient.
Indeed, if we refer to guidelines27 ,28 ,58 for complete reporting of diverse study designs, the transition studies included in our research have some gaps. All four elements—population, intervention, comparison and outcomes—show specific areas for possible improvement. For instance, in terms of population, age at transfer of the populations studied is rarely reported, although this characteristic is a key element of population description in the context of transition intervention.60 ,61 Concerning the intervention, it is important to clarify, for example, what is the educational background of the transition coordinator and what materials are used during education sessions of the intervention or if it was provided individually or in a group. ‘The usual care must also be described with a high level of detail. For example, authors should explain the content of the therapeutic education programme delivered in usual care, which varies greatly between different teams, and potentially has an impact on the preparation of youth during the transition’. Regarding the comparator, it is important to describe the context in which the transition took place when the comparison involves a historical group or another centre, and particularly the differences in terms of handover and structure. Furthermore, the modalities of transition for control groups receiving ‘usual care’ must be explicitly described. In complex interventions, the standard practice used as a control can be as complex as the intervention, and may change with time.55 Concerning outcomes, it is necessary to specify the timing of data collection, its frequency and the period between data collection and intervention or between data collection and transfer. In addition to the tools or technologies used, the mode of questionnaire administration has to be specified, knowing that variations in the results may result from the collection method.62
Finally, we need to consider that such interventional research should be replicated. Tools have been designed to identify the key domains and factors that facilitate implementation of complex health interventions,63 ,64 and these could be used in approaching the possible dissemination of the evaluated intervention on a large scale, as was done in the transition programme Ready Steady Go.65
Strengths and limitations
To the best of our knowledge, this is the first study focusing on the methodological issues used in transition interventions that examines the uptake of key recommendations in the field of transition. One of the strengths of this study is the extensive research of the literature in multiple databases and the inclusion of studies in progress. For some of the published studies, data were not available; this enabled us to observe reporting quality, but we have not contacted the authors to retrieve missing data.
We did not focus extensively on existing information which exist in guidelines and checklists developed in the field of complex interventions. It is however necessary that investigators wishing to develop or evaluate a transition action go beyond the references we cited in this review and applied appropriate existing recommendations.
We excluded qualitative assessment studies while quantitative studies are only part of the whole works of assessment transition programmes. We did not analyse the methodologies used and specific challenges of qualitative research evaluation. This remains to be done since the qualitative research can add value, for example, by providing more insightful and illuminating ways of understanding phenomena and the ways in which they can best be managed.
We selected solely those studies that had at least one part of their assessment of outcomes after transfer. This allows us to focus on studies that evaluate of the whole transition process. The whole process includes the preparation of transition which happens in paediatrics before transfer—and which is undoubtedly a very important part of the process—as well as transfer and the after transfer periods.6 To have access to the methodologies used in the most recent studies, we extended our search to an online trial registry. We used a single register. Regarding the exhaustiveness of protocols found, the International Committee of Medical Journal Editors (ICMJE) requires and recommends that all medical journal editors require registration of clinical trials in a public trials registry at or before the time of first patient enrolment as a condition of consideration for publication. One limitation is that this search is more specific to the trials than to quasiexperimental designs that could be eligible for this review. Protocols found with the register search are perhaps not representative of the whole comparative studies currently underway in the field of transition, with a possible over-representation of experimental trials.
We did not undertake an assessment of the risk of bias in RCTs, since most of these are in progress and methodological characteristics (details on randomisation or blinding procedures, intention to treat, and analysis plan) were not systematically described in the online registry. We conducted a bias evaluation without the use of a specific tool; however, we discussed the various sources of bias that we identified during the review. A specific risk of bias assessment tool is missing to evaluate the level of bias of non-randomised studies in the field of complex intervention.
Our study highlights the fact that few evaluative studies exist in the area of transition (26 published, 11 in progress) and few are randomised, even among studies in progress. This has consequences for the strength of evidence but can be explained in part by the complexity of studies of multifaceted interventions, with multiple stakeholders, multiple outcomes, difficult blinding, and in the context of paediatrics, low numbers. We have identified areas which still need improvement in terms of randomisation, recruitment and external validity, blinding, measurement validity, and standardised assessment and reporting (table 4). Some improvements could be implemented now on the basis of what is already known, and others require additional methodological research. Improvement of the methodology used for unbiased studies and the standardisation of measures are important for enabling interstudy comparisons to determine the most effective interventions.
Contributors ELR conceptualised and designed the study, designed the data collection instruments, collected data, carried out the initial analyses, drafted the initial manuscript, revised the manuscript and approved the final manuscript as submitted. HM collected data, participated in the interpretation of the data, revised the manuscript and approved the final manuscript as submitted. RB and CA provided methodological advices, interpretation of data, critically reviewed the manuscript and approved the final manuscript as submitted. SG-C, PJ and SG provided interpretation of data, critically reviewed the manuscript and approved the final manuscript as submitted. CA and ELR are the guarantors for the study.
Funding This project was supported by a grant from the Roche Foundation (contract number 2014RDB010) and by the French Ministry of Health. ELR was also supported by a doctoral contract funded by the UPMC Université Paris 6 Sorbonne Paris Cité.
Competing interests All authors have completed the ICMJE uniform disclosure form (at http://www.icmje.org/coi_disclosure.pdf) and declare: ELR reports grants from UPMC Université Paris 6 Sorbonne Paris Cité, during the conduct of the study. HM reports grants from Pfizer Foundation for child and adolescent health, outside the submitted work. SG reports grants from French Ministry of Health, during the conduct of the study. PJ reports personal fees and non-financial support from Pfizer, outside the submitted work. CA reports grants from Roche Foundation, during the conduct of the study.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The authors will give full access to our database that gathered data of individual studies included in this review. The request must be done by sending an email to firstname.lastname@example.org.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.