Article Text
PDF
Abstract
Objective Because most children with asthma now use inhaled corticosteroids (ICS), the added benefit of immunotherapy in asthmatic children needs to be examined. We re-assessed the effectiveness of subcutaneous (SCIT) and sublingual immunotherapy (SLIT) in childhood asthma treatment focusing on studies with patient-relevant outcome measures and children using ICS.
Methods We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to systematically search and appraise the evidence using predefined critical patient-relevant outcomes (asthma symptoms, asthma control and exacerbations). We searched to retrieve systematic reviews and randomised controlled trials on immunotherapy for asthma in children (1960–2017). We assessed the quality of the body of evidence with GRADE criteria.
Results The quality of the evidence for SCIT was very low due to a large risk of bias and indirectness (dated studies in children not using ICS). No effect of SCIT was found for asthma symptoms; no studies reported on asthma control. For asthma exacerbations, studies favoured SCIT. We have little confidence in this effect estimate, due to the very low quality of evidence. For SLIT, quality of the evidence was very low due to a large risk of bias, indirectness and imprecision. The outcome ‘asthma symptoms’ could not be calculated due to lack of standardisation and large clinical heterogeneity. Other predefined outcomes were not reported.
Conclusion The beneficial effects of immunotherapy in childhood asthma found in earlier reviews are no longer considered applicable, because of indirectness (studies performed in children not being treated according to current asthma guidelines with ICS). There was absence of evidence to properly determine the effectiveness or lack thereof of immunotherapy in asthma treatment in children with ICS.
- allergy
- asthma
- children
- adolescents
- immunotherapy
- efficacy
- GRADE
- systematic review
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This study is the first review evaluating immunotherapy in asthmatic children using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, focusing more on clinically relevant than on statistically significant differences in patient-relevant outcomes.
By using GRADE, we identified indirectness in previous systematic reviews in this field, which highlights a lack of applicable evidence.
A strength of the study is the use of predefined clinically relevant patient outcomes, rather than statistically significant differences.
A general limitation of a systematic review is the use of aggregated data, that, in theory might mask potential specific results.
This study has focused on critically appraising earlier evidence for nowadays practice, rather than endeavouring to be complete.
Introduction
Asthma affects 10%–15% of school-aged children. For children with persistent asthma, all international guidelines recommend daily controller treatment with inhaled corticosteroids (ICS), and reliever medication (short-acting β-2-agonists) as needed.1 2 Although many children achieve complete asthma control using this effective and safe treatment,1 some need additional treatment to obtain disease control.3 4 Identification and treatment of comorbidities in children with problematic severe asthma is part of the stepwise approach to improve asthma control in these children.5 6
The most common of these comorbidities in children with asthma is allergic rhinitis,5 symptoms of which occur in 60%–80% of asthmatic children.7 8 Allergic rhinitis shares a common pathophysiological pathway with asthma, which has been described as the united airway concept.9 Allergic rhinitis is associated with worse asthma control in children, and accumulating evidence suggests that treatment of allergic rhinitis with intranasal steroids improves rhinitis and asthma symptoms in these patients.7 10 11
When symptoms of allergic rhinitis cannot be sufficiently controlled with nasal corticosteroids and oral antihistamines,9 12 immunotherapy can be considered as additional treatment.13 Subcutaneous immunotherapy (SCIT) requires repeated injections with an allergen extract and is available for allergens such as grass and tree pollen and house dust mite. After disappointing results of low-dose preparations in drops, effective high-dose sublingual immunotherapy (SLIT) has now become available with grass pollen allergen extract in a daily sublingual tablet.14 15 A Cochrane systematic review (SR), first published in 2000, and last updated in 2010, reported beneficial effects of immunotherapy in children with asthma.16 Multiple studies in this latter review, however, were performed before or in the 1980s, when most children with asthma were not using ICS.
As part of the update of the Dutch paediatric guideline on childhood asthma, we evaluated the literature on the added value of SCIT and SLIT in childhood asthma.17 Our structured clinical question was to assess whether immunotherapy (subcutaneous or sublingual), as an add-on to usual care with daily ICS, improves asthma outcomes in children (6–12 years) and adolescents (>12 years) with persistent asthma and sensitisation to relevant aeroallergens (grass or tree pollen, house dust mite or combinations), with or without symptoms of allergic rhinitis.
Methods
We used GRADE approach (Grading of Recommendations Assessment, Development and Evaluation) to appraise and summarise the body of evidence. GRADE is an internationally approved standard for managing complex evidence reviews.18 In contrast to former grading systems, GRADE focuses on the quality of the total body of evidence, instead of judging single studies. Another important characteristic of GRADE is that predefined outcomes with thresholds for clinical relevance are being used.19 In earlier grading systems, the evidence was summarised using outcomes reported in studies, not necessarily being outcomes a guideline development group would be interested in.20 GRADE avoids the use of surrogate or intermediate outcomes, and uses outcomes and differences that are more clinically relevant to patients instead. Starting from a SR, for each outcome the quality of evidence can be downgraded or upgraded, for instance based on risk of bias, inconsistency, indirectness, possible publication bias and dose–response relation.
The guideline development group included an epidemiologist, paediatric respiratory physicians, paediatricians, an allergist, an ear-nose-throat specialist, a family physician, a lung function technician, a youth public healthcare physician and patient representatives. The guideline development group predefined clinically relevant outcomes and divided these into critical (contributing to the overall quality of evidence), important (also relevant to the content of the guideline) and not important outcomes. For each outcome, a minimal clinically important difference was defined a priori. The outcomes taken into account in our literature review are summarised in table 1, with corresponding minimal clinically important differences.21–24
Patient-relevant outcomes and clinical relevance
We applied a sensitive search strategy to retrieve all available evidence addressing the clinical question, focusing on SRs about asthma and immunotherapy in children. Literature searches were performed in March 2012 for the guideline (from 1960 onwards), and updated in April 2015 for the purpose of this review. A second update, including an expansion of the searching scope, was performed in June 2017 (see online supplementary table E1). In the original search, we searched in the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness and the Cochrane Central Trial Register. In the 2017 update, we also searched for SRs in the Medline and Embase databases (again from 1960 onwards). Two reviewers (EJG and MKT) independently screened the abstracts using predefined inclusion criteria: methodology (SRs), patients (children with allergic asthma) and SCIT and/or SLIT as an intervention. Animal studies, conference abstracts and studies published in languages other than English, Dutch and German were excluded. Differences between reviewers were resolved by consensus. Selected abstracts were critically appraised with respect to study population, intervention and methodological aspects (eg, systematic search and selection, inclusion of randomised controlled trials (RCTs)), which led to a further selection. An expert in the field (HdG) judged the selection for completeness.
Supplementary file 1
All included studies were summarised in evidence tables by two reviewers (EJG and MKT). SRs were critically appraised using the AMSTAR checklist (A Measurement Tool to Assess Systematic Reviews).25 AMSTAR scores range from 0 to 11, a higher score indicating better quality (less bias). The Jadad scale was used to assess the methodological quality of each included RCT.26 This score ranges from 0 to 5, a higher score denoting a better quality. All eligible studies together defined the body of evidence, of which the quality was determined (per relevant outcome and overall quality) and GRADE profiles were created. Results from SRs and RCTs were pooled, if possible, in meta-analyses using RevMan V.5. We calculated standardised mean differences for continuous outcomes, because of the usage of different symptom scales in the underlying studies. We calculated risk ratios for dichotomous outcomes, to compare the probability of these outcomes between the intervention and control groups. In the meta-analyses we used random effects models, because of the possibility of generalisation of the outcomes for different allergens, and tested the difference between intervention and control with the inverse variance method, since this method is typically used in meta-analyses to combine the results of independent studies. We reported 95% CIs (predefined significance level: 0.05). Conclusions were drawn, based on quality and content, per outcome and discussed in the expert group until consensus was reached.
Patient involvement
The guideline development group included patient representatives who helped defining our clinical question, approved outcome measures and assessed its clinical relevancy. The burden of interventions and patient considerations were assessed as part of GRADE evaluation. Patients were not directly involved in this SR since we reviewed published literature.
Results
Literature search and selection
Screening of titles and abstracts yielded 83 eligible studies, 10 of which fulfilled the inclusion criteria.16 27–35 After examining these 10 papers in full, five more studies were excluded (figure 1).
Literature search and selection. RCT, randomised controlled trial; SCIT, subcutaneous immunotherapy; SLIT, sublingual immunotherapy.
Experts in the guideline working group confirmed that no relevant publications were missed. The updated search (April 2015) revealed 43 additional studies, of which 2 fulfilled the inclusion criteria.36 37 Full text examination resulted in exclusion of these two studies. The extended and updated search in June 2017 resulted in 177 hits, of which 6 were selected to full paper study.38–43 These studies were SRs in the field of SCIT and/or SLIT in children with asthma. Most of the included RCTs in these reviews had already been included in the 2015 search. We only added RCTs of those reviews to our meta-analyses that have not been included earlier. As a result, we added one study.44
Results of SCIT
Description of studies
We retrieved one Cochrane SR on the effectiveness of SCIT in patients with asthma, including 90 RCTs with a total of 3792 patients.16 This was a high-quality review (AMSTAR score 10/11). Fourteen of the included RCTs were performed in children exclusively; another 24 included children and adults. In a few studies the age inclusion criteria were not clear. The characteristics of these review are summarised in an evidence table (see online supplementary table E2).16 29 32 Only nine RCTs included in these reviews reported on our predefined outcomes in children.45–53 In these nine studies different allergens or combinations were studied (house dust mite (3), dog dander (1), grass pollen (1), mould (1), grass pollen/house dust mite (1) and tailored combinations (2)). Two RCTs published after the 2010 Cochrane review were retrieved.34 35 In the first, the clinical efficacy of house dust mite-specific SCIT in 20 asthmatic children was compared with no intervention in 20 others; patients were followed up for 6 months.34 In the other, the effects of allergen-specific SCIT on corticosteroid dose in asthmatic children were evaluated.35 Details of all included RCTs are summarised in the evidence table (see online supplementary table E3).34 35 45–53
Supplementary file 3
Supplementary file 2
Quality of the evidence
Little information was given about the included studies in the Cochrane review; for example, follow-up was not stated. There were also other concerns about the quality of the literature, for example, not all studies were double blind and placebo controlled, and randomisation procedures were poor. Therefore we re-analysed the individual paediatric studies in the Cochrane review, plus the added studies.34 35 Jadad scores of the single studies are presented in table 2.
Jadad scores of RCTs on SCIT
The quality of the body of evidence for all critical and important outcomes was very low (table 3), mainly due to large risk of bias and indirectness. The large risk of bias was caused by a lack of allocation concealment, lack of information on follow-up and loss to follow-up. The reason for downgrading for indirectness was the publication year of the underlying studies; populations and interventions were considered inapplicable to current clinical practice.
GRADE evidence profile SCIT
Critical outcomes
Asthma symptoms
Four small studies carried out in children only reported this outcome in the Cochrane review.16 We extracted these results from the Cochrane review and updated these with the results from Tsai et al.34 Results are presented in figure 2.
Meta-analysis of SCIT versus placebo, outcome asthma symptoms. IV, inverse variance; Random, random effect model; SCIT, subcutaneous immunotherapy; Std, standardised.
The meta-analysis showed no significant effect of SCIT on asthma symptoms.
Asthma exacerbations
Five studies (published 1961–1984) in the Cochrane review, carried out in children only, reported this outcome.16 No relevant studies of sufficient quality were published afterwards. Our meta-analysis included 253 patients on immunotherapy and 153 on placebo. The pooled risk ratio was 0.47 (95%CI 0.31 to 0.72), favouring immunotherapy (see figure 3). The absolute risk reduction was 35%, giving a number needed to treat of 3.
Meta-analysis of SCIT vs placebo, outcome asthma exacerbations. IV, inverse variance; SCIT, subcutaneous immunotherapy.
No studies reported results on the critical outcome asthma control.
Important outcomes
No studies reported results on quality of life or lung function (forced expiratory volume in 1 s (FEV1)).
Results of SLIT
Description of studies and quality of the evidence
We retrieved two SRs on SLIT in patients with asthma.29 32 The updated search in 2017 resulted in the addition of one RCT.44 The characteristics of the SRs are summarised in evidence table (see online supplementary table E2). The quality of the reviews was moderate; both had an AMSTAR score of 7/11. Weaknesses included the absence of an ‘a priori design’, exclusion of grey literature, not assessing the likelihood of publication bias and not mentioning conflicts of interest in one review,29 and the absence of an a priori design, no information about excluded studies, too firm conclusions compared with the weak evidence and not assessing the likelihood of publication bias in the other.32 One review included both children and adults, and patients with asthma and/or rhinitis.29 Because of the quality concerns of both existing SRs, we set out to perform a meta-analysis of the original studies that fulfilled our selection criteria. Jadad scores of selected studies, as well as an overview of the outcomes of those studies, are presented in table 4.24 29 32 44 54–67 Study characteristics are summarised in the evidence table (see online supplementary table E4).44 54 55 57 58 60 63–65 We rated the quality of evidence to be very low, due to a large risk of bias, imprecision and indirect evidence.
Supplementary file 4
Summary of quality and outcome measures of selected RCTs in reviews Calamita et al, Penagos et al and added Pham-Thi et al 29 32 44
Critical outcomes
Asthma symptoms
Eight of the included studies reported on asthma symptoms. Different symptom scores were used, none of them standardised or validated. Clinical differences in asthma scores were not defined and most studies reported improvement in the treatment group as well in the control group. We were not able to compile a meta-analysis of the results of the individual studies, because of the use of various symptom scales in the included studies. Since studies did not report results in a clearly comparable way, reporting the results of the individual studies was considered unreliable.
Other critical outcomes
No studies reported results on the critical outcomes exacerbations and asthma control.
Important outcomes
Quality of life
Pham-Thi et al published results on quality of life using Childhood Asthma Questionnaires.44 The authors reported a difference in severity between SLIT and placebo in the younger population (age 6–11 years), but not in older children (age 12–16 years). It is not stated whether this difference is clinically relevant.
Lung function
Four studies reported results on lung function (FEV1). One of the studies reported no numeric data on lung function.64 One study reported no variance (SD), and no comparison of the baseline data.63 The two remaining studies reported on FEV1 percentage predicted,58 and reported no significant differences between treatment groups, neither at baseline nor at follow-up.
Discussion
Summary of main results
Our GRADE SR showed no evidence of a significant difference in asthma symptoms between SCIT and placebo in children with allergic asthma, but some evidence for a significant and clinically relevant reduction in asthma exacerbations was found in SCIT-treated children. We have little confidence in the effect estimate, however, due to a large risk of bias and indirectness. Thus, the true effect of SCIT on exacerbations and asthma symptoms in the target population of interest is likely to be substantially different from the estimate of effect. There was absence of evidence on the effects of SCIT on lung function, asthma control and quality of life in children with allergic asthma. There was no evidence for a beneficial effect of SLIT in reducing asthma symptoms and exacerbations, quality of life and lung function in children with allergic asthma. Our review does not address the efficacy of immunotherapy in children regarding complaints of allergic rhinitis and/or conjunctivitis, without having asthma.
Quality of the evidence/GRADE methodology
The overall quality of the evidence about the effectiveness of SCIT and SLIT was very low. This implicates that our confidence in the effect estimates is very limited. The true effect of SCIT and SLIT on patient-relevant asthma outcomes in children with asthma may be substantially different from our estimates of the effect. We cannot conclude that the possible desirable effects of SCIT and SLIT outweigh the undesirable effects (eg, influence on quality of life, adverse events or increased resource expenditure), nor can we reject that hypothesis. Our concerns about the quality of evidence are based on (very) serious risk of bias and indirectness in the underlying primary studies. First, the quality of many studies had to be downgraded because of risk of bias due to lack of allocation concealment, lack of information on follow-up and loss to follow-up. Second, included studies were heterogeneous in the patients included and allergen extracts used, with different dosing regimens and duration being studied, targeting different inhaled allergens. We have concerns about the potential different responses and the generalisability of the evidence. Thirdly, and most importantly, for SCIT, the quality of the body of evidence was downgraded because of indirectness, since patients in the original studies long ago are likely to differ considerably from patients nowadays.
Fourth, different studies used variable definitions of asthma exacerbations. We had to use ‘worsening of asthma’, which may not represent real-life patient-relevant exacerbations. This may decrease the applicability of the evidence. In addition, there were no studies using the predefined important outcomes quality of life and asthma control.
Finally, and most importantly, we have concerns on comparability of patients. Several included studies dated from the 1980s or earlier, when allergic rhinitis treatment with selective antihistamines and nasal corticosteroids was not available. Against the background of the united airway concept, the comorbidity allergic rhinitis in patients in these studies cannot be compared with patients in clinical practice today.9 Similarly, widespread use of ICS was not introduced in childhood asthma treatment until the 1990s.68 Most studies on SCIT in children with asthma were published decades ago, during the pre-ICS era. Although SCIT appeared to be effective in some of the included studies,49 50 we cannot draw conclusions from these findings, because the patients in the described studies represent an incomparable group when compared with the child with asthma in contemporary clinical practice. Specifically, it is unclear whether the beneficial effects found in the SR of earlier studies are applicable to children with asthma treated according to contemporary guidelines with daily ICS controller therapy.16
In our opinion and that of others, GRADE approach is superior to former methods of SRs, because it focuses on predefined patient-relevant outcomes, predefined minimally clinical important differences and because it judges the complete body of evidence. One RCT among paediatricians studied the influence of different guideline grading systems on clinician’s decisions.69 GRADE showed the largest change in direction on the clinical decision. However, the added value of GRADE on guideline implementation or patient care has not been formally evaluated, GRADE approach is still rather complex for non-methodologists.
To formulate recommendations for clinical practice, not only the body of evidence concerning effectiveness of an intervention is important. Recommendations should balance the benefits and harms of the intervention of interest, and take patient preferences and resource use into account. Since (after critical evaluation) no benefits of SCIT and SLIT for children with asthma were determined, we consider it unlikely that the benefits will exceed the harms. Patient preferences were included in the formulation of our guideline recommendations.
Agreements and disagreements with other studies or reviews
Using GRADE and re-analysing data from children with allergic asthma only, we came to different conclusions on the effectiveness of SCIT and SLIT in children with asthma than the authors of the original SRs. We believe this highlights the importance of using GRADE methodology to systematically review evidence for patient-relevant outcomes, not focusing on levels of evidence, but on underlying study validity, precision, directness and applicability in current clinical practice. The 2009 position paper on SLIT describes history, use and applicability of this treatment for allergic rhinitis.70 It positions SLIT in children as a safe and useful therapy above and after more regular treatment for allergic rhinitis. Potential positive treatment outcome for allergic asthma is however mainly based on literature in adults. We show the lack of evidence and lack of applicability of treatment of immunotherapy for asthma in children. Since we have worries on the applicability of evidence in adults on children (who are still developing their immune system), we think further studies that compare immunotherapy for the contemporary treatment of asthma in children are urgently needed to fill in this gap.
Recently, a Cochrane SR on SLIT for asthma found a similar lack of data for important outcomes (eg, exacerbations, symptom scores and quality of life) as we did.71 Contrary to our study, the authors did no separate analysis for adults and children, and patients with asthma were not separately analysed from patients without asthma.
Conclusions
Focusing on predefined patient-relevant outcomes, and critically appraising the body of evidence using original studies and GRADE methodology, our SR on the effects of immunotherapy in children with asthma came to different conclusions than previous SRs. We believe that this underscores the importance of using GRADE methodology in systematically reviewing evidence.
We found absence of valid applicable evidence on improvement of clinically relevant asthma outcomes in children with allergic asthma using SCIT or SLIT. This absence of evidence is due to serious risk of bias, large clinical heterogeneity between studies and most importantly due to lack of applicability because studies were performed in the pre-ICS era.
Since the effect of immunotherapy added to contemporary asthma treatment with daily controller therapy is not clear, the drawbacks of immunotherapy should be considered carefully. SCIT is a complex and intensive form of treatment, associated with a (very) long duration of treatment, and considerable burden to the patient with (monthly) injections under adequate medical supervision due to potential (however rare) dangerous side effects, and may have relatively high costs and resource use. In SLIT the risk of serious side effects is considerably smaller, but the other drawbacks of immunotherapy apply equally to this treatment. In our opinion therefore, when balancing the absence of evidence on a clear beneficial effect of SCIT or SLIT on clinically relevant patient outcomes in children with asthma with the considerable burden and costs of SCIT and SLIT, we do not recommend this treatment to children with asthma until further high-quality evidence from well-designed RCTs in children comparing SCIT or SLIT to contemporary asthma treatment becomes available.
Acknowledgments
We thank Nicole Boluyt, MD, PhD, Diemen, The Netherlands, for her help in the study design and revision of an earlier version of the manuscript.
References
Footnotes
E-JG and MKT contributed equally.
Contributors EJG designed the study, chaired the guideline working group, provided clinical input (eg, defined clinically relevant outcome measures, judged the literature review from a clinical point of view), wrote and revised the manuscript, and approved the final version. MKT designed the study, was methodologist of the guideline working group, provided methodological input (eg, provided literature search, selection, critical appraisal, GRADE evidence profiles), wrote and revised the manuscript, and approved the final version. HdG was member of the guideline working group, designed the study, revised the manuscript and approved the final version. PLPB provided clinical input, revised the manuscript and approved the final version. All authors meet full criteria for authorship, that is, all contributed substantially, revised, approved the final version of the manuscript and agreed to be accountable for all aspects of the work.
Funding This work was performed as part of the revision of the Dutch Guideline on Paediatric Asthma. This guideline development was financially supported by the Dutch College of Paediatricians NVK and a grant of the Dutch Federation of Medical Specialists SKMS.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Extra data can be accessed in the online repository. Apart from this, no additional data are available.