Article Text

Original research
Aluminium adjuvants versus placebo or no intervention in vaccine randomised clinical trials: a systematic review with meta-analysis and Trial Sequential Analysis
  1. Sara Russo Krauss1,
  2. Marija Barbateskovic1,
  3. Sarah Louise Klingenberg1,
  4. Snezana Djurisic1,
  5. Sesilje Bondo Petersen2,
  6. Mette Kenfelt3,
  7. De Zhao Kong4,5,
  8. Janus C Jakobsen1,6,
  9. Christian Gluud1,6
  1. 1The Copenhagen Trial Unit, Centre for Clinical Intervention Research, The Capital Region, Copenhagen University Hospital – Rigshospitalet, Copenhagen, Denmark
  2. 2Department of Occupational and Environmental Medicine, Copenhagen University Hospital – Bispebjerg and Frederiksberg, Copenhagen, Denmark
  3. 3Stationsvej 2, Farum, Denmark
  4. 4The Evidence-Based Medicine Research Center of Traditional Chinese Medicine, Liaoning University of Traditional Chinese Medicine, Shenyang, Liaoning, China
  5. 5Department of Evidence-based Chinese Medicine Research Centre, The Affiliated Hospital of Liaoning University of Traditional Chinese Medicine, Shenyang, Liaoning, China
  6. 6Department of Regional Health Research, The Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark
  1. Correspondence to Professor Christian Gluud; christian.gluud{at}ctu.dk

Abstract

Objectives To assess the benefits and harms of aluminium adjuvants versus placebo or no intervention in randomised clinical trials in relation to human vaccine development.

Design Systematic review with meta-analysis and trial sequential analysis assessing the certainty of evidence with Grading of Recommendations Assessment, Development and Evaluation (GRADE).

Data sources We searched CENTRAL, MEDLINE, Embase, LILACS, BIOSIS, Science Citation Index Expanded and Conference Proceedings Citation Index-Science until 29 June 2021, and Chinese databases until September 2021.

Eligibility criteria Randomised clinical trials irrespective of type, status and language of publication, with trial participants of any sex, age, ethnicity, diagnosis, comorbidity and country of residence.

Data extraction and synthesis Two independent reviewers extracted data and assessed risk of bias with Cochrane’s RoB tool 1. Dichotomous data were analysed as risk ratios (RRs) and continuous data as mean differences. We explored both fixed-effect and random-effects models, with 95% CI. Heterogeneity was quantified with I2 statistic. We GRADE assessed the certainty of the evidence.

Results We included 102 randomised clinical trials (26 457 participants). Aluminium adjuvants versus placebo or no intervention may have no effect on serious adverse events (RR 1.18, 95% CI 0.97 to 1.43; very low certainty) and on all-cause mortality (RR 1.02, 95% CI 0.74 to 1.41; very low certainty). No trial reported on quality of life. Aluminium adjuvants versus placebo or no intervention may increase adverse events (RR 1.13, 95% CI 1.07 to 1.20; very low certainty). We found no or little evidence of a difference between aluminium adjuvants versus placebo or no intervention when assessing serology with geometric mean titres or concentrations or participants’ seroprotection.

Conclusions Based on evidence at very low certainty, we were unable to identify benefits of aluminium adjuvants, which may be associated with adverse events considered non-serious.

  • immunology
  • infection control
  • public health

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information. Data sharing not applicable as all data are avilable in figures and tables in text or supplements.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • We seem to be the first to assess the benefits and harms of aluminium adjuvants by conducting a systematic review comparing aluminium adjuvants versus placebo or no intervention in combination with all types of vaccines.

  • We included 102 randomised clinical trials from a comprehensive search with no language limitations or restrictions on outcomes reported in the trials, type of aluminium adjuvant or type of vaccine.

  • The certainty of evidence is very low and this makes it difficult to draw firm conclusions.

Introduction

Vaccination is one of the major triumphs of modern medicine.1 2 Vaccination prevents infectious diseases, and the worldwide eradication of the deadly smallpox and the restriction of diseases such as poliovirus, measles and tetanus can largely be ascribed to the numerous successful mass vaccination programmes launched since the 1960s.1 2 Presently, COVID-19 vaccines are rolled out worldwide with speed to stop the COVID-19 pandemic.3 4 In addition to its intended effect, a vaccine may be accompanied by one or more harmful effects on administration. Harms may be considered non-serious (eg, mild, transient headache) or serious (eg, causing hospitalisation or death) and they may appear shortly after vaccine administration (eg, pain at the injection site) or belated (eg, autoimmune responses).

The human papilloma virus (HPV) vaccination programme was launched in the USA in 2006 in order to prevent HPV infection, one of the causes of cervical cancer and the second most common cancer in women.5 Despite that HPV vaccines have been assessed for efficacy (immunogenicity) in clinical trials, and approved based on their ability to raise a potent immune response against HPV and their ability to prevent persistent HPV infections,6 concerns have been raised about adverse events possibly related to the HPV vaccines formulation.7 8 Both the national vaccine adverse events reporting system in the USA and the European Union have received reports on a high number of adverse events suspected to be related to the HPV vaccination.8 However, no scientific evidence for an association was found.9 Several observational studies also failed to identify associations with clinical diagnoses.10–14 However, reasons to oppose these findings have been proposed.7 15 16

Vaccine toxicity, efficacy and effectiveness may originate from, or depend on a plethora of factors, including the vaccine components (eg, the antigen itself, the excipient or the adjuvant); interaction between different vaccine components; vaccine manufacture; overall vaccine composition; route of administration; dose; and number of booster vaccinations.17 Aluminium salts are widely used adjuvants, such as aluminium phosphate, aluminium hydroxide, aluminium potassium sulfate and amorphous aluminium hydroxyphosphate sulfate.18 They have been the standard adjuvants in vaccines against diphtheria, tetanus, and pertussis, haemophilus influenza type B, pneumococcus conjugates, hepatitis A, and hepatitis B.19 More recently, aluminium was coformulated with vaccines against HPV in the form of Adjuvant System 04 (aluminium hydroxide and monophosphoryl lipid A), aluminium hydroxide or amorphous aluminium hydroxyphosphate sulfate as well as in the worlds most used COVID-19 vaccines CoronaVac20 and Sinopharm Beijing Institute of Biological Products COVID-19 vaccine21 in the form of aluminium hydroxide.

The mechanism of action of aluminium, like for most adjuvants, is only partially understood. Its biological or physiological role is unknown. While aluminium is generally considered safe and is regularly ingested in food and water, it can be toxic based on the concentration, chemical form and the environment.22 Aluminium seems to have an impact on the immune system, which has rendered it useful as a vaccine adjuvant.19 23 Aluminium is believed to exert its adjuvant effects by stimulating Th2-type cell responses and antibody production through B cells activation,24 25 by activating the complement system, and by recruiting immune cells to the site of injection.24 26 27 At the injection site, aluminium promotes antigen uptake by specialised antigen-presenting immune cells, termed dendritic cells, as well as dendritic cell maturation.23 28 29 The consensus within the scientific community is that aluminium affects antigen uptake, induces danger signals, recruits various types of immune cells and elicits Th2 responses.30

One previous attempt to assess the effects of aluminium adjuvants with a review was undertaken in 2004 by Jefferson et al.31 The review covered existing evidence of adverse events to the aluminium-containing diphteria-tetanus-pertussis vaccine, but it did not assess benefits.31 Lin et al conducted the first meta-analysis on the efficacy of aluminium salts as an adjuvant for prepandemic influenza vaccines.32 Their results showed inferior seroprotection after aluminium-adjuvanted H5N1 vaccines compared with that conferred by non-adjuvanted counterparts; however, these findings only related to the prepandemic influenza vaccines. New adjuvants are being introduced continuously and the U.S. Food and Drug Administration (FDA) and the WHO do not require genotoxicity or cardiotoxicity studies of new aluminium adjuvants.33 The theory that aluminium adjuvant is responsible for symptoms following specific vaccine formulation is impossible to refute or prove based on the data from current clinical trials. For example, aluminium adjuvant has been administered to both the experimental and control groups in the vast majority of randomised clinical trials on HPV vaccines, thus masking aluminium adjuvant’s potentially harmful effects.34 Aluminium adjuvants, new or old, should be evaluated for benefits and harms on their own merits. While the consequences of adding aluminium to vaccines have been discussed broadly, no systematic review has been conducted to assess the effects of aluminium adjuvants across different types of vaccines.

The objectives of this review are to assess the benefits and harms of aluminium adjuvants vs placebo or no intervention in randomised clinical trials in relation to human vaccine development. Our aim was not to analyse the benefits and harms of vaccine formulations for prevention of a specific disease. The results of our systematic review could influence future vaccine formulation and bring on changes among policymakers and vaccine manufacturers to secure safe and efficient vaccines to people.

Methods

Detailed description of our methodology is in our prepublished protocol,35 PROSPERO protocol (CRD42017083013) and our online supplemental material.

Criteria for considering studies for this review

We searched for randomised clinical trials irrespective of type, status, date and language of publication. We included vaccine development trials comparing any type of aluminium adjuvant versus placebo or no intervention. We accepted any cointerventions of vaccines if planned to be delivered equally to the intervention groups. We used the trial results reported at maximum follow-up.

Types of outcome measures

Primary outcomes were serious adverse events,36 all-cause mortality and proportion of participants with the disease being vaccinated against. Secondary outcomes were health-related quality of life, non-serious adverse events and serological response.

Search methods for identification of studies

We searched the Cochrane Central Register of Controlled Trials (2021, Issue 7) in The Cochrane Library, MEDLINE Ovid (1946 to July 2021), Embase Ovid (1974 to July 2021), LILACS (Bireme; 1982 to July 2021), BIOSIS (Web of Science; 1969 to July 2021), Science Citation Index Expanded (Web of Science; 1900 to July 2021), and Conference Proceedings Citation Index-Science (Web of Science; 1990 to July 2021). In addition, we searched (September 2021) the Chinese Biomedical Literature Database (CBM), China Network Knowledge Information (CNKI), Chinese Science Journal Database (VIP) and Wanfang Database (online supplemental table S1). We also searched Google Scholar, The Turning Research into Practice (TRIP) Database, ClinicalTrials.gov (www.clinicaltrials.gov/), European Medicines Agency (EMA; www.ema.europa.eu/ema/), WHO International Clinical Trial Registry Platform (www.who.int/ictrp), The Food and Drug Administration (FDA; www.fda.gov) and pharmaceutical company sources for ongoing or unpublished trials (until March 2021). We applied EMA, FDA and several national medicines agencies (Australia, China, India, Japan, UK, Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Liechtenstein, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain and Sweden) for clinical study reports on trials fulfilling our inclusion criteria.

Data collection and analyses

Three review authors (SRK, SLK and MB) independently and in pairs screened titles and abstracts for inclusion of potentially eligible trials using Covidence (www.covidence.org). Following any unsolved disagreements, we asked a third author to arbitrate (JCJ or CG). The review author pair collected full-text trial reports/publications, and independently screened the full-texts and identified trials for inclusion. SRK extracted all data on all trials. SLK and MB each independently extracted half of the data. Extractions were compared and validated by SRK, SLK and MB and in case of disagreement, the same review authors consulted JCJ or CG.

Assessment of risk of bias in included studies

The review author pair (SRK, SLK and MB) independently assessed the risk of bias (RoB 1) of each included trial according to the recommendations in the Cochrane Handbook for Systematic Reviews of Interventions.37 We used the following bias risk domains: ‘allocation sequence generation’; ‘allocation concealment’; ‘blinding of participants and treatment providers’; ‘blinding of outcome assessment’; ‘incomplete outcome data’; ‘selective outcome reporting’ and ‘other bias’. We assessed the domains ‘blinding of outcome assessment’, ‘incomplete outcome data’ and ‘selective outcome reporting’ for each outcome. The trial was classified at overall ‘low risk of bias’ only if all the bias domains described in the previous paragraphs were classified at low risk of bias, or at ‘high risk of bias’ if any of the bias risk domains described above were classified at ‘unclear’ or ‘high risk of bias’.

We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system to assess the certainty of the body of evidence associated with each of the outcomes.38 We constructed a summary of findings table using the GRADEpro software.39 The GRADE system appraises the certainty of evidence based on the extent to which one can be confident that an estimate of effect or association reflects the item being assessed.

Measures of treatment effect and data synthesis

We visually investigated forest plots to assess the risk of statistical heterogeneity. We also assessed the presence of statistical heterogeneity using the χ2 test (threshold p<0.1) and measured the quantities of heterogeneity using the I2 statistic.40 41 We assessed reporting bias using funnel plots when 10 or more trials per comparison were included.

When the total proportion of participants experiencing any serious and non-serious adverse event was not reported, we extracted data from the highest proportion of participants experiencing an individual adverse event.

We performed subgroup analyses on (A) outcomes at low risk of bias compared with outcomes at high risk of bias or unclear risk of bias (collectively termed high risk of bias); (B) trials at low risk of vested interests compared with trials at high risks of vested interests;42 (C) according to aluminium adjuvants type; (D) according to different vaccines; (E) according to age groups; (F) according to different maximal follow-up periods; (G) according to participants’ health and (H) according to vaccines against extracellular or intracellular pathogens.

We assessed the potential impact of missing data with the ‘best-worst’ and ‘worst-best’ case scenarios.

Intervention effects were assessed with both random-effects model43 and fixed-effect model44 meta-analyses. The more conservative point estimate of the two (the analysis with the highest p value) was reported primarily. For analysis of the three primary outcomes, a p<0.025 was considered statistically significant45 because this would secure a familywise error rate below 0.05.

We analysed all primary and secondary outcomes using Trial Sequential Analysis (TSA) o.9.5.10 Beta software to control random errors.35

We included aluminium concentration (as described by trialist or manufacturer) as a covariate in meta-regression to assess whether the concentration influences the effect of aluminium adjuvant administration on outcomes.

Patient and public involvement

A patient and a representative of the public were involved in formulating the research question and the outcomes at the protocol stage. They were both involved in the interpretation and writing up of results. There are plans to disseminate the results of the research to the public and the relevant patient communities.

Results

Online supplemental figure S1 shows the flow of records obtained through electronic searches. We identified 15 958 records through database searching. We obtained 396 full-text reports that were assessed for eligibility. We excluded 280 records. We identified eight trials awaiting classification and six ongoing trials (online supplemental appendix 1).

We identified 102 randomised clinical trials including a total of 26 457 participants that fulfilled our inclusion criteria. Characteristics of included and excluded studies are given in online supplemental appendix 1. We were unable to identify any clinical study report from regulatory authorities that was eligible for inclusion in this review. We approached all corresponding authors to request missing information or explanations on unclear information and received some additional information from seven authors.

The 102 included trials were published between 1969 and 2021; 35 were conducted in the USA;46–80 13 in more than one country;81–94 6 in Canada;95–100 4 in China;101–104 4 in Belgium;105–108 4 in Africa;109–112 3 each in the UK,113–115 Taiwan116–118 and Australia;119–121 2 each in Thailand,122 123 Poland,124 125 Norway,126 127 Italy,128 129 Germany,92 130 Cuba,131 132 and Austria;133 134 and 1 each in Switzerland,135 Sweden,136 Singapore,137 Mali,138 Israel,139 India,140 France,141 Colombia,142 Chile and Bangladesh.143 Three trials did not report a country.144–146

Trial participants

The trials randomised different types of participants. Ninety trials randomised healthy participants; nine trials randomised participants with a disease diagnosis;50 57 60 71 90 91 119 129 134 and three trials did not describe the inclusion criteria of the participants.55 98 99

In regard of age, the trials randomised: infants (6 trials);86 113 125 132 143 145 children (11 trials);85 87 94 101 103 116 132 138 146–148 adolescents (2 trials);66 114 elderly (9 trials);50 60 62 83 97 100 111 119 139 and mixed populations (8 trials).57 78 91 102 104 108 112 123 Two trials did not specify the population type.55 134 The remaining 65 trials randomised adult participants.

Interventions and comparisons

Types of aluminium adjuvants

The included trials assessed different types of aluminium adjuvants: aluminium hydroxide (38 trials);49 50 57 59 67–70 72 74 81–83 88–90 92 93 96 101–103 105 106 110 112 114 118 122–124 126 133 134 140–142 144 aluminium phosphate (26 trials);46 47 53 60 62 63 66 75 80 86 87 97 108 109 111 115–117 121 131 132 136 139 145 146 149 alhydrogel (21 trials);48 51 52 54–56 58 73 76–79 98–100 107 113 120 125 127 137 amorphous aluminium hydroxyphosphate sulfate (2 trials);64 71 aluminium fluoride (1 trial);94 phosphate-treated aluminium hydroxide (1 trial);143 alhydrogel pretreated with phosphate buffer (1 trial);135 Adju-Phos (aluminium phosphate gel) (1 trial);95 aluminium potassium sulfate (1 trial);65 aluminium chloride (1 trial)61 and aluminium oxide (1 trial).150 Eight trials did not describe the type of aluminium adjuvant used.84 85 91 104 128–130 138

Vaccines against different viruses, bacteria, toxins or diseases

The included trials assessed the effects of vaccines against different viruses, bacteria, toxins or diseases: influenza (25 trials);49 50 54 56 67 68 84 88 98–100 102 104 115 117 118 121 126 130 133 137 139–141 150 Streptococcus pneumoniae (11 trials);74 85–87 94 108 111 135 143 145 146 respiratory syncytial virus (11 trials);60 62 63 79 80 82 96 97 105 119 144 human immunodeficiency virus (6 trials);47 59 75 110 128 129 Neisseria meningitidis (6 trials);53 61 66 109 127 138 Clostridium difficile (4 trials);57 69 78 83 dengue fever virus (4 trials);55 58 120 151 enterovirus (3 trials);101 103 116 Bacillus anthracis (3 trials);48 51 52 diphtheria and tetanus (2 trials);113 136 human papillomavirus (2 trials);65 72 Lyme borreliosis (2 trials);89 93 Haemophilus influenzae type B (2 trials);131 132 group B Streptococcus (2 trials);76 107 Staphylococcus aureus (2 trials);64 71 poliovirus (2 trials);124 125 Pseudomonas aeruginosa (2 trials);90 92 Alzheimer’s disease (2 trials);91 134 cytomegalovirus (2 trials);46 95 tetanus (1 trial);114 non-typeable Haemophilus influenzae (1 trial);107 Ross River virus (1 trial);81 hepatitis B (1 trial);112 malaria (1 trial);142 rabies and tetanus (1 trial);122 rabies (1 trial);123 Shigella flexneri (1 trial)77 and S. aureus and Candida albicans (1 trial).73

Overall, 41 trials assessed vaccines against extracellular pathogens (bacteria or toxins) and 61 trials assessed vaccines against intracellular pathogens (viruses).

Vaccine doses

The included trials administered different numbers of vaccine doses: 24% of the trials administered 1 dose;53 55 58 60 64 66 70 73 76 82 87 96 97 100 107 111 114 124 131 133 136 139 140 150 40% of the trials administered two doses;49–52 54 56 61–63 67 68 75 77 79 80 84 88 90 92 98 99 101–105 108 109 115–119 121 126 127 130 135 137 138 141 146 21% of the trials administered three doses;46 47 57 59 65 71 72 74 78 81 85 89 95 106 110 112 113 120 123 125 132 142 143 10% of the trials administered four doses;69 83 86 93 94 134 145 two trials administered five doses128 129 and one trial administered seven doses.91 Two trials did not specify the number of doses administered.122 144

Aluminium concentrations

The included trials used different aluminium concentrations ranging from 125 μg/dose to 6000 μg/dose.

Control groups

Two comparisons (from two trials) extracted in this review did not involve a vaccine (ie, comparison between saline placebo with or without aluminium).70 140 All the other control groups contained the same vaccine as the intervention group but without aluminium adjuvant.

Risk of bias within individual trials

Based on the information collected from the published reports and from authors, only 3/102 trials were at overall low risk of bias (all outcomes reported at low risk of bias). The remaining trials were at overall high risk of bias (online supplemental figure S2).

Effects of aluminium adjuvants

Serious adverse events

A total of 170/7627 (2.2%) participants who received aluminium adjuvants with or without vaccines suffered a serious adverse event vs 149/13 936 (1.1%) participants receiving no aluminium adjuvants with or without vaccines (risk ratio, RR 1.18, 95% CI 0.97 to 1.43; 21 563 participants; 62 trials; I2 0%; Bayes factor 548.28; very low certainty of evidence (figure 1, table 1, online supplemental figure S3). Visual inspection of the forest plot and I2 reveal no statistical heterogeneity. TSA showed that the cumulative Z-curve (blue full line with quadratic squares indicating each trial) touched the traditional boundary for harm. However, none of the trial sequential monitoring boundaries (etched curves above and below the traditional naive horizontal lines for statistical significance) were surpassed. The result is inconclusive as the required information size has not been achieved. The TSA-adjusted CI is 0.53 to 2.69 (Pc (proportion with an outcome in the control group) 1.0%, RR reduction or increase (RRR) 20%, alpha 2.5%, beta 10%, diversity 0%; diversity-adjusted required information size (DARIS) 110 696 participants) (online supplemental figure S4).

Figure 1

Meta-analysis of the effect of aluminium adjuvant compared with placebo or no intervention on the proportion of participants with one or more serious adverse events. M-H, Mantel-Haenszel.

Table 1

Summary of findings table

Subgroup analyses

Test for subgroup differences showed no difference when comparing the effects of aluminium adjuvants in trials at high risk of bias to trials at low risk of bias; in trials at risk of vested interest to trials at low risk of vested interest; trials according to different aluminium types; trials with different vaccines; trials with different participants’ ages; trials with different follow-up durations; trials with participants with different diagnoses compared with healthy participants; and trials assessing vaccines against different pathogens types (online supplemental Figures S5–S12).

Sensitivity analyses

A total of 21/7648 (0.3%) participants in the intervention group vs 18/13 954 (0.1%) participants in the control group were lost to follow-up. Incomplete outcome data alone seemed to have the potential to influence the result in the ‘worst-best’ case scenario analysis (online supplemental figure S13). The ‘best-worst’ case scenario analysis showed that incomplete outcome data did not have the potential to influence the result (online supplemental figure S14).

Meta-regression showed that the proportion of participants with serious adverse events was not affected by the aluminium concentration used in the vaccine (p=0.28).

Due to several trials with zero events, we performed meta-analysis also with OR. The results did not change (online supplemental figure S15).

Individual serious adverse events analyses

Meta-analyses showed no evidence of a difference between aluminium adjuvants vs control when assessing individual serious adverse events (online supplemental analysis S1). Individual serious adverse events reported only in one trial that were not possible to meta-analyse are shown in online supplemental table S2.

All-cause mortality

A total of 61/7782 (0.8%) aluminium participants died compared with 57/14 104 (0.4%) control participants (RR 1.02, 95% CI 0.74 to 1.41; 21 886 participants; 63 trials; I2 0%; Bayes factor 2.96; very low certainty evidence (figure 2, table 1, online supplemental figure S16). Visual inspection of the forest plot and I2 indicated no statistical heterogeneity. Funnel plot showed no publication bias. TSA showed that the accrued information for all-cause mortality was below 5% of the DARIS (Pc 0.4%, RRR 20%, alpha 2.5%, beta 10%, diversity 0%; DARIS 278 247).

Figure 2

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on all-cause mortality. M-H, Mantel-Haenszel.

Subgroup analyses

Test for subgroup differences showed no difference when comparing trials at high risk of bias to trials at low risk of bias; trials at risk of vested interest to trials at low risk of vested interest; trials with no vaccine cointervention to trials with vaccine cointervention; trials with different aluminium types; trials with different vaccines; trials with different participants’ ages; trials with different participants’ diagnoses; and trial assessing vaccines against different pathogens types (online supplemental figures S17–S23). Due to lack of relevant data, it was not possible to conduct the subgroup analyses on trials with different follow-up durations.

Sensitivity analyses

A total of 28/7909 (0.35%) participants in the aluminium group vs 20/14 173 (0.14%) participants in the control group were lost to follow-up. Incomplete outcome data alone seemed to have the potential to influence our result in the ‘worst-best’ case scenario showing a harmful effect of aluminium adjuvants compared with placebo (online supplemental figure S24). The ‘best-worst’ case analysis showed that incomplete outcome data did not have the potential to influence our result (online supplemental figure S25).

Meta-regression showed that the proportion of participants with all-cause mortality was not affected by the aluminium concentration used in the vaccine (p=0.88).

Due to several trials with zero events, we performed meta-analysis also with OR. The results did not change (online supplemental figure S26).

Participants with disease

Only two trials (one event) reported on the proportion of participants that developed the disease they were vaccinated against (online supplemental figure S27).

Adverse events considered non-serious

Out of the 67 trials reporting adverse events considered non-serious, 34 trials reported the overall proportion of participants with one or more adverse events considered non-serious. From the remaining 33 trials reporting on adverse events considered non-serious, we extracted data from the highest proportion of participants experiencing an individual adverse event.

A total of 3760/7098 (52.9%) aluminium participants experienced one or more non-serious adverse events compared with 4537/13 429 (33.8%) in control participants. Meta-analysis of these trials showed that aluminium adjuvants compared with placebo or no intervention may increase the proportion of participants with one or more adverse events considered non-serious (RR 1.13, 95% CI 1.07 to 1.20; participants=20 527; trials=67; I2=85%; Bayes factor 3.02E+26; the evidence was very uncertain (figure 3, table 1, online supplemental figure S28). TSA of non-serious adverse events shows that the cumulative Z curve crosses the boundary for harm, indicating that there was enough information to confirm that aluminium adjuvants compared with placebo or no intervention increases the risk of one or more non-serious adverse events (TSA Pc 33.5%, RRR 20%, alpha 2.5%, beta 10%, diversity 78%; DARIS 28 384. TSA adjusted CI 1.06 to 1.23 (online supplemental figure S29).

Figure 3

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on the proportion of participants with one or more non-serious adverse events. M-H, Mantel-Haenszel.

Visual inspection of the funnel plot and regression-based Harbord test showed no publication bias or small-study effects (beta=0.99).

Subgroup analyses

Test for subgroup differences was statistically significant in the subgroup analysis according to vaccine type (p<0.00001; online supplemental figure S30) and age (p=0.007; online supplemental figure S31).

Test for subgroup differences showed no difference when comparing the effect of aluminium adjuvants in trials at low risk of bias to trials at high risk of bias; in trials at low risk of vested interest to trials at risk of vested interest; in trials with different aluminium salts; in trials with different follow-up durations; in trials with participants with different health status; and trials assessing vaccines against different pathogen types (online supplemental figures S32–S37).

Sensitivity analyses

A total of 195/7392 (2.6%) participants in the aluminium group vs 186/13 341 (1.4%) participants in the control group were lost to follow-up. Incomplete outcome data did not have the potential to influence our results.

We included aluminium concentration (as described by trialists) as a covariate in meta-regression to assess whether aluminium concentration has an impact on the effect sizes of the proportion of participants with adverse events considered non-serious. Meta-regression showed that the proportion of participants with adverse events considered non-serious was not affected by the aluminium concentration used in the vaccine (p=0.68)

Individual non-serious adverse events

We performed meta-analysis on each of the 145 reported individual adverse event considered non-serious. Mainly, local injection site reactions were increased in the aluminium group (online supplemental analysis S2 and table S3).

Serological response

Serological response was assessed by different analytical assays and was reported as either geometric mean titre (GMT, 31 trials) or geometric mean concentration (GMC, 11 trials).

Meta-analyses showed no or little evidence of a difference between aluminium adjuvants vs placebo or no intervention when assessing GMT or GMC (figures 4 and 5, online supplemental figures S38 and S39).

Figure 4

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on the geometric mean titres grouped by analytical assay. IV, inverse variance.

Figure 5

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on the geometric mean concentrations grouped by analytical assay. IV, inverse variance.

Subgroup and sensitivity analyses for serology

Subgroup and sensitivity analysis for the serological response is reported in online supplemental figures S40–S43.

Seroprotection

Meta-analysis showed that there was no evidence of a difference between aluminium adjuvants compared with placebo or no intervention when assessing seroprotection (RR 0.95, 95% CI 0.77 to 1.18; trials=14; I2 78%. Bayes factor 3.11; low certainty of evidence) (figure 6, online supplemental figure S44). Visual inspection of the forest plot and I2 statistics indicated high heterogeneity (I2 78%).

Figure 6

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on seroprotection. M-H, Mantel-Haenszel.

Subgroup and sensitivity analyses for seroprotection

Subgroup and sensitivity analyses for seroprotection is reported in online supplemental text and figure S45 and S46.

Discussion

This review included 102 randomised clinical trials assessing a total of 26 457 participants. Aluminium adjuvants versus placebo or no adjuvants may have no effect on the proportion of participants with one or more serious adverse events and on all-cause mortality, but the evidence was very uncertain. Two trials reported on the proportion of participants with the disease they were vaccinated against. However, only one event was reported. None of the trials reported on quality of life. Aluminium adjuvants versus placebo or no adjuvants seem to increase the proportion of participants with one or more adverse events considered non-serious, but the evidence was very uncertain. We found no or little evidence of a difference between aluminium adjuvants vs placebo or no intervention when assessing geometric mean titres or concentrations. Aluminium adjuvants versus placebo or no intervention may have no effect on participants without seroprotection, but the evidence was very uncertain.

Strengths and weaknesses

We seem to be the first to conduct a systematic review comparing aluminium adjuvants versus placebo or no intervention in any type of vaccine. We followed our peer-reviewed protocol which was published before the literature search began,35 and we conducted the review using the methods recommended by Cochrane.37 152 We reported our review according to the PRISMA statement153 (online supplemental table S4).

Our systematic review has several limitations. Despite our inclusion criteria being broad, we could only find phase I or II trials that met our inclusion criteria. This limitation is because phase III or IV trials of marketed vaccines are mainly designed with an active comparator (another vaccine or alleged ‘placebo’ with aluminium), and therefore, these trial designs did not match the inclusion criteria of our review.

Another limitation of the applicability of our results is that we chose maximum follow-up as our time point of primary interest. This approach does not allow us to make conclusions on the effect of aluminium adjuvants on safety and immunogenicity after each individual vaccine dose (for those trials having multiple vaccine injections).

Another limitation is that we identified high clinical heterogeneity, especially within the immunogenicity outcome. Included trials did not use the same assays to assess the serological response and most of the trials had multiple assays performed. We chose to analyse only the geometric mean titre or concentration reflecting the data from the first assay presented in the publication; however, we are aware that this might limit the strength of our findings.

We chose to merge multiple groups in those trials that used vaccines with different antigen concentrations. In so doing, we were unable to conclude whether the effect of aluminium adjuvants is correlated to the effect of different antigen concentrations.

Only two trials (one event in total) reported on the proportion of participants with disease they were vaccinated against and none of the authors provided us with such data when contacted by email. Therefore, our conclusion regarding the effect of aluminium adjuvants on the immunogenicity is based on the surrogate outcome of the serological response to vaccine measured by different assays and on the seroprotection values as defined by the trialists.

Of the 62 trials that reported on serious adverse events, 8 trials reported only vaccine-related serious adverse events.49 51 81 100 104 107 108 115 Of the 62 trials that reported on serious adverse events, 14 trials reported that serious adverse events occurred but these were not assigned per intervention group because they were considered unrelated to the vaccine (a total of 107 serious adverse events reported to having occurred but not described per intervention group).50 66–68 77 83 84 86 91 97 118 121 134 146 Only 7/102 authors contacted provided us with all or some of the data requested (see characteristics of included studies in the online supplemental appendix 1).

Out of the 67 trials that reported on adverse events considered non-serious, 34 trials reported the overall proportion of participants with one or more adverse event considered non-serious.54 56 58 64 66 69 70 72 77–79 82 83 85 87 90 92 95 97 102 103 105–107 110 113 117 124 126 133 137 139 145 149 154 From the remaining 33/67 trials reporting on adverse events considered non-serious, we extracted data from the highest proportion of participants experiencing an individual adverse event.46 49–51 53 62 76 80 81 84 86 91 93 100 101 109 111 114 115 118 120 121 124 125 128 129 135 136 140–143 150 A substantial number of trials reported only solicited adverse events instead of solicited and unsolicited adverse events as a combined outcome. This limitation may have resulted in an underestimation of the unsolicited adverse events that might have occurred but were not reported.

Agreements and disagreements with other studies or reviews

Jefferson et al reviewed evidence of adverse events after exposure to aluminium-containing vaccines against diphtheria, tetanus and pertussis, alone or in combination, compared with identical vaccines, either without aluminium or containing aluminium in different concentrations.31 They included three randomised trials, four semirandomised trials, and one cohort study. They found that in young children, vaccines with aluminium hydroxide caused significantly more erythema and induration than plain vaccines and significantly fewer reactions of all types. In older children, there was an association with local pain lasting up to 14 days. Despite a lack of good-quality evidence, the authors surprisingly recommend against any further research on this topic.

Lin et al conducted the first meta-analysis on the efficacy of aluminium salts as an adjuvant for prepandemic influenza vaccines.32 They included a total of nine randomised clinical trials (published during 2006–2013), including 22 comparisons in 2467 participants that compared aluminium-adjuvanted H5N1 vaccines versus non-adjuvanted counterparts.32 Their results showed an inferior seroprotection after aluminium-adjuvanted H5N1 vaccines compared with that conferred by non-adjuvanted counterparts. Furthermore, H5N1 vaccines with aluminium adjuvants were associated with a significantly higher risk of pain/tenderness at the injection site during the 7 days after the first vaccination and after the second dose vs the non-adjuvanted counterparts.

Jørgensen et al set out to assess the benefits and harms of the HPV vaccines in clinical study reports obtained from the European Medicines Agency and GlaxoSmithKline from 2014 to 2017.155 They included 24 randomised clinical trials comparing an aluminium-adjuvanted HPV vaccine vs a placebo or active comparator in healthy participants of all ages. They found that at four years follow-up, the HPV vaccines decreased HPV-related precursors to cervical cancer and treatment procedures but increased serious nervous system disorders (exploratory analysis) and general harms.131 As the trials included in their review were primarily designed to assess benefits and not adequately designed to assess harms, the extent to which the benefits outweigh the harms was unclear.

In agreement with Lin et al,32 our systematic review does not find an increased serological response of aluminium-adjuvanted vaccines compared with that conferred by non-adjuvanted counterparts. Also, in agreement with both Jefferson et al31 and Lin et al32 we find an increase in local injection site reactions after administration of aluminium-adjuvanted vaccines.

Implications for practice and research

Considering the lack of good-quality evidence to assess beneficial and harmful effects of adding aluminium to vaccines as presented here, relevance of this adjuvant should be investigated in future studies. Questions on aluminium form, concentration and size remain unanswered due to scarcity or lack of data. Questions on the effects of aluminium adjuvants on vaccine effectiveness also remain unanswered.

Future randomised clinical trials in humans should be conducted according to the ethical principles that have their origin in the Declaration of Helsinki, and that are consistent with the International Council for Harmonisation Good Clinical Practice guidelines and the applicable regulatory requirement(s).156 157 Such trials should be designed in accordance with guidelines for clinical trials (Standard Protocol Items: Recommendations for Interventional Trials)158 and reported in accordance with the Consolidated Standards of Reporting Trials.159

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information. Data sharing not applicable as all data are avilable in figures and tables in text or supplements.

Ethics statements

Patient consent for publication

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors SRK, MB, SLK, SD, SBP, MK, DZK, JCJ and CG. Conception of the review: SD, SBP, MK, JCJ and CG. Co-ordination of the review: JCJ and CG. Search strategies and search for literature: SLK and DZK. Collection of data for the review: SRK, SLK and MB. Assessment of the risk of bias in the included trials: SRK, SLK and MB. Analysis of data: SRK, MB, JCJ and CG. Assessment of the certainty in the body of evidence: SRK, MB and CG. Interpretation of data: SRK, MB, JCJ and CG. Writing first draft of the review: SRK. Revision of the review: MB, SLK, SBP, MK, JCJ and CG. All authors approved of the current version (for publication).

  • Funding The Danish State is the largest single funder of the Copenhagen Trial Unit, Centre for Clinical Intervention Research, The Capital Region, Copenhagen University Hospital − Rigshospitalet, Copenhagen, Denmark. Award/Grant number is not applicable. We also thank Fact Care for economical support.

  • Disclaimer The views and opinions expressed in this review are those of the authors and do not necessarily reflect those of the Danish State or the Copenhagen Trial Unit.

  • Competing interests The authors would like to thank Dimitrinka Nikolova for outstanding advices and guidance. The authors would like to thank Aleksandra Mazur for contribution in data extraction. The authors would like to thank Elena von Rohden for help with German translation. The authors would like to thank the Copenhagen Trial Unit for providing salaries for SRK, MB, SLK, SD, JCJ and CG. Award/Grant number is not applicable. The authors would like to thank the Cochrane Central Editorial Unit for valuable feedback on an earlier version of this manuscript. MK: cofounder of HPV_update.dk.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.