Table 1

Main reasons for disagreements in cases of a different interpretation of the same information

Risk of bias ItemMain reasons for disagreementsN (%)*Examples of support for judgement from the review†
Random sequence generationConsider differently incomplete or unclear description73 (83)‘States ‘cluster randomisation by computer’’; Low risk of bias
‘Cluster randomisation by computer. No further information provided’; Unclear risk of bias
Confusion with allocation concealment9 (10)‘Allocation was done using sealed envelopes containing name of one of the two groups.’; Low risk of bias
Allocation concealmentConsider differently incomplete or unclear description49 (33)‘Not specified.’; High risk of bias
‘Method of concealment not described.’; Unclear risk of bias
Consider differently envelopes description26 (17)‘Sequentially numbered sealed envelopes’. Does not state if opaque envelopes.’; Unclear risk of bias
‘Sequentially numbered sealed envelopes.’; Low risk of bias
Random sequence generated by computer or external centre considered enough for low risk21 (14)‘Treatment was allocated based on the computer-generated no list.’; Low risk of bias
Confusion in the definition of the item19 (13)‘Researchers attempted to contact all patients seen by physicians during 1 month’; High risk of bias
Confusion with blinding15 (10)‘Participants were told to which compound they had been allocated.’; High risk of bias
Confusion with random sequence generation6 (4)‘Computer-generated randomised lists.’; Low risk of bias
Blinding of participants and personnelAssess risk differently if blinding was not feasible because of the type of intervention20 (36)‘Not possible to blind participants’; Low risk of bias
‘Participants were not blinded for provided treatment. This is inherent to study design’; High risk of bias
Outcome considered not influenced by blinding12 (21)No information given about whether patients were blind to physician allocation but treatment outcomes judged unlikely to be affected by lack of blinding’; Low risk of bias
Consider differently information of ‘double blind’9 (16)‘Quote: ‘ . . . patients were randomised in double-blind conditions . . . ‘Comment: probably done’; Low risk of bias
Quote: ‘double blind conditions’. No further details.’; Unclear risk of bias
Consider differently incomplete or unclear description7 (12)‘Researchers were blind until after the baseline assessment. participants were not blinded.’; Unclear risk of bias
‘Not possible to blind participants to intervention. Insufficient information to make a judgement about blinding of therapists’; High risk of bias
Confusion in the definition of the item5 (9)‘Described as an ‘open-label’ pilot study.’; Low risk of bias
Blinding of outcome assessmentConsider differently incomplete or unclear description24 (34)‘Not explicitly discussed in the publish study, it was assumed to be open label’; High risk of bias
‘Not described in published study’; Unclear risk of bias
Outcome considered not influenced by blinding16 (23)‘Not stated, but it was unlikely that the outcome was influenced by lack of blinding’; Low risk of bias
Consider differently patient-reported outcomes when patients are blinded or not to the intervention9 (13)‘Comment: depression assessed by patient self-report’; High risk of bias
‘Insufficient information available to assess’; Low risk of bias
Consider differently information of ‘double blind’9 (13)‘Quote: ‘ . . . double blind’ Comment: probably done’; Low risk of bias
‘Quote: ‘double blind conditions’. No further details.’; Unclear risk of bias
Assess risk differently if blinding was not feasible because of the type of intervention6 (9)‘Blinding not possible due to intervention’; High risk of bias
‘Unclear blinding of outcome assessment’; Low risk of bias
Incomplete outcome dataUse different cut-off for the rate of missing data57 (26)‘11 withdrawals (10%).’; Low risk of bias
‘Comment: there were post-randomisation drop-outs’; High risk of bias
Focus on no versus reasons/precise report of missing data28 (13)‘20 drop-outs (27.2%) with 4 deaths (3 males, 1 female) from cardiovascular events’; High risk of bias
‘Numbers and reasons for drop-outs and withdrawals in all intervention groups were described.’; Low risk of bias
Consider differently incomplete or unclear description27 (12)‘Women who were untraceable or unsuitable for follow-up were excluded, other losses included as smokers’; Low risk of bias
‘167/1287 (12.9%) (C=83, I=84) excluded from analysis due to moving away, being untraceable or deemed unsuitable for follow-up (eg, miscarriage). 1120 in sample. 51/1287 non-responders were included as continuing smokers.’ High risk of bias
Consider differently intention-to-treat (ITT) analysis25 (11)147 randomised; r4 in the letrozole group and 3 in the laparoscopic ovarian drilling   dropped out of the trial, all for non-compliance. However, ITT analysis was not conducted.’; Unclear risk of bias
7 women lost to follow-up, but similar (3 vs 4) in both groups; losses due to non-compliance’; Low risk of bias
Consider differently report of ‘no missing data’22 (10)Did not report number of withdrawals. Comment: all patients who were randomised were included in the final analysis. ITT analysis was conducted.’; Unclear risk of bias
‘It does not appear that there were any withdrawals or drop-outs Low risk of bias
Consider differently imputation of missing data20 (9)‘Imputation method not described’; Unclear risk of bias
Drop-out rate was not significant’; Low risk of bias
Use different cut-off for difference in the rate missing data between different arms/comparisons13 (6)Drop-out higher in placebo group (35% vs 25% in budesonide group). ITT used.’; High risk of bias
Similar rates of withdrawal between arms. Withdrawals: 36 BUD, 51 placebo’; Low risk of bias
  • *Number of RCTs disagreeing for this reason; percentage over the total of disagreements for different interpretation.

  • †When two extracts are reported, they refer to the same study.

  • RCT, randomised controlled trial.