Timeline cluster: a graphical tool to identify risk of bias in cluster randomised trials
BMJ 2016; 354 doi: https://doi.org/10.1136/bmj.i4291 (Published 16 August 2016) Cite this as: BMJ 2016;354:i4291
- Agnès Caille, associate professor in biostatistics1 2 3,
- Sally Kerry, reader in medical statistics4,
- Elsa Tavernier, PhD student in biostatistics2 3,
- Clémence Leyrat, research fellow in biostatistics5,
- Sandra Eldridge, professor of biostatistics4,
- Bruno Giraudeau, professor of biostatistics1 2 3
- 1Université François-Rabelais, Tours, France
- 2CHRU de Tours, INSERM CIC1415, Tours, France
- 3Centre de recherche Epidémiologie et Biostatistique, INSERM U1153, Paris, France
- 4Centre for Primary Care and Public Health, Queen Mary University of London, London, UK
- 5Department of Medical Statistics, London School of Hygiene and Tropical Medicine, London, UK
- Correspondence to: A Caille agnes.caille{at}med.univ-tours.fr
- Accepted 8 July 2016
Summary points
Cluster randomised trials can be at risk of bias when participants are identified and recruited after randomisation
Reports of cluster randomised trials often fail to adequately describe the recruitment process and whether participants and trial staff are blinded to allocation status at key stages of the trial
This article presents a graphical tool depicting the time sequence and blinding status of the different stages of a cluster randomised trial, together with examples to help researchers describe the storyline of such trials
Our graphical tool should be used at both the protocol and report writing stages to clarify the trial process and to help identify the risk of bias
In cluster randomised trials, clusters of subjects such as hospitals or family practices are randomised rather than people themselves.1 Cluster randomised trials are used for evaluating health service organisation and health policy, often with complex interventions targeted at the level of the cluster, the individual, or both. Randomisation should prevent allocation bias at the cluster level provided that it is properly conducted, but differences in individual level characteristics between the intervention arms can be reintroduced because of the relative timing of participant identification and recruitment, and cluster randomisation. Indeed, the usual chronology of an individual randomised trial with first recruitment and then randomisation of participants can be reversed in cluster randomised trials: the identification and recruitment of participants often take place after randomisation, which could lead to identification or recruitment bias (hereafter called recruitment bias).2 3 Because blinding is rarely possible for interventions assessed in cluster randomised trials, previous knowledge of the allocation from recruiters or participants can influence who is approached and who agrees to participate in a trial. This might lead to different recruitment rates between arms as well as imbalance in participant characteristics.4 5 6 Some solutions proposed to prevent recruitment bias include the identification and recruitment of participants before cluster randomisation or recruitment of participants by a blinded and independent person.7 These solutions should be considered whenever possible, but they are not always feasible or applied. Furthermore, cluster randomised trials are prone to other biases, usually encountered when blinding is lacking—namely, performance bias and detection bias.8 Performance bias refers to systematic differences between the care delivered to experimental and control arms other than the intervention under investigation. Knowledge of the allocation by participants, intervention providers, or other trial staff leads to a risk of performance bias by contamination (with delivery of one of the trial interventions in clusters, or participants allocated to receive the other intervention) or by difference in the delivery of cointerventions. Because cluster randomised trials are pragmatic, the control arm often consists of usual care (or no intervention). Thus a particular attention might be paid to information provided to participants and care providers in this group, in that precise information about the experimental intervention could change their behaviour during the trial and lead to contamination. Detection bias refers to systematic differences between arms in how outcomes are assessed. As in individual randomised trials, in cluster randomised trials, knowledge of the allocation by outcome assessors can affect outcome measurement, in particular when outcome measurement involves some judgment from the assessor or is directly reported by the participant.
To assess the risk of bias, an accurate description of the distinct procedures is required, but despite existing recommendations,4 the reporting of both the recruitment process and the blinding status for participants and trial staff is often incomplete.9 10 To help researchers with this description, we developed a graphical tool depicting the sequence and blinding of the different steps of a cluster randomised trial and whether the intervention arms are treated the same or not.
Development of the Timeline cluster tool
The working group consisted of AC, SK, CL, and SE, all statisticians who have been involved in the planning, analysis, and reporting of cluster randomised trials as well as in methodological and statistical research on this design. Early in 2015, AC initiated a first version of the graphical tool from real cluster randomised trials. In August 2015, the working group attended a one day meeting to discuss this first version of the graphical tool. During the meeting, decisions were made by informal consensus regarding stages that need to be reported and how to better represent cluster and participant levels as well as about blinding. After this first meeting, AC developed a second version of the graphical tool. ET helped AC in the design of this second version (and later versions). Documents were shared by email, and several email iterations took place. Feedback was requested from the whole working group. We also incorporated feedback received from a presentation of the graphical tool at a meeting on current developments in cluster randomised trials and stepped wedge designs held in London under the auspices of the Royal Statistical Society in October 2015 (all members of the working group were part of the 31 attendees at this meeting). Pilot use of the tool was performed by chief investigators (n=3) and statisticians (n=2) on published or ongoing cluster trials. The latest version of the Timeline cluster tool also takes into account editors’ and reviewers’ comments.
The Timeline cluster tool
The Timeline cluster tool consists of a diagram and table displayed together (see figs 1-3⇓ for examples). The diagram represents the sequence of stages of the trial process using successive boxes. Randomisation of clusters is a key stage and is symbolised by a two way black arrow. All the following stages should be reported when applicable: the identification and recruitment of both clusters and participants, randomisation, intervention delivery, and baseline and outcome assessment of participants. The cluster level is symbolised by a ring and the participant level by a stick figure. Blinding status is depicted by the background colour of the boxes. If blinding is complete (all involved protagonists are blinded to allocation), the background of the box is black. If blinding is lacking (no involved protagonist is blinded), the background is white. If blinding is partial (blinding has been used to avoid bias, but not all protagonists are blinded to allocation), the background is grey. Examples of partial blinding include when some protagonists are blinded but others are not or when some protagonists are masked to the hypotheses of the trial. Also stages occurring before randomisation must have a black background not because of blinding (as no allocation has been made) but because these steps cannot be affected by subsequent allocation. When the control arm receives usual care only, neither symbol appears on the right of the box because nothing is added by the trial at this stage for clusters or participants compared with standard care. Stages that differ between the intervention arms (at least intervention delivery stage) should be represented by two separate rectangles drawn on each side of the dotted line. The table should at least provide justification for blinding status and other essential details to interpret the diagram, such as the information provided to participants within each arm. The remaining information added in the table is at the user’s discretion and replaces what would have been reported in the full text.
Fig 2 Example of Timeline cluster diagram for cluster trial with risk of recruitment bias: cluster trial evaluating a hip protector to reduce hip fractures in older adults13
Fig 3 Example of Timeline cluster diagram for cluster trial with prevention of recruitment bias: the ELECTRA trial, evaluating a specialist nurse intervention to reduce unscheduled asthma care in a multiethnic area14
Examples
A cluster randomised trial with identification and recruitment of participants before randomisation
The PEACH trial11 12 is a cluster randomised trial of coaching of people with type 2 diabetes by practice nurses. The Timeline cluster diagram for this trial (fig 1⇑) shows that first the clusters are identified and recruited, then participants are identified and recruited, and then baseline characteristics are collected before the clusters are randomised. All these stages are performed before randomisation, so the corresponding rectangles have black backgrounds and there is no risk of recruitment bias. For the intervention delivery, in the experimental arm the rectangle has a ring plus a stick figure to the right, and in the control arm, neither symbol. We can conclude that the intervention in the experimental arm is delivered at both cluster and individual levels and consists of usual care in the control arm. The background of the rectangle is white because of no blinding at that stage. The table confirms no blinding for general practitioners, practice nurses, and participants, so we cannot exclude performance bias. Finally, blinding is complete for the outcome, change in glycated haemoglobin (HbA1c) level from baseline to 18 months; there is no risk of detection bias. The equivalent description is about 700 words in the protocol publication11 and 600 words in the trial report.12
A cluster randomised trial with identification and recruitment of participants after randomisation
Figure 2⇑ is the Timeline cluster diagram for a cluster randomised trial evaluating a hip protector to reduce hip fractures in older adults.13 Before randomisation, clusters are identified and recruited, then participants are identified and assessed. After randomisation, participants are recruited without blinding because recruiters and participants are aware of the allocation and there is a risk of recruitment bias. Blinding is lacking for the intervention delivery targeted at the individual level in the experimental arm and consisting of usual care in the control arm, leading to possible performance bias. Blinding is complete for the primary outcome, hip fracture recorded at the participant level and documented by radiographs. The potential for recruitment bias is confirmed by the trial results, with a higher rate of consent and a lower proportion of participants with a history of falls or severe cognitive impairment at baseline in the control arm compared with the experimental arm.13
A cluster randomised trial with identification and recruitment of participants after randomisation with measures to prevent bias
The ELECTRA trial14 is a cluster randomised trial of a specialist nurse intervention to reduce unscheduled asthma care in a multiethnic area. Even though the identification, recruitment, and baseline assessment of individual participants are performed after randomisation of clusters, rectangles for these stages all have a black background because some measures are used to obtain complete blinding and thus prevent recruitment bias (fig 3⇑). Further details provided in the table indicate that a blinded researcher is used to identify and recruit participants. There is no blinding for the intervention, targeted first at the cluster level before participant recruitment, then at the participant level after participant recruitment in both study arms. Some measures are used to avoid detection bias: general practitioners who complete patient records are not blinded, but researchers who extract the primary outcome from the general practice records are blinded; this partial blinding for outcome assessment is represented by the rectangle’s grey background.
Comparison with other graphical representations
The Timeline cluster diagram is distinct from the Consolidated Standards of Reporting Trials (CONSORT) flowchart,4 which shows the flow of clusters and participants by number approached, randomly assigned, receiving the allocated intervention, and included in the analysis for the primary outcome, together with the justification for losses and exclusions. The flowchart does not provide information on the chronology of the different stages or blinding of stages. In brief, the CONSORT flowchart is the “How many” and “Why (some participants are excluded)” of the trial, whereas the Timeline cluster diagram is the “When” and “How.” Our graphical tool is also distinct from other proposed diagrams that aim to better describe complex interventions either to clarify the timing and differences between arms of their different components (PaT plot method)15 or to depict interactions between intervention providers at several levels (cascade diagram).16 These diagrams provide more detail about the intervention delivery stage only and are useful to enable reproducibility of tested interventions. All these graphical methods are complementary.
Flexibility of the Timeline cluster tool
Our tool offers great flexibility for most cluster trial designs because stages can be added, repeated, or removed. For a cluster trial with no need to recruit participants because individual data are obtained from routinely collected data, as in the IRIS trial,17 the participant recruitment stage can be removed, with only the participant identification stage retained. Also, a given stage can be performed at different times depending on the intervention arm, as in the cluster trial of Cheyne and colleagues,18 which evaluated the use of an algorithm to diagnose active labour: women provided consent at admission in the experimental arm and on the postnatal wards in the control arm. This situation of a differential timing for participant recruitment could be represented by a participant recruitment box before intervention delivery in the experimental arm and after intervention delivery in the control arm. For a cluster trial with a repeated cross sectional design as the one reported by Murphy and colleagues,19 in which clusters are followed over time but participants change during the trial, the need to repeat the identification and recruitment of participants after intervention delivery could be represented by a loop starting and ending on the central dotted line of the diagram. A cluster crossover trial such as REPHVIM trial20 could be depicted by adding two crossing and ascending arrows to depict the switch from one intervention to the other at the end of the first period.
See the supplementary file for Timeline cluster diagrams corresponding to these four trials.
Discussion
We have proposed a simple and adaptable graphical approach to represent the chronology, blinding, and differences between arms in a cluster randomised trial that allows a quick overview of a given study. The Timeline cluster diagram can be useful at both the design and the reporting stages of a cluster randomised trial. At the design stage, the tool might help researchers identify threats to internal validity and consider ways to improve the methodology of their trial, such as use of a recruiter blinded to allocation status. It could also help to adequately implement the trial process in each cluster. At the reporting stage, a more detailed version of the tool can be provided by completing the table with what actually happened during the trial. Providing a precise and adequate description of what was done will help readers understand the timeline of a trial and appraise the risk of bias. We recommend that future users provide an interpretation of the Timeline cluster diagram in the Discussion section of their trial report. Most often, risk of bias depends on the amount of boxes with black, grey, and white backgrounds; indeed, the more black backgrounds, the lower the risk of bias, and the more white backgrounds, the greater the risk of bias, with grey reflecting intermediate or possible risk of bias. However, background colour is not completely associated with the risk of bias level: if the background colour is black, there is no risk of bias at the corresponding stage, but if the background colour is grey or white, the risk of bias must be assessed in the light of other information. For example, if the primary outcome is survival, and outcome assessors of vital status are not blinded to allocation, the box for outcome assessment must have a white background even though there is only low risk of detection bias.
Weaknesses in design and conduct as well as incomplete reporting of biomedical studies are important leading causes of research waste.21 22 Risk of bias, specifically recruitment bias, is a problem in cluster randomised trials, with remaining room for improvement. We believe that our graphical approach could help achieve better management and reporting of cluster randomised trials, allowing for an informed assessment of the risk of bias. We have received positive feedback from the investigators and statisticians who have used the current version of the Timeline cluster tool, but we anticipate that some further enhancement will probably be required. Therefore, we encourage suggestions from readers and feedback from the practical experience of future users.
Footnotes
We thank Catherine Laporte, Chris Gale, and Ken Haguenoer, investigators of cluster randomised trials, and Patty Chondros and Karla Hemming, statisticians, for testing and giving feedback on the Timeline cluster graphical tool; and the reviewers of this manuscript for helpful comments. A Word template of the Timeline cluster graphical tool is available on request from the corresponding author.
Contributors: BG had the idea of the graphical tool. AC conceived the graphical tool and drafted the original manuscript. SK, ET, CL, SE, and BG helped in the creation of the graphical tool and contributed to the final version of the manuscript. AC is the guarantor.
Competing interests: We have read and understood the BMJ policy on declaration of interests and declare the following: none.
Provenance and peer review: Not commissioned; externally peer reviewed.