Table 2

Summary of general characteristics of included tools

CharacteristicSummary data (n=18 tools)
Type of tool
 Domain-based16 (89%)
 Checklist1 (6%)
 Scale1 (6%)
Scope of tool
 Assessment of reporting bias only5 (28%)
 Assessment of multiple sources of bias/quality13 (72%)
Types of reporting bias assessed
 Bias due to selective publication only0 (0%)
 Bias due to selective non-reporting only6 (33%)
 Bias in selection of the reported result only3 (17%)
 Bias due to selective publication and bias due to selective non-reporting4 (22%)
 Bias due to selective non-reporting and bias in selection of the reported result5 (28%)
Total number of items in the tool7 (5–13)
Number of items relevant to risk of reporting bias1 (1–2)
Number of response options for risk of reporting bias judgement3 (3–3)
Types of study designs to which the tool applies
 Randomised trials only5 (28%)
 Systematic reviews only3 (17%)
 Non-randomised studies of interventions only2 (11%)
 Randomised trials and non-randomised studies of interventions2 (11%)
 Non-randomised studies of interventions or exposures2 (11%)
 Other (cross-sectional studies, animal studies, network meta-analyses, prognosis studies)4 (22%)
Level of assessment of risk of reporting bias
 Study as a whole9 (50%)
 Specific outcome/result in a study5 (28%)
 Specific synthesis of studies4 (22%)
Data sources used to inform tool content*
 Literature review (eg, of items in existing tools or empirical evidence)9 (50%)
 Ideas generated at expert consensus meeting8 (44%)
 Pilot feedback on preliminary version of the tool7 (39%)
 Data from psychometric or cognitive testing†5 (28%)
 Other (eg, adaptation of existing tool)5 (28%)
 Delphi study responses2 (11%)
 No methods stated2 (11%)
Guidance available
 Brief annotation per item/response option9 (50%)
 Detailed guidance manual4 (22%)
 Worked example for each response option2 (11%)
 Detailed annotation per item/response option1 (6%)
 None2 (11%)
  • Summary data given as number (%) or median (IQR).

  • *The percentages in this category do not sum to 100% since the development of some tools was informed by multiple data sources.

  • †Psychometric testing includes any evaluation of the measurement properties (eg, construct validity, inter-rater reliability, test–retest reliability) of a draft version of the tool. Cognitive testing includes use of qualitative methods (eg, interview) to explore whether assessors who are using the tool for the first time were interpreting the tool and guidance as intended.