Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial

Introduction Transparent and accurate reporting is essential for readers to adequately interpret the results of a study. Journals can play a vital role in improving the reporting of published randomised controlled trials (RCTs). We describe an RCT to evaluate our hypothesis that asking peer reviewers to check whether the most important and poorly reported CONsolidated Standards of Reporting Trials (CONSORT) items are adequately reported will result in higher adherence to CONSORT guidelines in published RCTs. Methods and analysis Manuscripts presenting the primary results of RCTs submitted to participating journals will be randomised to either the intervention group (peer reviewers will receive a reminder and short explanation of the 10 most important and poorly reported CONSORT items; they will be asked to check if these items are reported in the submitted manuscript) or a control group (usual journal practice). The primary outcome will be the mean proportion of the 10 items that are adequately reported in the published articles. Peer reviewers and manuscript authors will not be informed of the study hypothesis, design or intervention. Outcomes will be assessed in duplicate from published articles by two data extractors (at least one blinded to the intervention). We will enrol eligible manuscripts until a minimum of 83 articles per group (166 in total) are published. Ethics and dissemination This pragmatic RCT was approved by the Medical Sciences Interdivisional Research Ethics Committee of the University of Oxford (R62779/RE001). If this intervention is effective, it could be implemented by all medical journals without requiring large additional resources at journal level. Findings will be disseminated through presentations in relevant conferences and peer-reviewed publications. This trial is registered on the Open Science Framework (https://osf.io/c4hn8).

I, the Submitting Author has the right to grant and does grant on behalf of all authors of the Work (as defined in the below author licence), an exclusive licence and/or a non-exclusive licence for contributions from authors who are: i) UK Crown employees; ii) where BMJ has agreed a CC-BY licence shall apply, and/or iii) in accordance with the terms applicable for US Federal Government officers or employees acting as part of their official duties; on a worldwide, perpetual, irrevocable, royalty-free basis to BMJ Publishing Group Ltd ("BMJ") its licensees and where the relevant Journal is co-owned by BMJ to the co-owners of the Journal, to publish the Work in this journal and any other BMJ products and to exploit all rights, as set out in our licence.
The Submitting Author accepts and understands that any supply made under these terms is made by BMJ to the Submitting Author unless you are acting as an employee on behalf of your employer or a postgraduate student of an affiliated institution which is paying any applicable article publishing charge ("APC") for Open Access articles. Where the Submitting Author wishes to make the Work available on an Open Access basis (and intends to pay the relevant APC), the terms of reuse of such Open Access shall be governed by a Creative Commons licence -details of these licences and which Creative Commons licence will apply to this Work are set out in our licence referred to above.
Other than as permitted in any relevant BMJ Author's Self Archiving Policies, I confirm this Work has not been accepted for publication elsewhere, is not being considered for publication elsewhere and does not duplicate material already published. I confirm all authors consent to publication of this Work and authorise the granting of this licence. allocation concealment (50%). 21 This lack of transparency is a major limiting factor for readers 118 who assess an article in order to find the answer to a specific question; it is also a major reporting of methodological aspects of RCTs" and "the type of changes requested by peer 144 reviewers" found that peer review did lead to some improvement in reporting. 26

146
The role of peer reviewers and expectations of them is varied. 29 While CONSORT checklists 147 are sometimes available for peer reviewers to check, they are not typically instructed to assess The population will be defined on two levels: included journals and included manuscripts.

167
Inclusion criteria for journals: short summary information sheet will be included as part of the email invitation sent to journal 174 editors. If a journal is eligible, and the editor agrees to take part, the editor will need to provide 175 access to their editorial system (e.g. ScholarOne, Editorial Manager) to enable the external 176 researcher (BS) to screen and randomise eligible manuscripts. In cases where this is not 177 possible, we will explore with individual journals if it would be possible to grant limited access 178 (e.g. only rights to screen studies) or to handle the different steps without access to the editorial 179 system (e.g. screening through automated reports; intervention provided by a journal staff 180 member) and that the emails for the intervention would be sent by a member of the editorial 181 team.

183
Inclusion criteria for manuscripts 184  All new manuscript submissions reporting the primary results of RCTs, which the 185 journal editor has decided to send out for external peer review. Since the 10 chosen

186
CONSORT checklist items (C-short) are applicable to different study designs, we will 187 include all manuscripts reporting the primary results of RCTs regardless of study design 188 (e.g. parallel group trial, cluster trial, superiority trial, non-inferiority/equivalence trials).

189
Exclusion criteria for manuscripts 190  Manuscripts clearly presenting secondary trial results, additional time points, economic 191 analyses, or any other analyses.

192
 Manuscripts which are clearly labelled as a pilot or feasibility study or animal studies.

193
 Manuscripts not sent for peer review. Details of journal manuscript submission and peer review processes, including consent and 196 potential confidentiality issues will be discussed in detail with each journal by teleconference 197 and/or face to face prior to the journal agreeing to take part to ensure that randomisation of 198 manuscripts is feasible.

200
In participating journals, the external researcher (BS) will check at least twice a week (by 201 screening automated submission lists) all research manuscripts that are sent out for external 202 peer review. As soon as the first invited peer reviewer accepts the invitation to review, the 203 manuscript will be randomised to the intervention or control arm (see "Randomisation" for more 204 details). It is possible that this process might be slightly different amongst different included 205 journals (e.g. that team members of a journal might be involved in the screening if limited or 206 no access to the journal's editorial system is granted). After accepting to review a manuscript, peer reviewers will receive the automated, journal 212 specific standard email with general information as per each journal's usual practice (e.g.

213
where to access the manuscript, date the peer review report is due).

215
Intervention group: C-short plus usual practice

216
After accepting to review a manuscript, peer reviewers will receive the automated, journal 217 specific standard email with general information (identical to control group). In addition, peer 218 reviewers will receive an additional email from the editorial office that includes a short version 219 of the CONSORT checklist (C-short) together with a brief explanation of the items either as a 220  C-short checklist are addressed in the manuscript and to request authors to include these 223 items if they are not adequately reported. This second email (see appendix 1), containing the 224 C-short checklist together with a brief explanation, is not generated automatically within the 225 existing journal editorial systems (e.g. ScholarOne or Editorial Manager); it will be sent 226 manually by a researcher (BS) from the journal's editorial system or by a member of the 227 journal's staff. In both cases the email will appear to have come from the editorial office (not 228 the researcher).

230
Development of the C-short checklist and explanation of items 231 For the development of C-short we chose the 10 most important and poorly reported 232 CONSORT items as identified by a group of CONSORT experts in a previous study conducted 233 by Hopewell and colleagues. 28 The selection of the items was based on expert opinion and 234 empirical evidence whenever available. 28 In addition, to enable peer reviewers to better 235 understand the items, we added a short explanation for each of the 10 items. These short 236 explanations were extracted and amended from the CONSORT explanation and elaboration 237 paper 10 and from COBWEB which is an online writing aid tool. 30 The short explanation was 238 discussed and adapted by the scientific committee. The primary outcome of this study will be the difference in the mean proportion of adequately 243 reported C-short items in published articles between the two groups.  articles considering each sub-item (see "Assessment of outcomes") as a separate item.

251
 Time from assigning an editor to the first decision (as communicated to the author after 252 the first round of peer-review).

253
 Proportion of manuscripts rejected after the first round of peer review.

254
 Proportion of manuscripts that will be published in the journal under study.

256
Additional outcomes: 257  Exploratory analysis of available peer reviewer comments (i.e. any references to 258 CONSORT).

259
For journals where peer reviewers' comments are subsequently published alongside the 260 published article, we will examine the peer reviewers' comments for any reference to 261 CONSORT and trial reporting. We will contact those journals which do not make peer 262 reviewers' comments publicly available, to see if reviews could be provided for such analyses 263 under the condition that only anonymised data will be published.

265
Assessment of outcomes: 266 The outcomes will be assessed independently by two (blinded or at least partially blinded; see 267 "blinding") outcome assessors with expertise in the design and reporting of clinical trials. Any 268 disagreement will be resolved by consensus or if necessary by consulting a third assessor. To 269 ensure consistency between reviewers, we will first pilot the data extraction form; any 270 disparities in the interpretation will be discussed and the data extraction form will be modified 271 accordingly.

273
Adequate reporting of items will be assessed in duplicate from published full-text publications 274 following the same instructions as provided by the CONSORT C-short checklist. 10 The

275
following checklist items have, due to their complexity, sub-items which will be extracted  conduct, analysis, and reporting.

287
All items will be judged as either "yes" meaning adequately reported, "no" meaning not 288 adequately reported or not reported at all, or "NA" meaning that this sub-item is not applicable 289 for this RCT. Items with different sub-items will only be judged as adequately reported if all 290 relevant sub-items were adequately reported.

292
The outcomes "time from assigning an editor to the first decision", "proportion of manuscripts 293 rejected after the first round of peer-review", and "proportion of manuscripts that will be 294 published in the journal under study" will be extracted directly from the journal's editorial system 295 or provided by the journal.

Participant timeline
298 The overview of the study schedule, including enrolment, intervention and assessments is 299 presented in Table 2. For the sample size calculation, we hypothesised in a first scenario ( Table 3) that the 303 intervention C-short will result in a 25% relative increase in adequate reporting compared to 304 the control (meaning that 70% of items will be adequately reported in the intervention group 305 and 56% in the control group). This is based on a proportion of adequate reporting of 0.56 for 306 the 10 most important and poorly reported items found in the control group of a previous study 307 (meaning that a mean of 56% of the 10 most important and poorly reported items were 308 reported). 28 The standard deviation (SD) in the same study was 0.23. However, we calculated 309 our sample size to account for a slightly larger variability in our data (SD = 0.25). To 310 demonstrate a significant difference with a power of 90% and a type 1 error at 5%, a total of 311 136 published articles will be required in this scenario (68 per treatment arm; based on a two 312 sided t-test).

314
Two authors of this protocol, working for PLOS ONE (IP and AC), one of the participating 315 journals, pointed out that 3 out of the 10 assessed items (i.e. item "Registration", "Protocol",

316
and "Funding") should always be implemented in submissions to their journal given their policy 317 requirements for clinical trials. Assuming that this journal will recruit a high proportion of 318 manuscripts, and that also other journals might update their templates, we increased the 319 sample size in a second scenario, in which all these 3 items would have an overall adherence 320 of 90% in the control arm (Table 3). This would entail an overall baseline adherence with the 321 10 C-short items of 71%. Based on a two sided t-test, a sample size of 166 (83 per treatment 322 arm) will have a power of 80% to find a 15% relative increase (71% adherence in control group;

324
Since the final sample size will be based on the number of articles published, rather than on 325 the number of manuscripts randomised, eligible manuscripts will be randomised until 83 326 articles are published in each arm (resulting in no less than 166 articles), to avoid loss of power 327 due to potential imbalance between arms. Recruitment will be stopped as soon as both arms reach the sample size of 83. After recruitment has stopped we will wait three months so that 329 manuscripts, which are still in production, can be published. Manuscripts which are published 330 after the three month period will be excluded 331 332

333
Manuscripts meeting the eligibility criteria and sent out for external peer review by the journals 334 will be randomised into one of the two groups (allocation 1:1). The randomisation list will be 335 created by the Study-Randomizer © system 31 using random block sizes between 2 and 8 and 336 stratified by journal. As soon as the first peer reviewer accepts the invitation, the manuscript 337 will be included and randomised to one of the two study arms. One of the investigators (BS) 338 will log onto the Study-Randomizer © system 31 and enter the study identification number (ID; 339 provided by the journal), the study title, and the journal the study was submitted to.

340
Subsequently, all additional peer reviewers accepting the invitation to review the same 341 manuscript will receive the same group assignment as the first peer reviewer.

343
Authors will be blinded to the intervention. Editors will not be actively informed about the 344 randomisation (possible exception listed under "Interventions"). To avoid potential bias, peer 345 reviewers and manuscript authors will not be informed of the study hypothesis, design and 346 intervention.

348
Outcomes will be assessed in duplicate (see "Assessment of outcomes"). At least one outcome 349 assessor will be blinded. Due to restricted resources the investigator conducting the 350 randomisation (BS) might be involved in the data-extraction from published manuscripts. All quantitative variables will be described using means and standard deviations, or medians 354 and interquartile ranges in case severe departures from a normal distribution are identified.

355
Data distributions will be inspected visually (i.e. by histograms) instead of performing formal 356 statistical tests for normality. Categorical variables will be described using frequencies and 357 percentages. For the primary and secondary outcomes, we will estimate the mean difference 358 between the two groups and report them with respective 95% confidence intervals. No interim 359 analysis will be conducted.

361
Populations of analysis

362
The main population for analysis will be all manuscripts randomised and accepted for 363 publication in the participating journals. In contrast to RCTs conducted with patients, where 364 losses to follow-up need to be carefully considered (e.g. multiple imputation of missing data),

365
we are only interested in the reporting adherence of RCTs that are published. Hence we will 366 exclude randomised manuscripts that were not published from the main analysis. All outcomes 367 will be calculated based on the main population. The secondary outcome "Time to the first 368 decision", will additionally be calculated considering all randomised manuscripts (including the 369 ones which were not published). For all analyses a p-value of 0.05 (5% significance level) will 370 be used to indicate statistical significance. Exact p-values will be presented up to three decimal 371 places. We anticipate there will be no missing data in this study, neither at the individual C-372 short items, nor at the manuscript level. This is due to the study design, which will include only 373 the randomised manuscripts that are accepted for publication. We will analyse if the rate of 374 manuscripts rejected after the first round of peer-review and if the proportion of manuscripts 375 that will be published differentiate amongst the two study arms (both secondary results). The effect of the intervention will be estimated as the mean difference in the proportion of C-379 short items adequately reported between the study arms. If the data on the primary outcome 380 is normally distributed, groups will be compared using an unpaired Student's t-test. If the data 381 is not normally distributed, comparisons will be performed using a non-parametric equivalent 382 test (i.e. Wilcoxon-Mann-Whitney test).

383
Analysis of secondary endpoints

384
To investigate the effect of the intervention on the secondary outcomes, mean differences with 385 respective 95% confidence intervals will be reported. If normality is not observed for any of the 386 continuous secondary outcomes, the same strategy adopted for the primary outcome (use of 387 a non-parametric equivalent to the Student's t-test) will be used.

389
Pre-specified subgroup analysis

390
No formal subgroup comparative analysis is planned for the primary or secondary outcomes.

391
However, the effect of the intervention on the primary outcome within subgroups will be 392 presented using forest plots to visually examine whether it may differ according to some

414
The raw data extracted from the included published manuscripts can be made openly 415 accessible in an anonymised way (i.e. giving the included RCT a number instead of identifying 416 them). Derived/aggregated data, including anonymised information generated from the 417 journal's editorial system, will be stored and made available to the research community when 418 the project ends (see also "Publication policy and access to data"). Where appropriate, the 419 researcher who has access to the journal's editorial system (BS) and anyone else who will see 420 the identifiable data will sign a confidentially agreement with the participating journals,
Dear *Title, Name*, We thank you for accepting to peer-review a manuscript for *journal name*. As we are trying to improve the reporting for randomised controlled trials according to the CONSORT guidelines, we would like to ask if you could check whether the following most important and poorly reported items are adequately implemented as indicated in the     (1)(2)(3). Besides the complexity and the high associated costs of conducting RCTs (4-6), there are major issues with their reporting that often make it difficult for researchers, clinicians, patients or policymakers to interpret the current evidence on a specific topic (7,8). Chronologically, the most prominent difficulties in reporting consist of (i) poor reporting in study protocols for RCTs (9)(10)(11)(12); (ii) a substantial fraction of trials are not registered, prematurely discontinued (most common due to difficulties with recruitment) and not published (13,14); and (iii) that published RCTs are often poorly reported (7).
For clinicians, scientists and decision makers, published articles are often the only way to know how a study was conducted. In order to judge the internal and external validity of RCTs, it is crucial that these articles present transparent, accurate and unbiased information about the methods and conduct of the RCT. is important to note, that adhering to the CONSORT Statement does not mean that the study is of high quality. However, reporting all items from the CONSORT list will enable readers to adequately judge the quality of RCTs.  (3,(25)(26)(27)(28)(29)(30). Despite some improvement in reporting following the implementation of the CONSORT Statement, there still remain major reporting deficiencies in published RCTs (31).

For example, Odutayo and colleagues showed that a large proportion of RCTs published in
December 2012 in PubMed did not define the primary outcome (31%), did not state the sample size calculation (45%) and did not explain the method of allocation concealment (50%) (32).
This lack of transparency is a major limiting factor for the reader who assesses an article in order to find the answer to a specific question; it is also a major problem for scientists who perform systematic reviews and meta-analyses. Thus, some published trials may not be included in the meta-analysis because of their lack of transparency. Chan showed (25, 33) that 50% of efficacy outcomes and 65% of safety outcomes could not be included in meta-analyses because of how they were reported. Furthermore, even if these trials are included in systematic reviews and meta-analyses, an adequate risk of bias assessment is often not possible due to the poor reporting quality. Nevertheless, the main consequence of the lack of transparency is the risk of accepting treatments that are ineffective or cause serious adverse events (34). In a study published in 2016 authors of RCTs were asked by journal editors to use the webbased CONSORT tool at the manuscript revision stage (38). Authors who were randomly allocated to the intervention had access to a tool which allowed them to combine different CONSORT extensions (according to study design, medical field) to generate customised checklists. In the control group, authors had access to a CONSORT flow diagram generator.
The goal was to improve the reporting of CONSORT items with a simple webtool. However, a quarter of all authors either wrongly selected a CONSORT extension or failed to select an extension, indicating that further education is needed in terms of when and how to implement CONSORT extensions. inviting an additional statistical peer-reviewer (40,41)). Therefore, it is unlikely that these interventions will be implemented in the vast majority of journals, especially not in smaller journals with limited resources. A study examining "the nature and extent of changes made to manuscripts after peer review, in relation to the reporting of methodological aspects of RCTs" and "the type of changes requested by peer reviewers" found that peer review did lead to some improvement in reporting (40).
Building on these findings we plan to evaluate the impact of inviting peer reviewers to explicitly use a short version of the CONSORT checklist (including a short explanation of those items) as part of their review process. If this intervention deems to be effective, it could be easily implemented by all medical journals without needing additional resources at a journal level.

Hypothesis
We propose an RCT to evaluate the impact of asking peer reviewers to use a short version of the CONSORT checklist when reviewing a manuscript of an RCT and whether it improves the completeness of reporting. Our hypothesis is that reminding peer reviewers of the CONSORT items (including a short explanation of those items) will result in higher adherence to CONSORT guidelines in published RCTs. We only selected a limited number of the CONSORT items because we did not want to deter peer reviewers with too much information. Since peer reviewing in general can be burdensome, we felt that this approach is more promising than listing all items, risking that the information will be ignored. The short version of the CONSORT checklist is based on the same items described in a previous study as the 10 most important and underreported CONSORT items (38).

Main objective
The main objective of this study is to evaluate the impact of asking peer reviewers during the standard peer-review process to ask them to use a short version of the CONSORT checklist (C-short) and whether it will improve the reporting in published RCTs compared to manuscripts where the peer reviewers underwent usual practice.

Trial design
This study is a multicentre RCT with articles being the unit of randomisation ( Figure 1; allocation ratio 1:1). A multicentre parallel arm RCT with randomisation at the individual article level was chosen instead of a cluster RCT because the risk of any "contamination" on journal level is not given as the intervention will be implemented by an external researcher (i.e. BS).
The possibility of contamination due to the possibility that peer reviewer are invited to assess several RCTs and are randomised into both arms was judged as relatively small and therefore we do not plan to adjust for clustering by journal. The journal staff (i.e. editors) will not be actively told which manuscript was allocated to the proposed intervention and which to the control group.  of the requirements for participation and a short summary information sheet will be included as part of the email invitation sent to journal editors. If a journal is eligible, and agrees to take part, the journal will also need to provide access to their journal editorial system (e.g. ScholarOne, Editorial Manager) to enable the external researcher (i.e. BS) to screen and randomise eligible manuscripts. In cases this is not possible, we will explore with separate journals if it would be possible to grant limited access (e.g. only rights to screen studies) and that the emails from the intervention would be sent by a person from the editorial team.
We will include all submitted manuscripts reporting RCTs for which the journal decides to send out for external peer review. Since the 10 chosen CONSORT checklist items are applicable to different study designs, we will include all RCTs regardless of study design (e.g. parallel group trial, cluster trial, superiority trial, non-inferiority trial). Articles presenting clearly secondary trial results, additional time points, economic analyses, or any other analyses derived from an RCT dataset not including the study's main results will be excluded. Furthermore, RCTs which are clearly labelled as a pilot or feasibility study or randomise animals or cells instead of individuals will be excluded.
Details of journal manuscript submission and peer review processes, including, consent and potential confidentiality issues will be discussed in detail with each journal by teleconference and/or face to face prior to the journal agreeing to take part to ensure that randomisation of manuscripts is feasible. We considered conducting randomisation at the level of the journal (i.e. cluster RCTs). However, in order to make the intervention as easy and simple to implement (and with little or no additional effort from the journal) we believe that randomisation at the manuscript level -with an external researcher implementing the intervention within the existing journal management systems -will be the most efficient study design.
In participating journals, the external investigator (BS) will have access to the editorial management software (e.g. ScholarOne or Editorial Manager) and will check at least twice a week (using automated report lists) all research manuscripts that are sent out for external peer review. As soon as the first peer-reviewer accepts the invitation to review, the manuscript will be randomised to the intervention or control arm (see "Randomisation" for more details). It is possible that this process might be slightly different amongst different included journals.

Interventions
Experimental group: C-short plus usual practice After accepting to review an article, peer reviewers will receive the automated, journal specific standard email with general information as per each journal's usual practice (e.g. where to access the manuscript, date when the peer review report is due). In addition, peer-reviewers who received a manuscript which was randomised to C-short will receive an additional email including a short version of the CONSORT checklist (C-short) (either within the email or a as an attachment; based on the preferences and possibilities of the journal) focusing on the 10 most important and most poorly reported items (Table 1; as previously defined by a group of experts of the CONSORT Group (38)). Peer-reviewers will be asked to pay particular attention to items in the C-short checklist and request authors to report on these items, if not already adequately reported. This second email, containing the C-short checklist, is not generated automatically within the existing journal editorial management system (e.g. ScholarOne or Editorial Manager); it will be sent by the investigator who has access to the journal editorial system (BS). An example of this additional email is presented within the appendix (appendix 1; the exact wording might be changed according to the preferences of the participating journals). At least twice a week the editorial management system will be checked for each journal and if a peer reviewer has accepted an invitation to review, an email containing the Cshort intervention will be generated and sent. It might be possible that some journals will only provide the right to access and read manuscripts in the editorial management system, but not to send emails. If this is the case, the corresponding Editor (or designated person within the journal) will be informed to send the emails.
Development and testing of the short explanation of the C-short items: We chose the 10 most important and poorly reported CONSORT items as identified by a group of CONSORT experts in a previous study conducted by Hopewell and colleagues (38). The selection of the items was based on expert opinion and empirical evidence whenever available (38). In addition, we have added a short explanation for each of the 10 items. These short explanations were extracted and amended from the CONSORT explanation and elaboration paper (21) and from COBWEB which is online writing aid tool (42). The short explanation was discussed and adapted by the scientific committee.
Control group: Usual practice: After accepting to review an article, peer reviewers will receive the automated, journal specific standard email with general information as per each journal's usual practice (e.g. where to access the manuscript, date until when the peer review report is due). However, they will not receive the second email, sent by the investigator who has access to the journal editorial system (BS) which contains the C-short checklist.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60 F o r p e e r r e v i e w o n l y 12 Table 1: The ten most important and poorly reported CONSORT items as defined by a group of experts on the CONSORT statement (38). For better understanding key features were summarised within a short explanation (extracted from the CONSORT explanation and elaboration paper (21) as well as from the COBWEB tool (42)).

Item Section
CONSORT item Short explanation 1 Outcomes (6a) Completely defined pre-specified primary outcome measure, including how and when they were assessed Is it clear (1) what the primary outcome is (usually the one used in the sample size calculation), (2)  Method used to generate random allocation sequence Does the description make it clear if the "assigned intervention is determined by a chance process and cannot be predicted"? 4 Allocation concealment (9) Mechanism used to implement random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Is it clear how the care provider enrolling participants was made ignorant of the next assignment in the sequence (different from blinding)? Possible methods can rely on centralised or "third-party" assignment (i.e., use of a central telephone randomisation system, automated assignment system, sealed containers). 5 Blinding (11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) Is it clear if (1) healthcare providers, (2) patients, and (3) outcome assessors are blinded to the intervention? General terms such as "double-blind" without further specifications should be avoided. 6 Outcomes and estimation (17a/b) For the primary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence intervals) Is the estimated effect size and its precision (such as standard deviation or 95% confidence intervals) for each treatment arm reported? When the primary outcome is binary, both the relative effect (risk ratio, relative risk) or odds ratio) and the absolute effect (risk difference) should be reported with confidence intervals. 7 Harms (19) All-important harms or unintended effects in each group Is the number of affected persons in each group, the severity grade (if relevant) and the absolute risk (e.g. frequency of incidence) reported? Are the number of serious, life threatening events and deaths reported? If no adverse event occurred this should be clearly stated. 8 Registration (23) Registration number and name of trial registry Is the registry and the registration number reported? If the trial was not registered, it should be explained why. 9 Protocol (24) Where trial protocol can be accessed Is it stated where the trial protocol can be assessed (e.g. published, supplementary file, repository, directly from author, confidential and therefore not available)? 10 Funding (25) Sources of funding and other support (such as supply of drugs) and role of funders Are (1) the funding sources, and (2) the role of the funder(s) described?

Outcomes
Primary outcome: The primary outcome of this study will be the difference of the mean proportion of adequately reported items of the 10 most important and poorly reported CONSORT items between the two intervention arms.
Secondary outcomes: Secondary outcomes will include the following:  Mean proportion of adequate reporting of the 10 most important and poorly reported CONSORT items, considering each sub-item (see also "Assessment of outcomes") as a separate item.
 Mean proportion for each of the 10 most important and poorly reported CONSORT items separately (including also separate analysis of sub-items).
 Time from assigning an academic editor until the first decision (as communicated to the author after the first round of peer-review).
 Proportion of articles directly rejected after the first round of peer-review.
 Proportion of articles published.
Additional outcomes: For journals where peer reviewer comments are subsequently published alongside the published article, we will examine the peer reviewer comments for any reference to CONSORT and trial reporting. We will contact those journals which do not make peer reviewer comments publicly available, to see if they still could be used for such an analyses under the condition that only anonymised data will be published.

Data collection methods:
The outcomes will be assessed independently by two (blinded or at least partially blinded; see "blinding") outcome assessors with expertise in the design and reporting of clinical trials. Any disagreement will be resolved by consensus or if necessary by consulting a third assessor. To ensure consistency between reviewers, we will first pilot the data extraction form; any disparities in the interpretation will be discussed and the data extraction form will be modified accordingly.
Adequate reporting of items will be assessed from published full-text publications adhering to the CONSORT C-short checklist (21). The following included items have sub-items which will be extracted separately:  All items will be judged as either "yes" meaning adequately reported, "no" meaning not adequately reported, or "NA" meaning that this sub-item is not applicable for this RCT. Items with different sub-items will only be judged as adequately reported if all relevant sub-items were adequately reported.
 Time from assigning an academic editor until the first decision: The day when the academic editor was assigned and the day of the first decision (e.g. major revision, minor revision, rejected) will be extracted to calculate the number of days until the first decision.
 Proportion of articles directly rejected after the first round of peer-review: Articles which were not invited for re-submission will be labelled and counted.
 Proportion of articles published: Articles which will be published will be counted and collected for data extraction.
The outcomes "time from assigning an academic editor until the first decision", "proportion of articles directly rejected after the first round of peer-review", and "proportion of articles published" will be extracted directly from editorial management software of the journal.

Participant timeline
The overview of the study schedule, including enrolment, intervention and assessments is presented in Table 2.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59   For the sample size calculation we hypothesise in a first scenario ( Table 3) that the intervention C-Short will result in a 25% relative increase in adequate reporting compared to the control (meaning that 70% of items will be adequately reported in the intervention group and 56% in the control group). This is based on the rate of reporting of the 10 most important and poorly reported items was 0.56 (meaning that a mean of 56% of the 10 most important and poorly reported items were reported) in the control group of a previous study called WebCONSORT (38). The standard deviation (SD) in the same study was 0.23. However, we calculated our sample size to account for a slightly bigger variability in our data (SD = 0.25).To demonstrate a significant difference with a power of 90% and a type 1 error at 5% a total of 136 articles will be required in this scenario (68 per treatment arm; based on a two sided ttest).
The staff from one journal which will most likely be included (i.e. PLoS One) pointed out that 3 out of the 10 assessed items (i.e. item "Registration", "Protocol", and "Funding") should always be implemented given their template. Assuming that this journal will recruit a high proportion, and that also other journals might update their templates, we increased the sample size in a second scenario, in which all these 3 items would have an overall adherence of 90% in the control arm (Table 3). This would entail an overall baseline adherence with the 10 CONSORT-short items of 71%. Based on a two sided t-test, a sample size of 166 (83 per treatment arm) will have a power of 80% to find a 15% relative increase (71% adherence in control group; 82% adherence in intervention group; SD = 0.25; a type 1 error at 5%).
Since the final sample size will be based on the number of articles published, rather than on the number of manuscripts randomised, eligible RCTs will be included and randomised until the number of 83 published RCTs is reached in each arm (resulting in no less than 166 articles), to avoid loss of power due to potential imbalance between arms. Recruitment will be stopped as soon as both arms reach the sample size of 83. After recruitment stop we will wait three month so that manuscripts which are still in production can be published. Manuscripts which are published after the three month period will be excluded.

Randomisation and blinding
Articles, which meet the eligibility criteria as a primary report of an RCT, for which the journal decides to send out for external peer review will be randomised into one of the two groups (allocation 1:1). The randomisation list will be created by the study-randomizer system (43) using random block sizes between 2 and 8 and stratification by journal. As soon as the first peer-reviewer accepts the invitation, the manuscript will be included and randomised to one of the two intervention arms. One of the investigators (BS) will log onto the study randomizersystem (43) entering the study identification number (ID; provided from the Journal), the study title, as well as the journal the study was submitted to. Subsequently, all additional peerreviewers accepting the invitation to review the same manuscript will receive the same intervention (C-short plus usual practice or usual practice only) as the first peer-reviewer.
Authors will be blinded to the intervention allocation. Editors will not be actively informed about the randomisation (possible exception listed under "4.3 Interventions"). To avoid potential bias, peer reviewers and manuscript authors will not be informed of the study hypothesis, design and intervention.
Outcomes will be assessed in duplicate (see assessment of outcomes). At least one outcome assessors will be blinded. Due to restricted resources it might be possible that the investigator conducting the randomisation (BS) will be included in the data-extraction from published manuscripts. Outcomes from publications will be assessed and extracted in duplicate. Since this information is not confidential, we will use Google Forms for data extraction from published RCTs. Data entered will be validated for completeness.

Data management and confidentiality
Data from the editorial manager software (e.g. Title of manuscript, first author, randomisation ID, Journal, date when manuscript was accepted by and academic editor, date when the final decision was made, final decision, number of peer-reviewers who peer reviewed the manuscript, the peer review) will be extracted, anonymised and entered in a password protected database which is saved on a server from the University of Oxford. Data will be managed and curated according to University of Oxford regulations, which includes regular back-up (on a daily basis) of the virtual drives where the data are stored.
The raw data extracted from the included manuscripts can be made openly accessible in an anonymised way (i.e. giving the included RCT a number instead of identifying them).
Derived/aggregated data, including anonymised information generated from the journals' editorial manager software, will be stored and made available to the research community when the project ends (see also "8. Publication policy and access to data"). Where appropriate, the researcher who has access to the editorial manager software (BS) and anyone else who will see the identifiable data will sign a confidentially agreement with the participating journals, confirming that they will not share identifiable data with any other party. All quantitative variables will be described using means and standard deviations, or median and interquartile ranges in case severe departures from a normal distribution are identified.
Data distribution will be inspected visually (i.e. by histograms) instead of performing formal statistical tests for normality. Categorical variables will be described using frequencies and percentages. For the primary and secondary outcomes, we will estimate the difference between means between the two groups and report them with respective 95% confidence intervals.

Analysis of primary endpoint
The primary outcome will be the difference of the mean proportion of adequately reported items of the 10 most important and poorly reported CONSORT items. If the data on the primary outcome is normally distributed then the two groups (i.e. C-short plus usual practice vs. usual practice) will be compared using an unpaired Student's t-test to compare the unadjusted mean proportion of adequate reporting. If the data is not normally distributed, comparisons will be performed using a non-parametric equivalent test (i.e. Wilcoxon-Mann-Whitney test for testing whether the population medians of the two groups are the same).
For the analyses of the primary outcomes a p-value of 0.05 (5% significance level) will be used to indicate statistical significance and treatment effect (mean difference) reported with 95% confidence intervals (or median and respective interquartile ranges, in case of asymmetric distribution). Exact p-values will be presented up to three decimal places. We anticipate there will be no missing data in this study, neither at the individual C-short items, nor at the manuscript level. This is due to the study design, which will include only the randomised manuscripts that are accepted for publication.

Analysis of secondary endpoints
To investigate the effect of the intervention on the secondary outcomes, mean differences with respective 95% confidence intervals will also be reported for these outcomes. If normality is not observed for any of the continuous secondary outcomes, the same strategy adopted for the primary outcome (use of a non-parametric equivalent to the Student's t-test) will be used.
A p-value of 0.05 will indicate statistical significance for the observed treatment effect on the secondary outcomes. Exact p-values will be presented up to three decimal places. Similarly to the primary outcome, we anticipate there will be no missing data for any of the secondary

Pre-specified subgroup analysis
No formal subgroup comparative analysis is planned for the primary or secondary outcomes.
However, the effect of the intervention on the primary outcome within subgroups, will be presented using forest plots to visually examine whether it differs according to some variables, such as: (1) Journals that actively implement the CONSORT Statement (defined as requiring authors to submit a completed CONSORT checklist alongside their manuscript) vs. journals that are not actively implementing the CONSORT Statement; (2) sample size (n < 100 vs. n ≥ 100); and (3) impact factor (<5, 5.1-10; >10) as there is evidence that higher impact factor as well as higher sample size are associated with higher adherence to reporting guidelines (44).
These analyses will be exploratory, with the aim of supporting new hypothesis generation, rather than conclusive. The scientific committee is in charge of:  Participating in the elaboration of the protocol  Defining and validating the additional short explanation for each CONSORT item.
 Following the evolution of the committed study  Publishing the results of this study

Regulatory aspects
Ethical approval for this study will be sought from the Central University Research Ethics Committee (CUREC) of the University of Oxford. Any amendments in the conduct of the study, collection of outcomes or analysis will be reported to the CUREC. The tested intervention has the goal to improve the quality of published journals (i.e. the adherence to CONSORT) and could also be implemented as usual practice without testing at the journal level. In agreement with another study, testing a similar intervention (45), we think that it is ethical to conduct this study without obtaining written consent. The main reason for this procedure are the following:  Informing the authors and peer-reviewers would make it impossible to measure the effect of our intervention. In short, informing peer-reviewers and authors would create an artificial context which would not be comparable any more to the "real world context". Authors and peer-reviewers would most likely be much more aware of CONSORT if they received information about the study. Furthermore, being aware to participate in a study could strongly influence the natural behaviour of peer-reviewers (e.g. putting more effort into reviewing a manuscript than under "real world conditions") but also of authors.
 The intervention does not pose any risk of harms for authors and peer-reviewers.
 The intervention is not a medical intervention but rather tries to improve the research quality and journal processes.
 Several journal series (e.g. BMJ series) have Company Privacy Statements in place which clearly mention that research programmes might be in place for quality improvement.  The intervention could be part of the routine at any Journal without previous assessment of its efficacy.
 No data which identifies participating manuscripts will be published.

Publication policy and access to data
The results from this study will be published in a peer-reviewed journal irrespective of the study results. Authorship to publications will be granted according to the rules of the International Committee of Medical Journal Editors (ICMJE). We plan to publish the full anonymised dataset as a supplementary file together with the main publication.

Appendix
Appendix 1: Example of the email which will be sent out in the intervention arm (C-Short). The exact wording might be slightly adapted according to the journal preferences.
Dear *Title, Name*, We thank you for accepting to peer-review a manuscript for *journal name*. As we are trying to improve the reporting for randomised controlled trials according to the CONSORT guidelines, we would like to ask if you could check whether the following most important and poorly reported items are adequately implemented as indicated in the

Primary Registry and Trial Identifying Number
This trial was denied registration on ClinicalTrials.gov as the study is not a clinical study that assesses a health outcome in human subjects. Instead we registered the trial on the Open Science Framework (https://osf.io/c4hn8).

Secondary Identifying Numbers
Not applicable

Source(s) of Monetary or Material Support
No specific funding was acquired for this study. Benjamin Speich is supported by an Michael M Schlussel is funded by Cancer Research UK. The funders had no role in designing the study and will also have no role in conducting the study, or analysing and reporting study results.

Primary Sponsor
Sponsor:

Public Title
Impact of checklists to improve the reporting of randomised controlled trials published in biomedical journals

Scientific Title
Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: a randomised controlled trial Running title: CONSORT for Peer Review (CONSORT-PR) Study identifier: CONSORT-PR

Countries of Recruitment
Multinational (Centres are Biomedical journals)

Health Condition(s) or Problem(s) Studied
Reporting in published randomised controlled trials 13. After accepting to review a manuscript, peer reviewers will receive the automated, journal specific standard email with general information as per each journal's usual practice (e.g. where to access the manuscript, date the peer review report is due).

Control group: Usual practice
Intervention group: C-short plus usual practice After accepting to review a manuscript, peer reviewers will receive the automated, journal specific standard email with general information (identical to control group). In addition, peer reviewers will receive an additional email from the editorial office that includes a short version of the CONSORT checklist (C-short) together with a brief explanation of the items either as a table within the email or as an attachment. Peer reviewers will be asked to check whether the items in the C-short checklist are addressed in the manuscript and to request authors to include these items if they are not adequately reported.

Key Inclusion and Exclusion Criteria
The population will be defined on two levels: included journals and included manuscripts.
Inclusion criteria for journals: Included journals must: i) endorse the CONSORT Statement by mentioning it in the journals' Instruction to Authors; ii) have published primary results of at least five RCTs in 2017 (identified using a PubMed search).

Inclusion criteria for manuscripts •
All new manuscript submissions reporting the primary results of RCTs, which the journal editor has decided to send out for external peer review. Since the 10 chosen CONSORT checklist items (C-short) are applicable to different study designs, we will include all manuscripts reporting the primary results of RCTs regardless of study design (e.g. parallel group trial, cluster trial, superiority trial, non-inferiority/equivalence trials).
Exclusion criteria for manuscripts • Manuscripts clearly presenting secondary trial results, additional time points, economic analyses, or any other analyses.

•
Manuscripts which are clearly labelled as a pilot or feasibility study or animal studies.
• Manuscripts not sent for peer review.

Study Type
This study is a multicentre RCT with submitted manuscripts as the unit of randomisation (allocation ratio 1:1).

17.
Sample Size 166 Since the final sample size will be based on the number of articles published, rather than on the number of manuscripts randomised, eligible manuscripts will be randomised until 83 articles are published in each arm (resulting in no less than 166 articles), to avoid loss of power due to potential imbalance between arms.

Primary Outcome(s)
 The primary outcome of this study will be the difference in the mean proportion of adequately reported C-short items in published articles between the two groups.

Key Secondary Outcomes
• Mean proportion of adequately reported C-short items in published articles considering each item separately.
• Difference in mean proportion of adequately reported C-short items in published articles considering each sub-item (see "Assessment of outcomes") as a separate item.

•
Time from assigning an editor to the first decision (as communicated to the author after the first round of peer-review).

•
Proportion of manuscripts rejected after the first round of peer review.
• Proportion of manuscripts that will be published in the journal under study.

21.
Ethics Review Ethical approval has been obtained from the Medical Sciences Interdivisional Research Ethics Committee of the University of Oxford (R62779/RE001).

Completion date
We expect that recruitment will be finished in summer 2021.

IPD sharing statement
We plan to make the anonymised dataset, including the data from the published articles, available as a supplementary file of the main publication.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59    Role of study sponsor and funders, if any, in study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication, including whether they will have ultimate authority over any of these activities 2_______ 5d Composition, roles, and responsibilities of the coordinating centre, steering committee, endpoint adjudication committee, data management team, and other individuals or groups overseeing the trial, if applicable (see Item 21a for data monitoring committee)

Introduction
Background and rationale Interventions for each group with sufficient detail to allow replication, including how and when they will be administered 9-10, Table 1, Appendix_____   Table  Table 2________ Sample size 14 Estimated number of participants needed to achieve study objectives and how it was determined, including clinical and statistical assumptions supporting any sample size calculations 13-14__________

Recruitment 15
Strategies for achieving adequate participant enrolment to reach target sample size 7-8, 13-14___

Methods: Assignment of interventions (for controlled trials)
Allocation: Sequence generation 16a Method of generating the allocation sequence (eg, computer-generated random numbers), and list of any factors for stratification. To reduce predictability of a random sequence, details of any planned restriction (eg, blocking) should be provided in a separate document that is unavailable to those who enrol participants or assign interventions 14________ Allocation concealment mechanism 16b Mechanism of implementing the allocation sequence (eg, central telephone; sequentially numbered, opaque, sealed envelopes), describing any steps to conceal the sequence until interventions are assigned 14_________ Implementation 16c Who will generate the allocation sequence, who will enrol participants, and who will assign participants to interventions 14_________ Definition of analysis population relating to protocol non-adherence (eg, as randomised analysis), and any statistical methods to handle missing data (eg, multiple imputation) 15 (no missing data expected)___

Methods: Monitoring
Data monitoring 21a Composition of data monitoring committee (DMC); summary of its role and reporting structure; statement of whether it is independent from the sponsor and competing interests; and reference to where further details about its charter can be found, if not in the protocol. Alternatively, an explanation of why a DMC is not needed 17_________ 21b Description of any interim analyses and stopping guidelines, including who will have access to these interim results and make the final decision to terminate the trial 13___________    22 This lack of transparency is a major limiting factor for readers 113 who assess an article in order to find the answer to a specific question; it is also a major 114 problem for scientists who perform systematic reviews and meta-analyses. . Therefore, it is unlikely that these interventions will be implemented in the majority of 134 journals, especially smaller journals with limited resources. Another study found that providing 135 authors with a web-based CONSORT tool, which combined different CONSORT extensions 136 and provided authors with a customised checklist, did not improve reporting when used at the 137 manuscript revision stage. 30 However, a study examining "the nature and extent of changes made to manuscripts after peer review, in relation to the reporting of methodological aspects of RCTs" and "the type of changes requested by peer reviewers" found that peer review did 140 lead to some improvement in reporting. 27

142
The role of peer reviewers and expectations of them is varied. 31  The population will be defined on two levels: included journals and included manuscripts.

189
 Manuscripts which are clearly labelled as a pilot or feasibility study or animal studies. After accepting to review a manuscript, peer reviewers will receive the automated, journal 209 specific standard email with general information as per each journal's usual practice (e.g.

210
where to access the manuscript, date the peer review report is due).

212
Intervention group: C-short plus usual practice

213
After accepting to review a manuscript, peer reviewers will receive the automated, journal 214 specific standard email with general information (identical to control group). In addition, peer 215 reviewers will receive an additional email from the editorial office that includes a short version 216 of the CONSORT checklist (C-short) together with a brief explanation of the items either as a   by Hopewell and colleagues. 30 The selection of the items was based on expert opinion and 231 empirical evidence whenever available. 30 In addition, to enable peer reviewers to better 232 understand the items, we added a short explanation for each of the 10 items. These short 233 explanations were extracted and amended from the CONSORT explanation and elaboration 234 paper 11 and from COBWEB which is an online writing aid tool. 33 The short explanation was 235 discussed and adapted by the scientific committee.

238
Primary outcome

239
The primary outcome of this study will be the difference in the mean proportion of adequately 240 reported C-short items in published articles between the two groups.

242
Secondary outcomes 243 Secondary outcomes will include the following:

246
 Difference in mean proportion of adequately reported C-short items in published 247 articles considering each sub-item (see "Assessment of outcomes") as a separate item.

248
 Time from assigning an editor to the first decision (as communicated to the author after 249 the first round of peer-review).

250
 Proportion of manuscripts rejected after the first round of peer review.

251
 Proportion of manuscripts that will be published in the journal under study.

254
 Exploratory analysis of available peer reviewer comments (i.e. any references to 255 CONSORT).

256
For journals where peer reviewers' comments are subsequently published alongside the 257 published article, we will examine the peer reviewers' comments for any reference to

258
CONSORT and trial reporting. We will contact those journals which do not make peer 259 reviewers' comments publicly available, to see if reviews could be provided for such analyses 260 under the condition that only anonymised data will be published.

262
Assessment of outcomes:

263
The outcomes will be assessed independently by two (blinded or at least partially blinded; see 264 "blinding") outcome assessors with expertise in the design and reporting of clinical trials. Any 265 disagreement will be resolved by consensus or if necessary by consulting a third assessor. To 266 ensure consistency between reviewers, we will first pilot the data extraction form; any 267 disparities in the interpretation will be discussed and the data extraction form will be modified 268 accordingly. Adequate reporting of items will be assessed in duplicate from published full-text publications 271 following the same instructions as provided by the CONSORT C-short checklist. 11 The

272
following checklist items have, due to their complexity, sub-items which will be extracted 273 separately. The sub-items are highlighted in the short explanation of the intervention (see 274  Table 1

284
All items will be judged as either "yes" meaning adequately reported, "no" meaning not 285 adequately reported or not reported at all, or "NA" meaning that this sub-item is not applicable 286 for this RCT. Items with different sub-items will only be judged as adequately reported if all 287 relevant sub-items were adequately reported.

289
The outcomes "time from assigning an editor to the first decision", "proportion of manuscripts 290 rejected after the first round of peer-review", and "proportion of manuscripts that will be 291 published in the journal under study" will be extracted directly from the journal's editorial system 292 or provided by the journal.

Participant timeline
295 The overview of the study schedule, including enrolment, intervention and assessments is 296 presented in Table 2. For the sample size calculation, we hypothesised in a first scenario ( Table 3) that the 300 intervention C-short will result in a 25% relative increase in adequate reporting compared to 301 the control (meaning that 70% of items will be adequately reported in the intervention group 302 and 56% in the control group). This is based on a proportion of adequate reporting of 0.56 for 303 the 10 most important and poorly reported items found in the control group of a previous study 304 (meaning that a mean of 56% of the 10 most important and poorly reported items were 305 reported). 30 The standard deviation (SD) in the same study was 0.23. However, we calculated 306 our sample size to account for a slightly larger variability in our data (SD = 0.25). To 307 demonstrate a significant difference with a power of 90% and a type 1 error at 5%, a total of 308 136 published articles will be required in this scenario (68 per treatment arm; based on a two 309 sided t-test).

311
Two authors of this protocol, working for PLOS ONE (IP and AC), one of the participating 312 journals, pointed out that 3 out of the 10 assessed items (i.e. item "Registration", "Protocol",

313
and "Funding") should always be implemented in submissions to their journal given their policy 314 requirements for clinical trials. Assuming that this journal will recruit a high proportion of 315 manuscripts, and that also other journals might update their templates, we increased the 316 sample size in a second scenario, in which all these 3 items would have an overall adherence 317 of 90% in the control arm (Table 3). This would entail an overall baseline adherence with the 318 10 C-short items of 71%. Based on a two sided t-test, a sample size of 166 (83 per treatment 319 arm) will have a power of 80% to find a 15% relative increase (71% adherence in control group; 320 82% adherence in intervention group; SD = 0.25; a type 1 error at 5%).

321
Since the final sample size will be based on the number of articles published, rather than on 322 the number of manuscripts randomised, eligible manuscripts will be randomised until 83 323 articles are published in each arm (resulting in no less than 166 articles), to avoid loss of power 324 due to potential imbalance between arms. Recruitment will be stopped as soon as both arms Manuscripts meeting the eligibility criteria and sent out for external peer review by the journals 331 will be randomised into one of the two groups (allocation 1:1). The randomisation list will be 332 created by the Study-Randomizer © system 34 using random block sizes between 2 and 8 and 333 stratified by journal. As soon as the first peer reviewer accepts the invitation, the manuscript 334 will be included and randomised to one of the two study arms. One of the investigators (BS)

335
will log onto the Study-Randomizer © system 34 and enter the study identification number (ID; 336 provided by the journal), the study title, and the journal the study was submitted to.

337
Subsequently, all additional peer reviewers accepting the invitation to review the same 338 manuscript will receive the same group assignment as the first peer reviewer.

340
Authors will be blinded to the intervention. Editors will not be actively informed about the 341 randomisation (possible exception listed under "Interventions"). To avoid potential bias, peer 342 reviewers and manuscript authors will not be informed of the study hypothesis, design and 343 intervention.

345
Outcomes will be assessed in duplicate (see "Assessment of outcomes"). At least one outcome 346 assessor will be blinded. Due to restricted resources the investigator conducting the 347 randomisation (BS) might be involved in the data-extraction from published manuscripts. All quantitative variables will be described using means and standard deviations, or medians 351 and interquartile ranges in case severe departures from a normal distribution are identified.

352
Data distributions will be inspected visually (i.e. by histograms) instead of performing formal 353 statistical tests for normality. Categorical variables will be described using frequencies and 354 percentages. For the primary and secondary outcomes, we will estimate the mean difference 355 between the two groups and report them with respective 95% confidence intervals. No interim 356 analysis will be conducted.

358
Populations of analysis

359
The main population for analysis will be all manuscripts randomised and accepted for 360 publication in the participating journals. In contrast to RCTs conducted with patients, where 361 losses to follow-up need to be carefully considered (e.g. multiple imputation of missing data),

362
we are only interested in the reporting adherence of RCTs that are published. As such, we will 363 exclude randomised manuscripts that were not published from the main analysis. All outcomes 364 will be calculated based on the main population. The secondary outcome "Time to the first 365 decision", will additionally be calculated considering all randomised manuscripts (including the 366 ones which were not published). For all analyses a p-value of 0.05 (5% significance level) will 367 be used to indicate statistical significance. Exact p-values will be presented up to three decimal 368 places. We anticipate there will be no missing data in this study, neither at the individual C-369 short items, nor at the manuscript level. This is due to the study design, which will include only 370 the randomised manuscripts that are accepted for publication. We will analyse if the rate of 371 manuscripts rejected after the first round of peer-review and if the proportion of manuscripts 372 that will be published differentiate amongst the two study arms (both secondary results). The effect of the intervention will be estimated as the mean difference in the proportion of C-376 short items adequately reported between the study arms. If the data on the primary outcome 377 is normally distributed, groups will be compared using an unpaired Student's t-test. If the data 378 is not normally distributed, comparisons will be performed using a non-parametric equivalent 379 test (i.e. Wilcoxon-Mann-Whitney test).

380
Analysis of secondary endpoints

381
To investigate the effect of the intervention on the secondary outcomes, mean differences with 382 respective 95% confidence intervals will be reported. If normality is not observed for any of the 383 continuous secondary outcomes, the same strategy adopted for the primary outcome (use of 384 a non-parametric equivalent to the Student's t-test) will be used.

386
Pre-specified subgroup analysis

387
No formal subgroup comparative analysis is planned for the primary or secondary outcomes.

388
However, the effect of the intervention on the primary outcome within subgroups will be 389 presented using forest plots to visually examine whether it may differ according to some evidence that higher impact factor as well as higher sample size are associated with higher 395 adherence to reporting guidelines. 35 Sub-group analysis at the journal level will only be 396 conducted when sufficient journals are in each group so that no results of individual journals 397 are revealed. All analyses will be exploratory, with the aim of supporting new hypothesis 398 generation, rather than being conclusive.

413
The raw data extracted from the included published manuscripts can be made openly 414 accessible in an anonymised way (i.e. giving the included RCT a number instead of identifying 415 them). Derived/aggregated data, including anonymised information generated from the 416 journal's editorial system, will be stored and made available to the research community when 417 the project ends (see also "Publication policy and access to data"). Where appropriate, the 418 researcher who has access to the journal's editorial system (BS) and anyone else who will see 419 the identifiable data will sign a confidentially agreement with the participating journals, 420 confirming that they will not share identifiable data with any other party. Publishers such as the  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59

483
The results from this study will be published in a peer reviewed journal irrespective of the study 484 results. Authorship of publications will be granted according to the criteria of the International

485
Committee of Medical Journal Editors (ICMJE). We plan to make the anonymised dataset,

638
understanding key features were summarised within a short explanation (extracted from the CONSORT explanation and elaboration paper 11 as well 639 as from the COBWEB tool 33 ).
Item Section CONSORT item Short explanation 1 Outcomes (6a) Completely defined pre-specified primary outcome measure, including how and when it was assessed Is it clear (1)  Method used to generate random allocation sequence Does the description make it clear if the "assigned intervention is determined by a chance process and cannot be predicted"? 4 Allocation concealment (9) Mechanism used to implement random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Is it clear how the care provider enrolling participants was made ignorant of the next assignment in the sequence (different from blinding)? Possible methods can rely on centralised or "third-party" assignment (i.e., use of a central telephone randomisation system, automated assignment system, sealed containers). 5 Blinding (11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) Is it clear if (1) healthcare providers, (2) patients, and (3) outcome assessors are blinded to the intervention? General terms such as "double-blind" without further specifications should be avoided. 6 Outcomes and estimation (17a/b) For the primary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence intervals) Is the estimated effect size and its precision (such as standard deviation or 95% confidence intervals) for each treatment arm reported? When the primary outcome is binary, both the relative effect (risk ratio, relative risk) or odds ratio) and the absolute effect (risk difference) should be reported with confidence intervals. 7 Harms (19) All-important harms or unintended effects in each group Is the number of affected persons in each group, the severity grade (if relevant) and the absolute risk (e.g. frequency of incidence) reported? Are the number of serious, life threatening events and deaths reported? If no adverse event occurred this should be clearly stated. 8 Registration (23) Registration number and name of trial registry Is the registry and the registration number reported? If the trial was not registered, it should be explained why. 9 Protocol (24) Where trial protocol can be accessed Is it stated where the trial protocol can be assessed (e.g. published, supplementary file, repository, directly from author, confidential and therefore not available)? 10 Funding ( 1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45 1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59 1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49   Example of the email which will be sent out in the intervention arm (C-Short). The exact wording might be slightly adapted according to the journal preferences.

Need for clinical research and epidemiologic transparency
There is substantial agreement that well conducted and reported randomised controlled trials (RCTs) generate the most trustworthy evidence when newly developed or already existing clinical interventions are evaluated (1-3). Besides the complexity and the high associated costs of conducting RCTs (4-6), there are major issues with their reporting that often make it difficult for researchers, clinicians, patients or policymakers to interpret the current evidence on a specific topic (7,8). Chronologically, the most prominent difficulties in reporting consist of (i) poor reporting in study protocols for RCTs (9)(10)(11)(12); (ii) a substantial fraction of trials are not registered, prematurely discontinued (most common due to difficulties with recruitment) and not published (13,14); and (iii) that published RCTs are often poorly reported (7). is of high quality. However, reporting all items from the CONSORT list will enable readers to adequately judge the quality of RCTs.  (3,(25)(26)(27)(28)(29)(30). Despite some improvement in reporting following the implementation of the CONSORT Statement, there still remain major reporting deficiencies in published RCTs (31).

For example, Odutayo and colleagues showed that a large proportion of RCTs published in
December 2012 in PubMed did not define the primary outcome (31%), did not state the sample size calculation (45%) and did not explain the method of allocation concealment (50%) (32).
This lack of transparency is a major limiting factor for the reader who assesses an article in order to find the answer to a specific question; it is also a major problem for scientists who perform systematic reviews and meta-analyses. Thus, some published trials may not be included in the meta-analysis because of their lack of transparency. Chan showed (25, 33) that 50% of efficacy outcomes and 65% of safety outcomes could not be included in meta-analyses because of how they were reported. Furthermore, even if these trials are included in systematic reviews and meta-analyses, an adequate risk of bias assessment is often not possible due to the poor reporting quality. Nevertheless, the main consequence of the lack of transparency is the risk of accepting treatments that are ineffective or cause serious adverse events (34). In a study published in 2016 authors of RCTs were asked by journal editors to use the webbased CONSORT tool at the manuscript revision stage (38). Authors who were randomly allocated to the intervention had access to a tool which allowed them to combine different CONSORT extensions (according to study design, medical field) to generate customised checklists. In the control group, authors had access to a CONSORT flow diagram generator.
The goal was to improve the reporting of CONSORT items with a simple webtool. However, a quarter of all authors either wrongly selected a CONSORT extension or failed to select an extension, indicating that further education is needed in terms of when and how to implement CONSORT extensions. inviting an additional statistical peer-reviewer (40,41)). Therefore, it is unlikely that these interventions will be implemented in the vast majority of journals, especially not in smaller journals with limited resources. A study examining "the nature and extent of changes made to manuscripts after peer review, in relation to the reporting of methodological aspects of RCTs" and "the type of changes requested by peer reviewers" found that peer review did lead to some improvement in reporting (40).
Building on these findings we plan to evaluate the impact of inviting peer reviewers to explicitly use a short version of the CONSORT checklist (including a short explanation of those items) as part of their review process. If this intervention deems to be effective, it could be easily implemented by all medical journals without needing additional resources at a journal level.

Hypothesis
We propose an RCT to evaluate the impact of asking peer reviewers to use a short version of the CONSORT checklist when reviewing a manuscript of an RCT and whether it improves the completeness of reporting. Our hypothesis is that reminding peer reviewers of the CONSORT items (including a short explanation of those items) will result in higher adherence to CONSORT guidelines in published RCTs. We only selected a limited number of the CONSORT items because we did not want to deter peer reviewers with too much information. Since peer reviewing in general can be burdensome, we felt that this approach is more promising than listing all items, risking that the information will be ignored. The short version of the CONSORT checklist is based on the same items described in a previous study as the 10 most important and underreported CONSORT items (38).

Main objective
The main objective of this study is to evaluate the impact of asking peer reviewers during the standard peer-review process to ask them to use a short version of the CONSORT checklist (C-short) and whether it will improve the reporting in published RCTs compared to manuscripts where the peer reviewers underwent usual practice.

Trial design
This study is a multicentre RCT with articles being the unit of randomisation (Figure 1; allocation ratio 1:1). A multicentre parallel arm RCT with randomisation at the individual article level was chosen instead of a cluster RCT because the risk of any "contamination" on journal level is not given as the intervention will be implemented by an external researcher (i.e. BS).
The possibility of contamination due to the possibility that peer reviewer are invited to assess several RCTs and are randomised into both arms was judged as relatively small and therefore we do not plan to adjust for clustering by journal. The journal staff (i.e. editors) will not be actively told which manuscript was allocated to the proposed intervention and which to the control group.  of the requirements for participation and a short summary information sheet will be included as part of the email invitation sent to journal editors. If a journal is eligible, and agrees to take part, the journal will also need to provide access to their journal editorial system (e.g. ScholarOne, Editorial Manager) to enable the external researcher (i.e. BS) to screen and randomise eligible manuscripts. In cases this is not possible, we will explore with separate journals if it would be possible to grant limited access (e.g. only rights to screen studies) and that the emails from the intervention would be sent by a person from the editorial team.
We will include all submitted manuscripts reporting RCTs for which the journal decides to send out for external peer review. Since the 10 chosen CONSORT checklist items are applicable to different study designs, we will include all RCTs regardless of study design (e.g. parallel group trial, cluster trial, superiority trial, non-inferiority trial). Articles presenting clearly secondary trial results, additional time points, economic analyses, or any other analyses derived from an RCT dataset not including the study's main results will be excluded. Furthermore, RCTs which are clearly labelled as a pilot or feasibility study or randomise animals or cells instead of individuals will be excluded.
Details of journal manuscript submission and peer review processes, including, consent and potential confidentiality issues will be discussed in detail with each journal by teleconference and/or face to face prior to the journal agreeing to take part to ensure that randomisation of manuscripts is feasible. We considered conducting randomisation at the level of the journal (i.e. cluster RCTs). However, in order to make the intervention as easy and simple to implement (and with little or no additional effort from the journal) we believe that randomisation at the manuscript level -with an external researcher implementing the intervention within the existing journal management systems -will be the most efficient study design.
In participating journals, the external investigator (BS) will have access to the editorial management software (e.g. ScholarOne or Editorial Manager) and will check at least twice a week (using automated report lists) all research manuscripts that are sent out for external peer review. As soon as the first peer-reviewer accepts the invitation to review, the manuscript will be randomised to the intervention or control arm (see "Randomisation" for more details). It is possible that this process might be slightly different amongst different included journals.

Interventions
Experimental group: C-short plus usual practice After accepting to review an article, peer reviewers will receive the automated, journal specific standard email with general information as per each journal's usual practice (e.g. where to access the manuscript, date when the peer review report is due). In addition, peer-reviewers who received a manuscript which was randomised to C-short will receive an additional email including a short version of the CONSORT checklist (C-short) (either within the email or a as an attachment; based on the preferences and possibilities of the journal) focusing on the 10 most important and most poorly reported items (Table 1; as previously defined by a group of experts of the CONSORT Group (38)). Peer-reviewers will be asked to pay particular attention to items in the C-short checklist and request authors to report on these items, if not already adequately reported. This second email, containing the C-short checklist, is not generated automatically within the existing journal editorial management system (e.g. ScholarOne or Editorial Manager); it will be sent by the investigator who has access to the journal editorial system (BS). An example of this additional email is presented within the appendix (appendix 1; the exact wording might be changed according to the preferences of the participating journals). At least twice a week the editorial management system will be checked for each journal and if a peer reviewer has accepted an invitation to review, an email containing the Cshort intervention will be generated and sent. It might be possible that some journals will only provide the right to access and read manuscripts in the editorial management system, but not to send emails. If this is the case, the corresponding Editor (or designated person within the journal) will be informed to send the emails.
Development and testing of the short explanation of the C-short items: We chose the 10 most important and poorly reported CONSORT items as identified by a group of CONSORT experts in a previous study conducted by Hopewell and colleagues (38). The selection of the items was based on expert opinion and empirical evidence whenever available (38). In addition, we have added a short explanation for each of the 10 items. These short explanations were extracted and amended from the CONSORT explanation and elaboration paper (21) and from COBWEB which is online writing aid tool (42). The short explanation was discussed and adapted by the scientific committee.
Control group: Usual practice: After accepting to review an article, peer reviewers will receive the automated, journal specific standard email with general information as per each journal's usual practice (e.g. where to access the manuscript, date until when the peer review report is due). However, they will not receive the second email, sent by the investigator who has access to the journal editorial system (BS) which contains the C-short checklist.   Method used to generate random allocation sequence Does the description make it clear if the "assigned intervention is determined by a chance process and cannot be predicted"? 4 Allocation concealment (9) Mechanism used to implement random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Is it clear how the care provider enrolling participants was made ignorant of the next assignment in the sequence (different from blinding)? Possible methods can rely on centralised or "third-party" assignment (i.e., use of a central telephone randomisation system, automated assignment system, sealed containers). 5 Blinding (11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) Is it clear if (1) healthcare providers, (2) patients, and (3) outcome assessors are blinded to the intervention? General terms such as "double-blind" without further specifications should be avoided. 6 Outcomes and estimation (17a/b) For the primary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence intervals) Is the estimated effect size and its precision (such as standard deviation or 95% confidence intervals) for each treatment arm reported? When the primary outcome is binary, both the relative effect (risk ratio, relative risk) or odds ratio) and the absolute effect (risk difference) should be reported with confidence intervals. 7 Harms ( The primary outcome of this study will be the difference of the mean proportion of adequately reported items of the 10 most important and poorly reported CONSORT items between the two intervention arms.
Secondary outcomes: Secondary outcomes will include the following:  Mean proportion of adequate reporting of the 10 most important and poorly reported CONSORT items, considering each sub-item (see also "Assessment of outcomes") as a separate item.
 Mean proportion for each of the 10 most important and poorly reported CONSORT items separately (including also separate analysis of sub-items).
 Time from assigning an academic editor until the first decision (as communicated to the author after the first round of peer-review).
 Proportion of articles directly rejected after the first round of peer-review.
 Proportion of articles published.
Additional outcomes: For journals where peer reviewer comments are subsequently published alongside the published article, we will examine the peer reviewer comments for any reference to CONSORT and trial reporting. We will contact those journals which do not make peer reviewer comments publicly available, to see if they still could be used for such an analyses under the condition that only anonymised data will be published.

Data collection methods:
The outcomes will be assessed independently by two (blinded or at least partially blinded; see "blinding") outcome assessors with expertise in the design and reporting of clinical trials. Any disagreement will be resolved by consensus or if necessary by consulting a third assessor. To ensure consistency between reviewers, we will first pilot the data extraction form; any disparities in the interpretation will be discussed and the data extraction form will be modified accordingly.
 Time from assigning an academic editor until the first decision: The day when the academic editor was assigned and the day of the first decision (e.g. major revision, minor revision, rejected) will be extracted to calculate the number of days until the first decision.
 Proportion of articles directly rejected after the first round of peer-review: Articles which were not invited for re-submission will be labelled and counted.
 Proportion of articles published: Articles which will be published will be counted and collected for data extraction.
The outcomes "time from assigning an academic editor until the first decision", "proportion of articles directly rejected after the first round of peer-review", and "proportion of articles published" will be extracted directly from editorial management software of the journal.

Participant timeline
The overview of the study schedule, including enrolment, intervention and assessments is presented in Table 2.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59   For the sample size calculation we hypothesise in a first scenario ( Table 3) that the intervention C-Short will result in a 25% relative increase in adequate reporting compared to the control (meaning that 70% of items will be adequately reported in the intervention group and 56% in the control group). This is based on the rate of reporting of the 10 most important and poorly reported items was 0.56 (meaning that a mean of 56% of the 10 most important and poorly reported items were reported) in the control group of a previous study called WebCONSORT (38). The standard deviation (SD) in the same study was 0.23. However, we calculated our sample size to account for a slightly bigger variability in our data (SD = 0.25).To demonstrate a significant difference with a power of 90% and a type 1 error at 5% a total of 136 articles will be required in this scenario (68 per treatment arm; based on a two sided ttest).
The staff from one journal which will most likely be included (i.e. PLoS One) pointed out that 3 out of the 10 assessed items (i.e. item "Registration", "Protocol", and "Funding") should always be implemented given their template. Assuming that this journal will recruit a high proportion, and that also other journals might update their templates, we increased the sample size in a second scenario, in which all these 3 items would have an overall adherence of 90% in the control arm (Table 3). This would entail an overall baseline adherence with the 10 CONSORT-short items of 71%. Based on a two sided t-test, a sample size of 166 (83 per treatment arm) will have a power of 80% to find a 15% relative increase (71% adherence in control group; 82% adherence in intervention group; SD = 0.25; a type 1 error at 5%).
Since the final sample size will be based on the number of articles published, rather than on the number of manuscripts randomised, eligible RCTs will be included and randomised until the number of 83 published RCTs is reached in each arm (resulting in no less than 166 articles), to avoid loss of power due to potential imbalance between arms. Recruitment will be stopped as soon as both arms reach the sample size of 83. After recruitment stop we will wait three month so that manuscripts which are still in production can be published. Manuscripts which are published after the three month period will be excluded.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59

Randomisation and blinding
Articles, which meet the eligibility criteria as a primary report of an RCT, for which the journal decides to send out for external peer review will be randomised into one of the two groups (allocation 1:1). The randomisation list will be created by the study-randomizer system (43) using random block sizes between 2 and 8 and stratification by journal. As soon as the first peer-reviewer accepts the invitation, the manuscript will be included and randomised to one of the two intervention arms. One of the investigators (BS) will log onto the study randomizersystem (43) entering the study identification number (ID; provided from the Journal), the study title, as well as the journal the study was submitted to. Subsequently, all additional peerreviewers accepting the invitation to review the same manuscript will receive the same intervention (C-short plus usual practice or usual practice only) as the first peer-reviewer.
Authors will be blinded to the intervention allocation. Editors will not be actively informed about the randomisation (possible exception listed under "4.3 Interventions"). To avoid potential bias, peer reviewers and manuscript authors will not be informed of the study hypothesis, design and intervention.
Outcomes will be assessed in duplicate (see assessment of outcomes). At least one outcome assessors will be blinded. Due to restricted resources it might be possible that the investigator conducting the randomisation (BS) will be included in the data-extraction from published manuscripts.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   18 Outcomes from publications will be assessed and extracted in duplicate. Since this information is not confidential, we will use Google Forms for data extraction from published RCTs. Data entered will be validated for completeness.

Data management and confidentiality
Data from the editorial manager software (e.g. Title of manuscript, first author, randomisation ID, Journal, date when manuscript was accepted by and academic editor, date when the final decision was made, final decision, number of peer-reviewers who peer reviewed the manuscript, the peer review) will be extracted, anonymised and entered in a password protected database which is saved on a server from the University of Oxford. Data will be managed and curated according to University of Oxford regulations, which includes regular back-up (on a daily basis) of the virtual drives where the data are stored.
The raw data extracted from the included manuscripts can be made openly accessible in an anonymised way (i.e. giving the included RCT a number instead of identifying them).
Derived/aggregated data, including anonymised information generated from the journals' editorial manager software, will be stored and made available to the research community when the project ends (see also "8. Publication policy and access to data"). Where appropriate, the researcher who has access to the editorial manager software (BS) and anyone else who will see the identifiable data will sign a confidentially agreement with the participating journals, confirming that they will not share identifiable data with any other party.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  All quantitative variables will be described using means and standard deviations, or median and interquartile ranges in case severe departures from a normal distribution are identified.
Data distribution will be inspected visually (i.e. by histograms) instead of performing formal statistical tests for normality. Categorical variables will be described using frequencies and percentages. For the primary and secondary outcomes, we will estimate the difference between means between the two groups and report them with respective 95% confidence intervals.

Analysis of primary endpoint
The primary outcome will be the difference of the mean proportion of adequately reported items of the 10 most important and poorly reported CONSORT items. If the data on the primary outcome is normally distributed then the two groups (i.e. C-short plus usual practice vs. usual practice) will be compared using an unpaired Student's t-test to compare the unadjusted mean proportion of adequate reporting. If the data is not normally distributed, comparisons will be performed using a non-parametric equivalent test (i.e. Wilcoxon-Mann-Whitney test for testing whether the population medians of the two groups are the same).
For the analyses of the primary outcomes a p-value of 0.05 (5% significance level) will be used to indicate statistical significance and treatment effect (mean difference) reported with 95% confidence intervals (or median and respective interquartile ranges, in case of asymmetric distribution). Exact p-values will be presented up to three decimal places. We anticipate there will be no missing data in this study, neither at the individual C-short items, nor at the manuscript level. This is due to the study design, which will include only the randomised manuscripts that are accepted for publication.

Analysis of secondary endpoints
To investigate the effect of the intervention on the secondary outcomes, mean differences with respective 95% confidence intervals will also be reported for these outcomes. If normality is not observed for any of the continuous secondary outcomes, the same strategy adopted for the primary outcome (use of a non-parametric equivalent to the Student's t-test) will be used.
A p-value of 0.05 will indicate statistical significance for the observed treatment effect on the secondary outcomes. Exact p-values will be presented up to three decimal places. Similarly to the primary outcome, we anticipate there will be no missing data for any of the secondary  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y 20 outcomes, as we will have access to the Editorial Management system of the included journals, where all relevant information is automatically reported.

Pre-specified subgroup analysis
No formal subgroup comparative analysis is planned for the primary or secondary outcomes.
However, the effect of the intervention on the primary outcome within subgroups, will be presented using forest plots to visually examine whether it differs according to some variables, such as: (1) Journals that actively implement the CONSORT Statement (defined as requiring authors to submit a completed CONSORT checklist alongside their manuscript) vs. journals that are not actively implementing the CONSORT Statement; (2) sample size (n < 100 vs. n ≥ 100); and (3) impact factor (<5, 5.1-10; >10) as there is evidence that higher impact factor as well as higher sample size are associated with higher adherence to reporting guidelines (44).
These analyses will be exploratory, with the aim of supporting new hypothesis generation, rather than conclusive.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  could also be implemented as usual practice without testing at the journal level. In agreement with another study, testing a similar intervention (45), we think that it is ethical to conduct this study without obtaining written consent. The main reason for this procedure are the following:  Informing the authors and peer-reviewers would make it impossible to measure the effect of our intervention. In short, informing peer-reviewers and authors would create an artificial context which would not be comparable any more to the "real world context". Authors and peer-reviewers would most likely be much more aware of CONSORT if they received information about the study. Furthermore, being aware to participate in a study could strongly influence the natural behaviour of peer-reviewers (e.g. putting more effort into reviewing a manuscript than under "real world conditions") but also of authors.
 The intervention does not pose any risk of harms for authors and peer-reviewers.
 The intervention is not a medical intervention but rather tries to improve the research quality and journal processes.
 No data which identifies participating manuscripts will be published.

Public Title
Impact of checklists to improve the reporting of randomised controlled trials published in biomedical journals

Scientific Title
Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: a randomised controlled trial Running title: CONSORT for Peer Review (CONSORT-PR) Study identifier: CONSORT-PR

Countries of Recruitment
Multinational (Centres are Biomedical journals)

Control group: Usual practice
Intervention group: C-short plus usual practice After accepting to review a manuscript, peer reviewers will receive the automated, journal specific standard email with general information (identical to control group). In addition, peer reviewers will receive an additional email from the editorial office that includes a short version of the CONSORT checklist (C-short) together with a brief explanation of the items either as a table within the email or as an attachment. Peer reviewers will be asked to check whether the items in the C-short checklist are addressed in the manuscript and to request authors to include these items if they are not adequately reported.

Key Inclusion and Exclusion Criteria
The population will be defined on two levels: included journals and included manuscripts.
Inclusion criteria for journals: Included journals must: i) endorse the CONSORT Statement by mentioning it in the journals' Instruction to Authors; ii) have published primary results of at least five RCTs in 2017 (identified using a PubMed search).

Inclusion criteria for manuscripts •
All new manuscript submissions reporting the primary results of RCTs, which the journal editor has decided to send out for external peer review. Since the 10 chosen CONSORT checklist items (C-short) are applicable to different study designs, we will include all manuscripts reporting the primary results of RCTs regardless of study design (e.g. parallel group trial, cluster trial, superiority trial, non-inferiority/equivalence trials).
Exclusion criteria for manuscripts • Manuscripts clearly presenting secondary trial results, additional time points, economic analyses, or any other analyses.

•
Manuscripts which are clearly labelled as a pilot or feasibility study or animal studies.
• Manuscripts not sent for peer review.

Study Type
This study is a multicentre RCT with submitted manuscripts as the unit of randomisation (allocation ratio 1:1).

17.
Sample Size 166 Since the final sample size will be based on the number of articles published, rather than on the number of manuscripts randomised, eligible manuscripts will be randomised until 83 articles are published in each arm (resulting in no less than 166 articles), to avoid loss of power due to potential imbalance between arms.

Primary Outcome(s)
 The primary outcome of this study will be the difference in the mean proportion of adequately reported C-short items in published articles between the two groups.

Key Secondary Outcomes
• Mean proportion of adequately reported C-short items in published articles considering each item separately.
• Difference in mean proportion of adequately reported C-short items in published articles considering each sub-item (see "Assessment of outcomes") as a separate item.

•
Time from assigning an editor to the first decision (as communicated to the author after the first round of peer-review).

•
Proportion of manuscripts rejected after the first round of peer review.
• Proportion of manuscripts that will be published in the journal under study.

21.
Ethics Review Ethical approval has been obtained from the Medical Sciences Interdivisional Research Ethics Committee of the University of Oxford (R62779/RE001).

Completion date
We expect that recruitment will be finished in summer 2021.

IPD sharing statement
We plan to make the anonymised dataset, including the data from the published articles, available as a supplementary file of the main publication.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59    Role of study sponsor and funders, if any, in study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication, including whether they will have ultimate authority over any of these activities 24-25_______ 5d Composition, roles, and responsibilities of the coordinating centre, steering committee, endpoint adjudication committee, data management team, and other individuals or groups overseeing the trial, if applicable (see Item 21a for data monitoring committee)

Introduction
Background and rationale Trial design 8 Description of trial design including type of trial (eg, parallel group, crossover, factorial, single group), allocation ratio, and framework (eg, superiority, equivalence, noninferiority, exploratory)

Methods: Participants, interventions, and outcomes
Study setting 9 Description of study settings (eg, community clinic, academic hospital) and list of countries where data will be collected. Reference to where list of study sites can be obtained

7-9__________
Eligibility criteria 10 Inclusion and exclusion criteria for participants. If applicable, eligibility criteria for study centres and individuals who will perform the interventions (eg, surgeons, psychotherapists)

7-9_
Interventions 11a Interventions for each group with sufficient detail to allow replication, including how and when they will be administered 9-10,    Table  Table 2________ Sample size 14 Estimated number of participants needed to achieve study objectives and how it was determined, including clinical and statistical assumptions supporting any sample size calculations 13-14__________

Recruitment 15
Strategies for achieving adequate participant enrolment to reach target sample size 7-8, 13-14___

Methods: Assignment of interventions (for controlled trials)
Allocation: Sequence generation 16a Method of generating the allocation sequence (eg, computer-generated random numbers), and list of any factors for stratification. To reduce predictability of a random sequence, details of any planned restriction (eg, blocking) should be provided in a separate document that is unavailable to those who enrol participants or assign interventions 14________ Allocation concealment mechanism 16b Mechanism of implementing the allocation sequence (eg, central telephone; sequentially numbered, opaque, sealed envelopes), describing any steps to conceal the sequence until interventions are assigned 14_________ Implementation 16c Who will generate the allocation sequence, who will enrol participants, and who will assign participants to interventions 14_________ Blinding (masking) 17a Who will be blinded after assignment to interventions (eg, trial participants, care providers, outcome assessors, data analysts), and how 14_____________  If blinded, circumstances under which unblinding is permissible, and procedure for revealing a participant's allocated intervention during the trial NA__________

Methods: Data collection, management, and analysis
Data collection methods 18a Plans for assessment and collection of outcome, baseline, and other trial data, including any related processes to promote data quality (eg, duplicate measurements, training of assessors) and a description of study instruments (eg, questionnaires, laboratory tests) along with their reliability and validity, if known. Reference to where data collection forms can be found, if not in the protocol

11-12__
18b Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols 15 (no missing data expected)__

Data management 19
Plans for data entry, coding, security, and storage, including any related processes to promote data quality (eg, double data entry; range checks for data values). Reference to where details of data management procedures can be found, if not in the protocol

16-17__
Statistical methods 20a Statistical methods for analysing primary and secondary outcomes. Reference to where other details of the statistical analysis plan can be found, if not in the protocol

14-16________
20b Methods for any additional analyses (eg, subgroup and adjusted analyses) 16_____ 20c Definition of analysis population relating to protocol non-adherence (eg, as randomised analysis), and any statistical methods to handle missing data (eg, multiple imputation) 15 (no missing data expected)___

Methods: Monitoring
Data monitoring 21a Composition of data monitoring committee (DMC); summary of its role and reporting structure; statement of whether it is independent from the sponsor and competing interests; and reference to where further details about its charter can be found, if not in the protocol. Alternatively, an explanation of why a DMC is not needed 24-25_________ 21b Description of any interim analyses and stopping guidelines, including who will have access to these interim results and make the final decision to terminate the trial 15___________ Harms 22 Plans for collecting, assessing, reporting, and managing solicited and spontaneously reported adverse events and other unintended effects of trial interventions or trial conduct NA__ How personal information about potential and enrolled participants will be collected, shared, and maintained in order to protect confidentiality before, during, and after the trial 17_ Declaration of interests 28 Financial and other competing interests for principal investigators for the overall trial and each study site 24_____________ Access to data 29 Statement of who will have access to the final trial dataset, and disclosure of contractual agreements that limit such access for investigators 20__ Ancillary and posttrial care 30 Provisions, if any, for ancillary and post-trial care, and for compensation to those who suffer harm from trial participation NA__ Dissemination policy 31a Plans for investigators and sponsor to communicate trial results to participants, healthcare professionals, the public, and other relevant groups (eg, via publication, reporting in results databases, or other data sharing arrangements), including any publication restrictions 20_________ 31b Authorship eligibility guidelines and any intended use of professional writers 20____________  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46