Article Text

Communication
Quality Output Checklist and Content Assessment (QuOCCA): a new tool for assessing research quality and reproducibility
  1. Martin E Héroux1,2,
  2. Annie A Butler1,2,
  3. Aidan G Cashin1,2,
  4. Euan J McCaughey1,3,
  5. Andrew J Affleck1,4,
  6. Michael A Green1,2,
  7. Andrew Cartwright1,
  8. Matthew Jones2,
  9. Kim M Kiely1,2,
  10. Kimberley S van Schooten1,2,
  11. Jasmine C Menant1,2,
  12. Michael Wewege1,2,
  13. Simon C Gandevia1,2
  1. 1 Neuroscience Research Australia, Sydney, New South Wales, Australia
  2. 2 University of New South Wales, Sydney, New South Wales, Australia
  3. 3 Queen Elizabeth National Spinal Injuries Unit, Queen Elizabeth University Hospital Campus, Glasgow, UK
  4. 4 Department of Neuropathology, Royal Prince Alfred Hospital, Camperdown, New South Wales, Australia
  1. Correspondence to Professor Simon C Gandevia; s.gandevia{at}neura.edu.au

Abstract

Research must be well designed, properly conducted and clearly and transparently reported. Our independent medical research institute wanted a simple, generic tool to assess the quality of the research conducted by its researchers, with the goal of identifying areas that could be improved through targeted educational activities. Unfortunately, none was available, thus we devised our own. Here, we report development of the Quality Output Checklist and Content Assessment (QuOCCA), and its application to publications from our institute’s scientists. Following consensus meetings and external review by statistical and methodological experts, 11 items were selected for the final version of the QuOCCA: research transparency (items 1–3), research design and analysis (items 4–6) and research reporting practices (items 7–11). Five pairs of raters assessed all 231 articles published in 2017 and 221 in 2018 by researchers at our institute. Overall, the results were similar between years and revealed limited engagement with several recommended practices highlighted in the QuOCCA. These results will be useful to guide educational initiatives and their effectiveness. The QuOCCA is brief and focuses on broadly applicable and relevant concepts to open, high-quality, reproducible and well-reported science. Thus, the QuOCCA could be used by other biomedical institutions and individual researchers to evaluate research publications, assess changes in research practice over time and guide the discussion about high-quality, open science. Given its generic nature, the QuOCCA may also be useful in other research disciplines.

  • education & training (see medical education & training)
  • protocols & guidelines
  • medical education & training
  • statistics & research methods
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Supplementary materials

Footnotes

  • Twitter @AidanCashin, @EuanMcCaughey, @Mattjones0203

  • MEH and AAB contributed equally.

  • Contributors MEH: conceptualisation, design of checklist, auditing manuscripts, interpretation of results, drafting and revising the manuscript. AAB: conceptualisation, design of checklist, auditing of manuscripts, analysis of data, interpretation of results, preparation of figures, drafting and revising the manuscript. AC, EJM: conceptualisation, design of checklist, auditing of manuscripts, interpretation of results, drafting and revising the manuscript. AJA, MAG: conceptualisation, design of checklist, auditing of manuscripts, interpretation of results, revising the manuscript. AC: conceptualisation, design of checklist, auditing of manuscripts, online implementation and data management, interpretation of results, revising the manuscript. SCG: conceptualisation, design of checklist, auditing of manuscripts, interpretation of results, revising the manuscript, guarantor who accepts full responsibility for the finished work and conduct of the study. MJ, KMK, KSvS, JCM, MW: auditing of manuscripts, interpretation of results, revising the manuscript. All authors approved the final manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.