Article Text

PDF

Measuring ability to assess claims about treatment effects: the development of the ‘Claim Evaluation Tools’
  1. Astrid Austvoll-Dahlgren1,
  2. Daniel Semakula2,
  3. Allen Nsangi2,
  4. Andrew David Oxman1,
  5. Iain Chalmers3,
  6. Sarah Rosenbaum1,
  7. Øystein Guttersrud4
  8. The IHC Group
    1. 1Norwegian Institute of Public Health, Oslo, Norway
    2. 2Makerere University College of Health Sciences, Kampala, Uganda
    3. 3James Lind Initiative, Oxford, UK
    4. 4Norwegian Centre for Science Education, University of Oslo, Oslo, Norway
    1. Correspondence to Dr Astrid Austvoll-Dahlgren; astrid.austvoll-dahlgren{at}fhi.no

    Abstract

    Objectives To describe the development of the Claim Evaluation Tools, a set of flexible items to measure people's ability to assess claims about treatment effects.

    Setting Methodologists and members of the community (including children) in Uganda, Rwanda, Kenya, Norway, the UK and Australia.

    Participants In the iterative development of the items, we used purposeful sampling of people with training in research methodology, such as teachers of evidence-based medicine, as well as patients and members of the public from low-income and high-income countries. Development consisted of 4 processes: (1) determining the scope of the Claim Evaluation Tools and development of items; (2) expert item review and feedback (n=63); (3) cognitive interviews with children and adult end-users (n=109); and (4) piloting and administrative tests (n=956).

    Results The Claim Evaluation Tools database currently includes a battery of multiple-choice items. Each item begins with a scenario which is intended to be relevant across contexts, and which can be used for children (from age 10  and above), adult members of the public and health professionals. People with expertise in research methods judged the items to have face validity, and end-users judged them relevant and acceptable in their settings. In response to feedback from methodologists and end-users, we simplified some text, explained terms where needed, and redesigned formats and instructions.

    Conclusions The Claim Evaluation Tools database is a flexible resource from which researchers, teachers and others can design measurement instruments to meet their own requirements. These evaluation tools are being managed and made freely available for non-commercial use (on request) through Testing Treatments interactive (testingtreatments.org).

    Trial registration numbers PACTR201606001679337 and PACTR201606001676150; Pre-results.

    • evidence based medicine
    • hared decision making
    • health literacy
    • outcome measurement
    • multiple-choice
    • patient education

    This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

    Statistics from Altmetric.com

    Footnotes

    • Collaborators The IHC Group: Leila Cusack, Claire Glenton, Tammy Hoffmann, Margaret Kaseje, Simon Lewin, Leah Atieno Marende, Angela Morrelli, Michael Mugisha, Laetitia Nyirazinyoye, Kjetil Olsen, Matthew Oxman, Nelson K Sewamkambo, Anne Marie Uwitonze.

    • Contributors AA-D, ØG and AO wrote the protocol and the IHC group provided comments on the protocol. AA-D coordinated all of the development and evaluation process with support from AO. AA-D, DS, AN, IC, AO and MO drafted the items with input from the IHC group. AA-D, IC and AO were responsible for revising the items iteratively, based on the feedback received through the different processes. SR and AA-D were responsible for the format designs. AA-D and KO approached the IHC advisory group and other methodologists for feedback. AN, DS, LC, AA-D, MO, SR, TH and MK conducted the interviews with end-users. AN, DS, KO, LAM, MM, MK, NS, AMU and AA-D were involved in pilots and administrative tests in Uganda, Kenya, Rwanda and Norway, respectively. AA-D analysed the data from the pilots. AA-D authored this manuscript with significant input from the rest of the IHC group.

    • Funding The IHC project is funded in part by the Research Council of Norway-GLOBVAC project 220603.

    • Competing interests None declared.

    • Ethics approval Ethical approval was sought by the IHC project representatives in each country.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement All data are published as part of this study. All Claim Evaluation Tools are available on request for non-commercial use.

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Linked Articles