Article Text

Download PDFPDF

Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial
  1. Linn Brandt1,2,
  2. Per Olav Vandvik1,2,
  3. Pablo Alonso-Coello3,4,
  4. Elie A Akl4,5,
  5. Judith Thornton6,
  6. David Rigau3,
  7. Katie Adams6,
  8. Paul O'Connor6,
  9. Gordon Guyatt4,
  10. Annette Kristiansen1,2
    1. 1Department of Internal Medicine, Sykehuset Innlandet Hospital Trust, Gjøvik, Norway
    2. 2Faculty of Medicine, Institute of Health and Society, University of Oslo, Oslo, Norway
    3. 3Iberoamerican Cochrane Centre, Biomedical Research Institute (IIB Sant Pau-CIBERESP), Barcelona, Spain
    4. 4Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
    5. 5Department of Internal Medicine, American University of Beirut, Beirut, Lebanon
    6. 6National Institute for Health and Care Excellence, Manchester, UK
    1. Correspondence to Linn Brandt; brandtlinn{at}


    Objectives To investigate practicing physicians' preferences, perceived usefulness and understanding of a new multilayered guideline presentation format—compared to a standard format—as well as conceptual understanding of trustworthy guideline concepts.

    Design Participants attended a standardised lecture in which they were presented with a clinical scenario and randomised to view a guideline recommendation in a multilayered format or standard format after which they answered multiple-choice questions using clickers. Both groups were also presented and asked about guideline concepts.

    Setting Mandatory educational lectures in 7 non-academic and academic hospitals, and 2 settings involving primary care in Lebanon, Norway, Spain and the UK.

    Participants 181 practicing physicians in internal medicine (156) and general practice (25).

    Interventions A new digitally structured, multilayered guideline presentation format and a standard narrative presentation format currently in widespread use.

    Primary and secondary outcome measures Our primary outcome was preference for presentation format. Understanding, perceived usefulness and perception of absolute effects were secondary outcomes.

    Results 72% (95% CI 65 to 79) of participants preferred the multilayered format and 16% (95% CI 10 to 22) preferred the standard format. A majority agreed that recommendations (multilayered 86% vs standard 91%, p value=0.31) and evidence summaries (79% vs 77%, p value=0.76) were useful in the context of the clinical scenario. 72% of participants randomised to the multilayered format vs 58% for standard formats reported correct understanding of the recommendations (p value=0.06). Most participants elected an appropriate clinical action after viewing the recommendations (98% vs 92%, p value=0.10). 82% of the participants considered absolute effect estimates in evidence summaries helpful or crucial.

    Conclusions Clinicians clearly preferred a novel multilayered presentation format to the standard format. Whether the preferred format improves decision-making and has an impact on patient important outcomes merits further investigation.


    This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

    Statistics from

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


    • Collaborators Rob Fracisco, Frankie Achille and Sarah Rosenbaum.

    • Contributors LB, POV, EAA, DR, KA and POC collected the data. LB, POV and AK had access to all of the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis. LB was principal investigator on the study. LB, AK, POV, PA-C and AE contributed to the conception, design, and ethical approval of the study. LB, POV and AK contributed to writing the first draft of the article; and LB, POV, PA-C, EA, JT, DR, KA, POC, GG and AK contributed to editing and approval of the final manuscript.

    • Funding This work was supported by Sykehuset Innlandet Hospital Trust under grant number 150194 (PLUGGED IN), The Research Council of Norway (VERDIKT) under grant 193022 (EVICARE) and European Community's Seventh Framework Programme (FP7/2007–2013) under grant agreement number 258583 (DECIDE project).

    • Competing interests All authors have completed the ICMJE uniform disclosure form. LB, AK, GG and POV are members of a non-profit research and innovation project MAGIC:, which has an open technical platform where the new DECIDE multilayered formats were prototyped. All authors were either co-investigators or collaborators of the DECIDE project. LB, AK, GG, POV, PA-C, EA, JT and DR are members of the Grade Working group. The strategy evaluated in the study is based on the GRADE approach. No other competing interests were declared.

    • Ethics approval Research ethics boards in each participating country: Institutional Review Board (IRB) of the American University of Beirut (Lebanon), Ethics committee and R&D at NICE (UK), Ethics committee of the Hospital Sant Pau (Spain), Regional Ethical Committee South East (Norway).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Extra data can be accessed via the Dryad data repository at with the doi:10.5061/dryad.2qv30.