Article Text

Download PDFPDF

Tracking the impact of research on policy and practice: investigating the feasibility of using citations in clinical guidelines for research evaluation
  1. David Kryl1,
  2. Liz Allen2,
  3. Kevin Dolby2,
  4. Beverley Sherbon3,
  5. Ian Viney3
  1. 1Research Information and Intelligence Branch, National Institute for Health Research, London, UK
  2. 2Strategic Planning & Policy Unit, Wellcome Trust, London, UK
  3. 3Strategy Group, Medical Research Council, London, UK
  1. Correspondence to Dr Beverley Sherbon; beverley.sherbon{at}


Objectives To investigate the feasibility of using research papers cited in clinical guidelines as a way to track the impact of particular funding streams or sources.

Setting In recent years, medical research funders have made efforts to enhance the understanding of the impact of their funded research and to provide evidence of the ‘value’ of investments in particular areas of research. One of the most challenging areas of research evaluation is around impact on policy and practice. In the UK, the National Institute of Health and Clinical Excellence (NICE) provide clinical guidelines, which bring together current high-quality evidence on the diagnosis and treatment of clinical problems. Research referenced in these guidelines is an indication of its potential to have real impact on health policy and practice.

Design This study is based on analysis of the authorship and funding attribution of research cited in two NICE clinical guidelines: dementia and chronic obstructive pulmonary disease.

Results Analysis identified that around a third of papers cited in the two NICE guidelines had at least one author based in the UK. In both cases, about half of these UK attributed papers contained acknowledgements which allowed the source of funding for the research to be identified. The research cited in these guidelines was found to have been supported by a diverse set of funders from different sectors. The study also investigated the contribution of research groups based in universities, industry and the public sector.

Conclusions The study found that there is great potential for guidelines to be used as sources of information on the quality of the research used in their development and that it is possible to track the source of the funding of the research. The challenge is in harnessing the relevant information to track this in an efficient way.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: and

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • To cite: Kryl D, Allen L, Dolby K, et al. Tracking the impact of research on policy and practice: investigating the feasibility of using citations in clinical guidelines for research evaluation. BMJ Open 2012;2:e000897. doi:10.1136/bmjopen-2012-000897

  • Contributors All authors (DK, LA, KD, BS and IV) contributed extensively to all parts of this paper and commented on the manuscript at all stages. All authors were involved in the design of the study with some guidance on direction and feasibility from RAND Europe. All the authors discussed the interpretation of the results of the study and agreed the outline content for this publication. DK wrote the initial draft of the paper, which was then reviewed and further shaped by all the authors. BS prepared the final version of the paper plus all related information and managed the submission and revision of this article.

  • Funding The authors are employed, or supported by, the Medical Research Council, the National Institute for Health Research and the Wellcome Trust.

  • Disclaimer This paper reflects the opinions of the authors and does not necessarily reflect the opinions of their respective organisations.

  • Competing interests All authors have completed the Unified Competing Interest form at (available on request from the corresponding author) and declare that (1) DK also holds the position of Research Leader at RAND Europe; (2) LA is a member of the ORCID Board of Directors and (3) DK, LA, KD, BS and IV have no non-financial interests that may be relevant to the submitted work.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Unpublished data from this study are not currently available.