Article Text
PDF
Abstract
Objectives While documented plans for patient and public involvement (PPI) in research are required in many grant applications, little is known about how frequently PPI occurs in practice. Low levels of reported PPI may mask actual activity due to limited PPI reporting requirements. This research analysed the frequency and types of reported PPI in the presence and absence of a journal requirement to include this information.
Design and setting A before and after comparison of PPI reported in research papers published in The BMJ before and 1 year after the introduction of a journal policy requiring authors to report if and how they involved patients and the public within their papers.
Results Between 1 June 2013 and 31 May 2014, The BMJ published 189 research papers and 1 (0.5%) reported PPI activity. From 1 June 2015 to 31 May 2016, following the introduction of the policy, The BMJ published 152 research papers of which 16 (11%) reported PPI activity. Patients contributed to grant applications in addition to designing studies through to coauthorship and participation in study dissemination. Patient contributors were often not fully acknowledged; 6 of 17 (35%) papers acknowledged their contributions and 2 (12%) included them as coauthors.
Conclusions Infrequent reporting of PPI activity does not appear to be purely due to a failure of documentation. Reporting of PPI activity increased after the introduction of The BMJ’s policy, but activity both before and after was low and reporting was inconsistent in quality. Journals, funders and research institutions should collaborate to move us from the current situation where PPI is an optional extra to one where PPI is fully embedded in practice throughout the research process.
- public and patient involvement
- research reporting
- PPI
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Plans for patient and public involvement (PPI) in research are required by many funders and ethical review boards, but actual implementation of PPI in practice is an under-reported area.
Low levels of reported PPI may mask actual PPI activity due to limited reporting requirements. We describe reporting rates in the presence and absence of a journal requirement to report this information.
We only sampled research published in one journal, but it was the change in The BMJ’s policy that enabled a before and after study.
We cannot attribute causation to the implementation of this policy as the introduction of the policy itself may have attracted more research papers describing studies that included PPI.
Background
Patient and public involvement (PPI) in research is defined as research actively carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them.1 In health research, the term public is interpreted broadly to include potential patients, carers, people who use health and social services, people or organisations that support users of health or social services and other interested members of the community. Terminology for PPI can be confusing and varies across cultures; terms can be used interchangeably. We consider PPI to be distinct from public engagement where research awareness is raised, and knowledge is shared and nurtured through fostering conversations between clinicians and researchers and the public, patients and carers. PPI is also distinct from being a participant in a research study. In PPI, the public become active partners in one or multiple aspects of the research, including generating the research question, grant writing, study design, study conduct, analysis, evaluation, cowriting publications and their dissemination.2 Involving patients and the public in research can improve the quality, consistency, content, experience and value of health research to end users.2–4 Studies involving patients as research partners have shown improvements in study recruitment and participation5 6, in policy formulation7 and clinical relevance.8 9 PPI in prioritising research questions may also reduce research ‘waste’ by putting greater focus on addressing the questions that matter to patients.10 PPI also acts as a catalyst for researchers, the public and decision-makers to brainstorm research problems and find solutions together.11–13
For at least 15 years, ethical review boards and funders in the UK have been stipulating that members of the public be involved in research design and study conduct. They recommend lay members play an active role on research steering boards and in approval committees.14 Practice varies by organisation and country. While there are minimal data on the effectiveness of these policies, the UK has seen an increase (from 67% in 2010 to 78% in 2012) in the documentation of planned PPI in research applications submitted to the UK’s National Research Ethics Service (NRES).15 The extent to which PPI is embedded in research is an under-reported area and while a few studies have addressed this area, they were conducted some time ago and limited to the UK setting when the requirements to include patients and the public in research was relatively new. Hanley et al 16 reported that one-third of 62 UK clinical trial coordinating centres surveyed had involved ‘consumers’ in the research they had conducted between 1990 and 1998. Barber et al 17 conducted a survey of 518 UK researchers with projects registered on the National Research Register (ie, projects funded by or of interest to the UK NHS) and due to be completed in 2002 and found 17% involved ‘consumers’. In 2002, Telford et al 18 reported that one-third of NHS trusts in the region surveyed were actively involving ‘consumers’ in the research process.
Reporting of PPI is hampered by inconsistent terminology and little consensus on reporting methods.2 Journals and peer reviewers do not usually request PPI information within manuscripts, so an absence of information about PPI in research papers could be attributable to either a lack of reporting or a lack of PPI activity. As part of its Patient Partnership Strategy, The BMJ, an influential international general medical journal, is actively promoting PPI in research,19 20 and in 2014 it introduced an innovative policy instructing submitting authors to report if and how they involved patients. Authors of research papers are required to include a PPI declaration in their Methods section under a subheading called Patient Involvement. This policy is intended to foster increased researcher/patient collaboration by shifting cultural expectations about the importance of conducting and reporting on PPI. Authors of all research paper submissions are required to state if and how patients were involved in developing the research question and outcome measures, designing and conducting the research, assessing the burden of the intervention (where applicable) and how the results will be disseminated to patients. Authors are also instructed to thank patients involved in the contributorship statement/acknowledgments. Where authors have not involved patients, they are asked explicitly to report this in the paper. Papers describing studies without patient involvement are not rejected solely for this reason. At the point of article acceptance, editors check that information about PPI activity is recorded in the manuscript and request this declaration if it is absent.
We describe a comparison of published research papers in The BMJ before and after the introduction of its PPI policy to assess how researchers are involving patients and the public in research, whether the policy leads to an increase in reporting of PPI and whether infrequent documentation of PPI information in research is a reporting phenomenon or an absence of PPI activity.
Methods
Sampling
We identified two samples of research papers published in The BMJ. The first sample was original research papers published under the research section of the journal in the 12-month period between 1 June 2013 and 31 May 2014, just before the introduction of new journal guidance to submitting authors about the need to report if and how they involved patients in the research described. The second sample included research papers published under the research section of the journal in the 12-month period between 1 June 2015 and 31 May 2016, 1 year after The BMJ introduced its PPI reporting policy for submitted research papers.
Inclusion criteria
We included all research papers published in the two sampling periods. No study designs were excluded as PPI is considered possible with studies that have no direct contact with participants for example, systematic reviews.21–23
Process
Each research paper in the pre-implementation sample was read independently by two assessors, and all information about how patients were involved in each stage of the research process was extracted. Footnotes and contributorship notes were also checked. For the post-implementation sample, the new PPI section entitled Patient Involvement within the Methods section was read by two assessors, and if PPI activity was indicated assessors then read the whole paper and extracted all information about PPI wherever the details were reported. Data were also extracted from the Acknowledgments section and the Contributorship statement for all papers in the post-implementation sample. For the few papers in the post-implementation sample where a PPI section was not included within the manuscript, the information was checked and extracted in the same way as for the pre-implementation sample. Funding sources were extracted and classified as industry (commercial), trust (charities, trusts, foundations), governmental (government research organisation funded by taxpayers or works commissioned and paid for by government) or none (no external funding received).
Patient involvement
The study was instigated by RS (former Patient Editor, The BMJ) and SS. AP, RS, TR are/were patients with long-term medical conditions committed to the involvement of patients in all stages of the research process and were involved in all phases of this study. Two BMJ patient reviewers, MH and RH, were involved with the interpretation of the results, editing and preparing the manuscript for publication and will be involved in the dissemination plan for the paper. All patient contributors meet the ICMJE criteria for authorship and as such we acknowledge their valuable contributions through coauthorship of this manuscript. PPI enriched our understanding of patient involvement and contributed to the accuracy, readability and relevance of the paper.
Results
Frequency of reporting
Between 1 June 2013 and 31 May 2014, The BMJ published 189 research papers of which 62 (33%) had a corresponding author based in the UK. Only one (0.5%) of the 189 included some information on PPI activity (table 1).24
Studies reporting some PPI in the research described: pre-implementation period
Between 1 June 2015 and 31 May 2016, following the introduction of the PPI policy, The BMJ published 152 research papers of which 37 (24%) had a corresponding author based in the UK. A total of 130 (86%) of the 152 papers included a PPI statement within the Methods section of the manuscript. Sixteen (11%) of the 152 papers reported some PPI activity (table 2). This was a 10-fold increase in the proportion of papers reporting PPI compared with pre-implementation, but still only a small proportion of the total number of papers published. Some papers included information in the Patient Involvement section that did not describe PPI, and it was sometimes hard to distinguish qualitative data collection for example, capturing patient’s perspectives from active participation in the research process.
Studies reporting some PPI in the research described: post-implementation period
Types of PPI reported
Table 3 shows a summary of PPI activity reported at each stage of the research process across the two sampling periods. As the pre-implementation sample only consisted of one article, it is not appropriate to compare the type of reporting between the two samples.
Type of PPI explicitly reported before and after the introduction of a mandatory reporting policy
Pre-implementation sample
The Clinical effectiveness of collaborative care for depression in UK primary care (CADET) cluster randomised controlled trial (RCT) comparing the clinical effectiveness of collaborative care with usual care in the management of patients with moderate to severe depression in the UK primary care setting included a patient advocate as a coauthor on the byline of the paper.24 The contributorship statement implies that the patient was a full member of the research team and as such would have had the opportunity to contribute to multiple aspects of the research, yet it is difficult to extrapolate explicitly which aspects and phases of the research the patient advocate contributed to other than ‘writing and editing the manuscript’. This case illustrates how in the absence of guidance on how and what to report for PPI, activity may go unreported. Before conducting this trial, the team developed the collaborative care intervention and captured the patients’ perspective through in-depth qualitative interviews with patients. Their phase II testing of the intervention indicated that it was acceptable to patients. It was difficult to untangle whether this was active PPI or qualitative research eliciting the patient perspective. The rationale for including this as PPI was that testing the acceptability using participant feedback guided the course of the research.
Post-implementation sample
The sixteen studies included eight RCTs,25–32 five cohort studies,33–37 one population-based study,38 one real-world effectiveness study39 and one systematic review and meta-analysis40 (table 2). Eleven of these 16 studies had a corresponding author based in the UK, one in the USA, two in the Netherlands, one in Mexico and one in Sweden. Patients were involved from the earliest stages of study design in grant applications or contributing to study protocols (19%), helping to set the research question or commenting on its importance (25%), ensuring the development or choice of outcome measures were important to patients (44%), assessing the burden of interventions or design of the intervention itself (44%) (table 3). In the study conduct phase, they participated as members of steering groups and research teams (38%), helped with participant recruitment or implementation (50%) and contributed to patient and public communication documents and materials (25%). They contributed to data analysis (6%) and the interpretation of study findings (13%), editing, revising and writing the manuscript (19%) and directly in the development of dissemination materials (19%) to a lesser extent.
Direct PPI initiated by the research team in the dissemination phase was described in three papers in the post-implementation sample.28 29 39 In two of these three papers, patient coinvestigators or contributors were credited with helping to ensure the dissemination materials, and methods of dissemination were appropriate to reach beyond the research community to general populations.29 39 In the third paper, patients and their families were described as "central to the dissemination of the baseline information, which helped to motivate community involvement during and beyond the study".28 In a further three papers, PPI in the dissemination phase was not explicitly reported but might be implied as patients or representatives were on the research steering committee and as such could have had the opportunity to contribute ideas to the format and method of dissemination.25–27 In addition, there were three papers from the QRisk team reporting identical statements about some aspects of PPI in dissemination,33–35 but this PPI was specific to the QResearch database, not the individual published studies leaving uncertainty as to the extent of PPI within the individual studies. Three papers reported that results would be disseminated through patient and charitable organisations who may, in turn, involve patients in the dissemination process, but there was no reported PPI in dissemination conducted by the research team.31 36 38
Illustrative examples of PPI
We describe four examples from the post-implementation sample showing how researchers involved the patients and the public in several stages of research. These examples were chosen as they describe a range of approaches to involving patients at different stages in the research process. For more details of how other papers in the postimplementation period reported PPI, see table 2.
In the UK’s Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT) trial, Gilbody et al 29 involved patients and members of the public in multiple ways, including the design, management and conduct of the trial. Input in the design of trial materials was received from patients who lived with depression and other common mental health problems. Trial management oversight included having patient members on the trial steering committee. A user-led organisation acted as grant coapplicant and as research collaborators. The authors reported that they carefully assessed the burden of the trial interventions on patients. Gilbody et al thank their patient and public contributors for their valuable input throughout the trial and plan to disseminate the results to trial participants and to seek PPI for developing the most appropriate dissemination method.
Andersson et al 28 describe an RCT to develop community mobilisation for dengue prevention in Nicaragua and Mexico. Protocol baseline results were discussed with patients, and they were intimately involved in design and implementation of the intervention. Local community leaders became research team collaborators, and they invited community volunteers to receive training as organisers and educators. Once trained, they conducted home and school visits to show dwellers the evidence of larval/pupal infestation in water receptacles, to inform households and schools of the mosquito’s life cycle and to counsel on ways to interrupt the cycle. The research team worked with local leaders who added interventions as their community work advanced. They worked together by agreeing on a common protocol to generate community-led interventions and to recommend participants for the intervention.
In the UK’s Comparison of the two most commonly used treatments for pyoderma gangrenosum (STOP GAP) trial, Ormerod et al 25 compared the two most commonly used treatments for pyoderma gangrenosum and involved patients in the trial design and conducting the study. Early on in the feasibility stage, patients prioritised the research question and selected the most relevant outcome measures. The methods of study recruitment were informed by discussions with patients in a focus group and two structured interviews. During the trial implementation phase, a patient joined the independent trial steering committee.
In the real world effectiveness of warfarin among ischemic stroke patients with atrial fibrillation: observational analysis from Patient-Centered Research into Outcomes Stroke Patients Prefer and Effectiveness Research (PROSPER) study, sponsored by the Patient-Centerd Outcomes Research Institute (PCORI), Xian et al 39 assessed the real-world effectiveness of warfarin among patients with ischaemic stroke with atrial fibrillation in the USA. They report partnering with patients throughout each stage of the research process. Patient coinvestigators worked iteratively with the research team to identify patient relevant research topics, healthcare priorities and meaningful outcomes for stroke survivors and then to develop patient-centred research questions. Patients also contributed to the statistical analysis plan and the interpretation of the results to ensure that findings were disseminated in the most effective way to the general population. The value of including longitudinal functional outcomes, as prioritised by stroke survivors and stakeholders, was credited as a study strength. Patient coinvestigators were credited with making ‘high-value contributions to both the design and implementation of the study’. Three patient coinvestigators were made coauthors on the manuscript.
Acknowledgement and thanks for specific patient contributions to the research
This section describes how input from patients and the public was acknowledged in the papers. In 2 of the 17 papers, patients were included as coauthors on the authorship byline.24 39 In six papers,25–27 29 30 39 patient contributors and advocates were thanked or acknowledged for their contributions. In an additional paper, the value of the role of the patient contributions was mentioned in the main text, but no specific thanks were given in the acknowledgments.28 A further paper named and thanked people for helping with manuscript drafts, but it was not specified whether they were patients.32 In some manuscripts, organisations were named, however, individual collaborators were not and this made it difficult to appraise individual patient versus organisational choices that may or may not have been informed by patients. In 8 of 17 papers, patient contributors were not named, thanked or the value of their contributions acknowledged in general.
Research funding
Governmental funding was accessed in all 17 studies reporting PI activity and all were led by academic principal investigators. Two studies30 32 were partially funded by charitable trusts.
Discussion
Based on this analysis of published research papers in The BMJ, we found that a higher proportion of papers reported PPI activity after the introduction of a policy requiring this information be included, but there was a low level of reporting of PPI both before and after the introduction of this policy. This implies that the absence of information about PPI in research papers is not solely due to a lack of reporting requirments but a lack of PPI activity in practice or unwillingness to report unsuccessful PPI. We took an inclusive approach and where any PPI was reported we counted it; but, some descriptions were of poor quality and lacked depth. However, some researchers who conducted the studies we sampled did find innovative ways of involving patients in all aspects of the research process. PPI in the early stages of research is recommended by organisations such as INVOLVE,41 Canada’s Strategy for Patient-Oriented Research (SPOR),42 and PCORI43 and widely considered to be best practice. We found examples of patients being involved from the earliest stages in grant applications and study design. They participated in steering groups, participant recruitment, the identification and development of outcome measures, pilot testing platforms, conceiving interview questions and implementation of the interventions. They were included in writing the manuscripts, revising drafts and in the dissemination of study results. Despite this key input, across 17 papers, only six acknowledged patients’ contributions and two included patient contributors as coauthors.
The implementation of PPI varies internationally. While authors in our study will have initiated their research prior to the introduction of The BMJ’s policy, ethical review boards and funding agencies in the UK have been requesting researchers to carry out and describe PPI for over 15 years. Recent data available on the extent of PPI in conducted research are sparse, but our findings are similar to earlier studies.16–18 While there has been some recent documentation on the rate of planned PPI in research submitted to the UK’s NRES (78% in 2012),15 research is not always conducted as planned, and it is important to evaluate the extent to which planned PPI gets implemented in practice. The discrepancy between our findings and those of NRES can partially be explained by the fact that we included all study designs, research from outside the UK and that the cohort of studies under ethical review in 2012 may not have been published yet. We included papers describing systematic reviews, data linkage studies and secondary analyses. Some might argue that it is more difficult to involve patients in these study designs and that this has led to an underestimate of the rate of reporting. However, The Agency for Healthcare Research and Quality Evidence-based Practice Centres, the Institute of Medicine, the Cochrane Collaboration and others have recommended PPI when conducting systematic reviews, we used this justification to include them.14 We recognise that many studies are secondary analyses of existing datasets or data linkage studies, and as such it may be harder to involve patients in the research process. However, there is still potential to include PPI in these types of studies and even in our study, we found examples where authors reported PPI in studies with these designs.33–35 38 40 For example, Amick et al 40 conducted a systematic review and meta-analysis of antidepressants and cognitive behavioural therapies and involved a representative of the National (US) Board of Directors of the National Alliance on Mental Illness in the refinement of the research topic and the development of the preliminary research questions and review criteria. The research question and a draft version of the report were posted online for input from all members of the public, including patients and their advocates. Saadatmand et al 38 conducted a population-based study in the Netherlands where they relied on regular contact with patient-led and national breast cancer organisations to guide them in making the outcome measures clear for patients and members of the public to make informed choices about treatment and breast cancer screening and treatment.
Our study has several limitations. First, at the time of publication for papers in the pre-implementation sample in our study, there was no standardised format for reporting PPI information. This made it challenging to glean from the papers if and how patients were involved and as such we may have underestimated the rate of involvement. Our focus was on how PPI was reported in the published papers rather than on contacting the authors for further information or searching supporting material published elsewhere. We acknowledge that some studies may have involved patients in a range of ways and not reported this in the published paper. Authors have to balance the need to write succinctly with providing enough detail so that the research can be replicated. As editors and peer reviewers do not usually request PPI reporting, authors may not have considered including it and some may perceive it to have less value than other aspects of the research. Second, we only included one journal in our analysis and the results may not be generalisable to other biomedical research. However, it was The BMJ’s change in editorial policy around requirements for reporting PPI that made this before and after study possible. Third, it is possible that the frequency of reporting PPI in studies published by The BMJ is higher than current research practice as The BMJ has an active policy of reporting this involvement and promoting patient partnership.19 20 Finally, we cannot attribute causation to the implementation of this policy as the introduction of the policy itself may have attracted more research papers describing studies that included PPI.
There is substantial variation in the quality and content of reported PPI in the literature with limited methodological guidance for ways to initiate, sustain or report public involvement in research.44 Concannon et al 45 developed a set of seven questions to guide researchers in reporting PPI. In 2017, the first international guidance for reporting of PPI in health and social care research (GRIPP2 reporting checklist)46 was published with the aim of improving the quality, transparency and consistency of the PPI evidence base and ensuring that practice is based on the best evidence. GRIPP2 is included in the Equator network (https://www.equator-network.org). Although it will take time to evaluate the influence of this reporting guideline in practice, it is a promising development. Clearer reporting of PPI could provide guidance for future research, set standards and improve best practice,47 reduce research ‘waste’, increase public goodwill and build a bank of viable ways to effectively involve patients and the public in research.10 We saw an increase in reporting of PPI after the introduction of a journal policy requiring this information be included. However, until requirements for PPI reporting are standardised and enforced by journals, funding institutions and sponsors, there is a danger that much research methodology is likely to remain unreported and will contribute to research waste.2 10
Conclusions
PPI needs to be seen as an integral part of the research process and key to increasing its value. Best practice for reporting PPI should be supported and encouraged as this serves to help others conduct and report more effective PPI. Funding organisations increasingly require researchers to involve patients and the public in their research,48 but closer monitoring of how planned PPI documented in grant applications transfers into practice is needed. Previous research has identified barriers to effective PPI,18 49–51 and these can be tackled by fostering collaboration, providing researchers with guidance and training on how to do PPI, and by providing adequate resources. Funders should ensure that dedicated provision is adequate to enable high-quality PPI and training and make reporting of PPI a mandatory requirement for continued funding. Researchers, patients and members of the public should share within their papers, and through dissemination materials what worked well and work in collaboration to bridge gaps and increase PPI awareness and understanding. Journals should play a role in encouraging researchers to report PPI in a replicable way. Requiring a PPI statement within the Methods section of research papers should be adopted by other journals to help integrate meaningful PPI into the research culture.
Acknowledgments
Our powerful and inspiring coauthor Rosamund Snow passed away before this work could be published. We are thankful for her insight and steadfast leadership as we launched the mandate for authors to report how they involved patients in their research. We thank Sarah Foster (BMJ intern) for her help with data extraction. We are thankful to The BMJ authors and their collaborators who made this research possible .
References
Footnotes
Dr. Snow died on 2 February 2017.
Contributors SSc, RS, AP and SSt designed the study. SP, AP and SS documented how patients were involved in the published studies. SSc and AP analysed the data. AP wrote the first draft of the manuscript with SSc. AP, SSc, MH, RH, SSt, SP, and TR contributed to the interpretation of the results and the writing of the manuscript for publication.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests SSc, RS, SP, TR are or were employed by The BMJ which has a patient partnership initiative and requires authors to report how they involved patients in their research. AP is the Patient Editor (Research and Evaluation) at The BMJ, SSt is the Co-editor in Chief of the journal Research Involvement and Engagement and is the lead author of the GRIPP2 reporting checklist mentioned in this paper. MH and RH are patient reviewers for The BMJ. MH is also a patient reviewer for PCORI, and RH is a patient reviewer for several grant application panels and organisations. All authors support the call for greater public involvement in research.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data available.