You are here

Article Text

Article menu
XML
Health services research
Research
Patient and public involvement in the early stages of clinical trial development: a systematic cohort investigation
  1. Carrol Gamble1,
  2. Louise Dudley1,
  3. Alison Allam2,
  4. Philip Bell3,
  5. Heather Goodare4,
  6. Bec Hanley5,
  7. Jennifer Preston6,
  8. Alison Walker4,
  9. Paula Williamson1,
  10. Bridget Young7
  1. 1Department of Biostatistics, University of Liverpool, Liverpool, UK
  2. 2Patient and Public Advisory Group member, York, UK
  3. 3Patient and Public Advisory Group member, Amlwch, UK
  4. 4Patient and Public Advisory Group member, Edinburgh, UK
  5. 5Patient and Public Advisory Group member, Hurstpierpoint, Hassocks, England
  6. 6Medicines for Children Research Network Coordinating Centre, University of Liverpool, Liverpool, UK
  7. 7Department of Psychological Sciences, University of Liverpool, Liverpool, UK
  1. Correspondence to Professor Carrol Gamble; R.Gamble{at}liverpool.ac.uk

Abstract

Background Randomised controlled trials (RCTs) are considered particularly likely to benefit from patient and public involvement (PPI). Decisions made by professional researchers at the outset may go on to have a significant impact on the potential for PPI contributions.

Objective To increase knowledge of PPI within the early development of RCTs by systematically describing the reported level, nature and acceptability of proposed PPI to the funders.

Methods Documentation from the outline application process for all RCTs that received funding from the Health Technology Assessment (HTA) Programme 2006–2010 was requested. For each application, data were extracted on trial characteristics, references to PPI in the development of the outline application and funding Board feedback, and plans for PPI in the full application and after the trial was funded.

Results 110 applications were eligible with outline applications available for 90 (82%). The cohort covered a wide range of interventions and conditions. 54% (49/90) provided some information about PPI. 26 (28.9%) indicated PPI within the development of the outline application itself; 32 (35.6%) planned involvement in the full application and 43 (48%) once the trial was funded. Recruitment at diagnosis and surgical interventions were less likely to describe PPI. Blinded trials and trials in which participants may receive placebo only, more frequently described PPI activity. The HTA commissioning Board feedback rarely referred to PPI.

Conclusions Incorporation of PPI within the development of the outline application or specification of plans for future involvement was low. Funder requests for applicants to provide information on PPI and justification for its absence should be welcomed but further research is needed to identify the impact of this on its contributions to research. Comments on PPI by reviewers should be directional rather than state that an increase is required. Challenges facing applicants in initiating PPI prior to funding need to be addressed.

  • STATISTICS & RESEARCH METHODS
  • HEALTH SERVICES ADMINISTRATION & MANAGEMENT
  • Consumer participation

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2014-005234

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This is the first study to systematically investigate patient and public involvement (PPI) early in the development of funding applications and across a wide range of trials.

    • Presents results of a large unselected cohort addressing concerns of selective reporting of case studies within the literature.

    • Is the first study to consider clinical and trial characteristics in relation to the model of PPI adopted.

    • Cohort is limited to a single funder covering all randomised controlled trials receiving funding between 2006 and 2010.

    • Documentation could not be made available for unsuccessful applications. However, PPI was not mandatory within the period of the cohort, and that on its own would not have led to a decision that the outline application should not progress.

    • As the cohort was identified by receipt of funding during the period rather than the year that the outline application was submitted for consideration, fewer trials were available in the extremes of the cohort.

    • Changes to funder requirements for PPI have changed and this may impact generalisability.

    Background

    Public involvement in research is described as research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them (http://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/, last accessed 5 September 2013). It has been suggested that clinical trials are particularly likely to benefit from patient and public involvement (PPI).1 ,2 PPI can start at various stages of a trial and influence many aspects. Increased recognition that patients and public are stakeholders in research has led to increasing calls that they be represented within that research process. The UK Department of Health (2006) Best Research for Best Health guidelines3 state that PPI must be included in all stages of the research process including priority setting, defining research outcomes, selecting research methodology, patient recruitment, interpretation of findings and dissemination of results. Many funding bodies now request researchers to provide evidence of PPI when submitting their proposals.4 The National Institute of Health Research (NIHR) Health Technology Assessment (HTA) programme encourages applicants to consider whether the ‘scientific quality, feasibility or practicality of the proposal can be improved by patient and public involvement’ (http://www.hta.ac.uk/public/).

    Staniszewska et al5 discussed the importance of PPI being implemented at the design stage of studies to optimise the impact and relevance of research. During the design stages of a trial many decisions are made that determine the relevance and conduct of the proposed research: the precise specification of the research question, including the outcomes to be measured; visit schedules; methods of data collection; recruitment and consent procedures. Fudge et al6 suggest that decisions made by professional researchers at the outset of a study have a cumulative and significant influence on the potential for PPI to impact on a study and that involvement is more difficult to achieve once studies are under way. It is therefore important to consider PPI activity in the early stages of the research process. However, little is known about how or when researchers incorporate PPI or what impact may stem from that involvement. Concerns have been expressed that the existing literature is selectively reported with many reports aiming to ‘make the case’ or ‘convince the sceptics’ about PPI.7 Furthermore, as much reporting has involved single case studies, generalisability of the PPI literature is limited.7

    The aim of this paper is to increase the knowledge of PPI within the early stage design and development of clinical trials by systematically studying an unselected cohort of outline applications that were in receipt of funding from the HTA programme between 2006 and 2010. Specific objectives are to identify whether and how PPI is described within the early development of a grant application for funding; examine how PPI contributions within the development of the outline application were reviewed by the HTA Board; describe applicants’ PPI plans should their application be successful; and describe variations in PPI in relation to time of funding and trial characteristics.

    Methods

    We examined a cohort of randomised controlled trials (RCTs) that received funding from the NIHR HTA) programme in the UK between 2006 and 2010. During the period of the cohort the HTA programme encouraged PPI, and allowed for resources to support PPI activity to be requested; however, PPI was not mandatory. Inclusion of a PPI contributor on the Trial Steering Committee (TSC) was recommended. The HTA programme has a two-stage application process, comprising an outline application followed by a full application for those successful at the outline stage (http://www.hta.ac.uk/funding/clinicaltrials/index.shtml, accessed 30 April 2012). We requested all documentation relevant to the outline applications from the HTA. This comprised the commissioning briefs (if applicable) in which the topic of research driven by funders rather than applicants, outline applications and the minutes of the relevant Board meetings at which the outline applications were considered and which contained feedback for the applicant. Prior to the release of these documents the NIHR HTA contacted the chief investigators of the trials involved informing them that their names were released as these were readily available within the public domain. We signed a confidentiality agreement and the NIHR HTA redacted sensitive information regarding budget information and names of co-applicants (not available within the public domain) before releasing the documentation to us.

    Once the trial was funded we established a PPI advisory group of five members with experience of providing PPI in RCTs. Members were recruited through an open and transparent process. Advertisements for members were placed on various PPI websites and applicants who expressed an interest were sent an application pack which included a detailed remit and role description. Applicants were then short listed and invited to take part in an informal interview over the phone, and had the opportunity to ask further questions about the role of the group. To assist in extracting and analysing the information on PPI within the outline applications an access database was developed by CG. The advisory group commented on the data extraction tool and made recommendations for changes to variable descriptors and coding categories. All suggestions were discussed in a face-to-face meeting and agreement reached on their incorporation within the project.

    Data were extracted to characterise the cohort and to describe PPI activity within the application process and plans for involvement once trial funding was secured. Trial characteristics linked to trial complexity or thought to be barriers or facilitators to recruitment8 were also extracted. In brief, the extracted data included characteristics of the trial design and setting, proposed start date, disease or condition under study, type of intervention, participant characteristics, recruitment setting and any text that described or was relevant to PPI. Data were extracted by three reviewers (LD, JP and CG). The extracted text was anonymised by replacing any identifying details with a general term in brackets [term], or using [...] to indicate removed text (see online supplementary table S1).

    Text extracts were examined to determine whether PPI had been described and if so whether PPI had been undertaken during the development of the outline application (OL), was planned to occur during the development of the full application (F) and/or was planned to occur after a trial was funded (T). The text was also examined to determine the role of the PPI contributor's input. This was categorised as: managerial (M); responsive (R) and membership of Trial Steering Committee (TSC). We categorised PPI contributors as managerial if they were described as co-applicants or involved in the management of the trial. We categorised PPI contributors as having a responsive role if descriptions of their input were largely confined or targeted towards a particular aspects of the application or trial, or if PPI contributions were on an ‘as required’ basis. Descriptions of PPI in the outline applications were often limited, and we were therefore required to devise coding ‘rules’ (based on our knowledge of grant application development and clinical trial implementation processes) in order to categorise the descriptions. The codes were developed by CG and LD after reading the PPI descriptions and then reviewed by JP. CG and LD independently categorised the PPI descriptions. Disagreements between CG and LD were discussed and agreement reached. All categorisations were cross checked by JP. The coding rules (box 1) and the classifications (see online supplementary table S1) were reviewed by the PPI advisory group and no changes were requested.

    Box 1

    Coding rules used to inform the categorisation of patient and public involvement (PPI) descriptions in outline applications

    Coding rules for categorising stage of involvement

    1. If a PPI contributor is described as a member of the research team or ‘lay member’ categorise them as inputting across all future stages of the study.

    2. The design of the study is determined within the full application stage so if a PPI contributor is described as inputting into the design of the study categorise their input as starting no later than at development of the full application.

    3. If a PPI contributor's role is confined to Trial Steering Committee membership (which is usually agreed by the funders and follows the funding decision) categorise their input as starting after the full application regardless of the tense of the sentences describing their involvement.

    Coding rules for categorising role

    • If a PPI contributor's role is described as managerial or as a co-applicant or referred to as a part of the team categorise their role as managerial.

    • If a PPI contributor's role is confined to a panel or an advisory group categorise their contribution as responsive.

    • If a PPI contributor's role is limited to a specific aspect of the trial then categorise their input responsive.

    The cohort was identified as RCTs that actively received funding from the NIHR HTA programme between 2006 and 2010. In considering trends over time, the year the outline application was submitted was used.

    Specific conditions were selected to consider in more detail. These were identified either by their strong history of PPI or the establishment of an NIHR Clinical Research Networks for a specific condition (http://www.nihr.ac.uk/infrastructure/Pages/infrastructure_clinical_research_networks.aspx, last accessed 5 September 2013). The categories were mental health, pregnancy and childbirth, HIV/AIDS, cancer, stroke, paediatrics, diabetes and dementias and neurodegenerative diseases.

    Categorical data were summarised using descriptive statistics with numbers and percentages. χ2 Tests or Fishers exact tests were used as appropriate.

    Ethical approval was obtained from the University of Liverpool Institutional Ethics Board and the study was funded by INVOLVE and managed by NIHR HS&DR.

    Results

    Between 2006 and 2010, 111 RCTs received funding from the NIHR HTA Programme. Of these 110 were required to submit an outline application and were eligible for inclusion within the cohort. The dates of submission of the associated outline applications were between 2003 and 2008. Copies of the outline applications were available for 90 (82%) and comments from the Board on the outline application were available for 77 (70%). There were five trials in which there was a disagreement between data extractors, each of which related to the presence/absence of PPI text and each were resolved by referral to the application form.

    Table 1 provides a summary of the conditions and interventions. Trials were funded across a wide range of clinical conditions, most of which were for long-term conditions, with the most common area of study being trials in mental health (17%). The majority of trials (82.2%) were aimed at treatment of a condition with 14.4% being prevention trials. The trials used a wide range of interventions. Over a third investigated a medicinal product and just under a fifth each evaluated behavioural interventions (18.9%), surgical techniques (17.8%) and devices (16.7%).

    View this table:
    Table 1

    Summary of conditions and interventions

    Table 2 describes the characteristics of trial participants and features of the trial designs. Over three-quarters of the trials recruited adults only, with paediatric trials accounting for 16.7% of the cohort. The majority of the trials were not gender-specific and approximately a quarter recruited participants at the time of diagnosis. Trial recruitment was most commonly conducted within secondary care (61%). Just under a quarter of the trials involved blinding of the treating clinical team or the participants. Just over 15% of trials used a placebo with all participants receiving an active intervention in one-third of these, indicating the use of a double-dummy design. For example, in a double-dummy trial comparing two treatments A and B trial participants would receive either treatment A and a placebo of treatment B, or a placebo of treatments A and B.

    View this table:
    Table 2

    Trial participant and design characteristics

    Of the 90 outline applications for which documentation was available 49 (49/90=54%) provided some information on PPI. Extracts from each application along with the classifications on the stage and role of PPI are provided in online supplementary table S1. Table 3 summarises the classifications across the trials. The first row of table 3 shows that there were 19 applications in which the text provided within the outline described PPI occurring at all three stages (within the outline application, full application and once the trial was funded). Of these 19 applications the role was managerial (M) at each stage in 13. In the remaining six there was variation in the roles across the stages, or it was unclear when a statement indicated that PPI would occur but no detail was provided to allow classification. An example of a description that we were unable to categorise is: Investigators have, and will continue to, collaborate with service users.

    View this table:
    Table 3

    Summary of stages of PPI by stage and role

    Twenty-six applications (26/90=28.9%) specified a role of PPI within the development of the outline application. In 13 this was managerial, on a responsive basis in seven, unclear in two and ‘other’ approaches used in four (eg, a patient survey or pilot feedback). Within the ‘other’ approaches it was difficult to determine conclusively whether this was PPI or whether they were examples of data collection aimed at ascertaining the public opinion. In the three applications that specified use of a survey the extent of the distribution of the survey was unclear in two. PPI was planned to occur within the full application for 32 trials (35.6%). In 18 this was managerial, responsive in 9 and unclear in 5. Forty-three (48%) applications indicated that PPI was planned after the trial was funded. This was as: managerial in 22; responsive in 6; a member of the TSC in 8; unclear in 5 and other in 2. Table 4 provides a summary of the specification of PPI within the outline applications by the disease area. The numbers of outline applications by year with and without details of PPI are displayed in figure 1, with figure 2 showing the percentage of applications with PPI. Figure 1 shows a general trend for increasing the number of funded applications; however, the proportion of those containing PPI fluctuates, ranging from approximately one half to two-thirds (figure 2), with the exception of 2003, for which only one application was available.

    View this table:
    Table 4

    PPI specified within the outline application by disease area

    Figure 1
    Figure 1

    Number of outline applications by the year in which the application was made (PPI, patient and public involvement).

    Figure 2
    Figure 2

    Percentage of outline applications containing patient and public involvement (PPI) details by the year in which the application was made. The number of trials included within each year is indicated at the top of each bar.

    Figure 3 and table 5 provide the data by year for specific conditions. Both of the diabetes trials were in children and were coded as paediatric, and there were no HIV/AIDS trials within the cohort. Figure 3 suggests declining rates of PPI in paediatric and mental health, with other areas including the general other category demonstrating an increase over time; however, as shown in table 5, the numbers in some categories were small.

    View this table:
    Table 5

    Number of cumulative applications by disease/condition category

    Figure 3
    Figure 3

    Cumulative percentages of outline applications by disease/condition (PPI, patient and public involvement).

    Table 6 shows the associations between trial design characteristics and characteristics of the condition under study, with consideration of PPI within the outline application.

    View this table:
    Table 6

    PPI specified within the outline application by condition and trial characteristics

    Of the intervention aims, numbers were too small to draw conclusions about involvement in diagnostic studies, but prevention trials more frequently described PPI than intervention trials. Trials involving educational, behavioural or physical interventions were more likely to have provided details of PPI in the outline application than trials involving drugs or devices. Surgical trials were significantly less likely to have provided details of PPI.

    The settings for trial recruitment did not appear to impact on specification of PPI, while recruiting participants at the point of diagnosis was associated with less PPI.

    Forty-three trials were described as being blind and these trials were associated with increased frequency of describing PPI. Of these trials 23 involved blinding of the outcome assessor only. Only 14 trials involved a placebo, and of these, participants in 5 trials all received an active intervention with the placebo used as a double dummy. The allocation of a placebo only to one arm of the trial was significantly associated with greater frequency of PPI detail.

    Only two trials were in rare conditions. Both of these trials provided detail of PPI within the outline application. While there appeared to be some increase in describing PPI when the condition was long term there did not appear to be any influence based on impact of life expectancy.

    Board comments on outline applications

    NIHR HTA Board minutes were available for 77 (77/110 70%) outline applications. Only nine (9/77 12%) Board minutes gave feedback on PPI representation and two of these did so indirectly (table 7). One comment was supportive of the PPI described in the outline, which involved a PPI contributor as a co-applicant.

    View this table:
    Table 7

    Board comments on outline applications relating to patient and public involvement (PPI)

    The corresponding outline applications were also available for 64 (84%) of the 77 applications for which Board minutes were available. Of these applications 25 (25/64=39%) gave no information about PPI.

    Eight of the nine sets of Board comments that were made about PPI expressed the need for applicants to increase PPI. Six contained no detail about PPI within the application. Of these six, two were drug trials, three were exercise interventions and one was comparing an invasive with a non-invasive intervention. Two were recruiting participants with addiction: smoking and alcohol; two were recruiting elderly participants, one was recruiting infants, and one was recruiting participants with a long-term chronic debilitating condition.

    In the two outline applications which had given some details on PPI, comments from the Board were to further increase PPI. In one application the PPI contributors were employed within an NIHR condition-specific network in roles relating to PPI. These individuals also had relevant experience as carers or patients of the condition. In the other application the PPI contributor was a co-applicant with an unrelated clinical background (midwife) and was the mother of a child with the condition being studied.

    Discussion

    Summary

    This is the first study to systematically investigate whether and how PPI is described within the earliest stages of trial development across a cohort of studies. The cohort covers a variety of clinical conditions and interventions, and this study is the first to examine associations between trial design, intervention and condition characteristics and applicants’ plans for PPI. Key findings are summarised in box 2. PPI was described in only half of the applications at the outline stage, with only a quarter indicating that PPI contributions were enlisted at the outline application stage itself. There was some evidence to suggest that the further the trial deviates from routine clinical practice the more likely the application is to describe PPI, and PPI was particularly frequent in applications for blinded trials or trials allocating participants to placebo only. This may indicate the beginning of a risk-based approach to PPI. However, applications for trials recruiting at the point of diagnosis or trials with surgical interventions were less likely to describe PPI. For trials recruiting at the time of diagnosis this may reflect the difficulty of identifying PPI representatives close to the time of their diagnosis, while for surgical trials this may reflect the lower frequency and therefore evolution of research practices within their RCTs.

    Box 2

    What is already known on this topic

    • There is an increased awareness of the need for patient/public involvement in research

    • There is an expectation from funders of research that patient and public involvement (PPI) should be incorporated within that research

    • There is selective reporting of PPI within the published literature.

    What this study adds

    • A systematic analysis of PPI in a cohort of randomised controlled trials receiving funding from the National Institute of Health Research Health Technology Assessment programme between 2006 and 2010

    • Only half of applications consider PPI within the early stages of the development of the research, with only a quarter having involvement within the development of the outline application itself

    • There is evidence that particular conditions, and design considerations, impact on whether PPI is likely to be considered within the early stages of development

    • There is insufficient consideration of PPI at the early stages by funding Boards.

    There were some indications that planned PPI contributions would increase as the application progressed from outline to full application through to funded trial. However, the proportion of applications describing PPI did not appear to increase across the years of the cohort.

    Strengths and weaknesses of the study

    This study used a cohort of trials funded by the NIHR HTA programme. To be eligible for inclusion within the cohort, applicants were required to have received funding for the trial between 2006 and 2010. Documentation could not be made available for unsuccessful applications. Consequently this study does not link details of PPI within the application to the successful award of the grant. However, while the HTA Programme encouraged PPI during this period, via its guidance notes to applicants and web information, and allowed it to be budgeted for, PPI was not mandatory within the period of the cohort, and that on its own would not have led to a decision that the outline application should not progress. As the cohort was identified by receipt of funding during the period rather than on the year that the outline application was submitted for consideration, fewer trials were available in the extremes of the cohort, but this approach was a consequence of the archiving system. The key limitation is the historical nature of this cohort. The period of the cohort should be considered when generalising the findings to the present day, due to revisions to the guidance for applicants and application forms in relation to PPI. From 2012 HTA boards have started to include PPI membership, and the standard NIHR application form has been introduced, with revised sections requiring applicants to clearly define their PPI involvement with guidance notes to applicants clearly stating that there is now an ‘expectation’ for ‘active involvement’ of patients and the public in the research it supports (http://www.nets.nihr.ac.uk/__data/assets/pdf_file/0005/67685/OutlineGuidanceNotesSAF.pdf, last accessed 28 April 2014). In addition, attitudes towards PPI may have changed and this may influence how early researchers engage with PPI contributors. However, further research is needed to consider this and its potential for impact.

    It has been argued that researchers use a tokenistic approach to patient involvement and the level of detail on PPI plans within this cohort may lend some strength to that argument. In order to summarise the details of PPI across the cohort some means of categorising descriptions of PPI activity was necessary. The lack of detail provided within the applications required clear coding rules to be established; however, these were clearly documented. Establishing coding rules was considered reasonable, as within the peer review of an outline application similar assumptions would need to be made by reviewers considering the PPI descriptions provided.

    The results should be interpreted with some caution, owing to the small numbers within some categories, and this caution applies to the results of the statistical tests, which were not powered to detect differences and should be considered as exploratory.

    Strengths and weaknesses in relation to other studies

    There are many examples or accounts of PPI within the published literature focusing on particular trials or topic areas9–12 and concerns have been raised about selective reporting.1 This is the first study to systematically investigate PPI early in the development of funding applications and across a wide range of trials. A previous paper on PPI within clinical trial units13 (CTUs) concluded that involvement appears to be growing. While the majority of applications within the current cohort included collaborations with CTUs, we found no clear trend of PPI activity increasing within this time. A standard operating procedure (SOP) for PPI in CTUs has been recently developed; however, its focus is on PPI after funding has been awarded.14 The SOP suggests that 1% of the research budget be allocated to PPI; however, other more detailed costing models are available.15

    If PPI is accepted as the best practice with early involvement, as it provides the greatest opportunity to maximise its contribution, then implementation of PPI and its descriptions within outline applications need to be increased and improved with justifications and methods for the approaches taken. It is noteworthy that no application within the cohort examined provided this level of information. The NIHR HTA programme has made changes to the outline application form, which now requests details of PPI and justification if it is absent. This may lead to increased descriptions but may not be associated with better involvement: a view which has some support from the current guidance, which states, “Whilst patient and public involvement (PPI) may not always be needed for all types of research, it is always relevant for HTA trials. Many PPI sections on the application are unconvincing to our consumer referees and Board members.” (http://www.nets.nihr.ac.uk/funding/hta-researcher-led/Preparing-a-full-application-for-the-Clinical-Trials-and-Evaluation-Board.pdf, last accessed 20 November 2013).

    Descriptions should specify how PPI has been incorporated within the development of the outline application and the plans for involvement should the application progress. However, the research community needs to consider the challenges facing researchers in developing PPI at the early design stage. In particular, the often short timeframes within which outline applications have to be developed and PPI contributors identified, and the lack of funding to support this work prior to the grant award, are major obstacles to enhancing PPI at an early stage in trial development. In England, a PPI bursary scheme has recently been launched by some of the NIHR Research Design Service (RDS), but is limited to those in receipt of advice from the RDS and has been shown to be beneficial.16 ,17 There are few other alternatives to funding development activity at the pre-award stage.15

    Although funding body policies support PPI, this support was not usually evident in the HTA Board feedback to applicants at the outline stage. This may be due to the difficulties in assessing PPI given the lack of detail provided within applications, or indicate that at the outline stage the weight of the decision about progression is on the perceived relevance of the question and design. When feedback about PPI was given this did not provide any guidance on how it should be addressed. Statements about the need to improve PPI should be supported with guidance on what PPI contributions would be appropriate within the trial being considered. Peer reviewers and Board members who are asked to comment on PPI should be supported in doing so. Adoption of critical appraisal guidelines may be beneficial in achieving this.18 From 2012 NIHR HTA funding Boards include a PPI representative. The impact of this on the frequency of comments and their content is not yet known.

    Our findings also raise questions about what sort of experience qualifies (or disqualifies) individuals to be PPI contributors, as identified within the comments from the Board. Regardless of whether they have relevant personal experience, should individuals with professional clinical or research experience become PPI contributors? Does an individual's experience in a professional capacity somehow weaken their ability to adopt a patient or carer perspective, or does a professional background strengthen a person's ability to contribute to the research? Current NIHR HTA guidance states that “To achieve the aim of bringing fresh eyes to the work of the HTA programme a patient or member of the public should not normally be a health practitioner, manager or researcher” (http://www.hta.ac.uk/PPIguidance/section1_2.shtml, last accessed 26 September 2013).

    There is also confusion about whether it is acceptable for individuals who have a PPI role in a professional capacity as well as relevant personal experience to provide PPI. Concerns have been previously raised about PPI representatives becoming professionalised,19 and evidence is needed on whether this is a legitimate concern. This should be rationalised against the fact that PPI representatives are generally individuals who are selected by researchers based on their attributes, and their ability to ‘represent’ the wider population of interest may be questioned, however, this may not impact their ability to communicate their unique ‘lay’ perspective.20

    Future research

    The lack of detail on PPI at the early stages of research development needs to be addressed. There is a keen interest in determining whether the perceived benefits of PPI lead to tangible benefits for research. The activities of PPI contributors need to be tracked to provide insights on when and what type of involvement may be linked to impact. Further research tracing the evolution of PPI within this cohort is being undertaken.

    Acknowledgments

    The authors would like to acknowledge the help and support of the NIHR HTA in establishing the cohort documentation.

    References

    1. ↵
      1. Staley K
      . Exploring impact: public involvement in NHS, public health and social care research. Eastleigh: INVOLVE, 2009.
    2. ↵
      1. Boote J,
      2. Baird W,
      3. Sutton A
      . Public involvement in the design and conduct of clinical trials: a review. Int J Interdiscip Soc Sci 2011;5:91–111.
      OpenUrl
    3. ↵
      Department of Health. Best research for best health: a new national health research strategy. London: Department of Health, 2006.
    4. ↵
      1. Robinson L,
      2. Newton J,
      3. Dawson P
      . Professionals and the public: power or partnership in health research? J Eval Clin Pract 2010;18:276–82.
      OpenUrl
    5. ↵
      1. Staniszewska S,
      2. Jones N,
      3. Newburn M,
      4. et al
      . User involvement in the development of a research bid: barriers, enablers and impacts. Heal Expect 2007;10:173–83.
      OpenUrl
    6. ↵
      1. Fudge N,
      2. Wolfe R,
      3. McKevitt R
      . Involving older people in health research. Age Ageing 2007;36:492–500.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Bastian H
      . Consumer and researcher collaboration in trials: filling the gaps. Clin Trials 2005;2:3–4.
      OpenUrlFREE Full Text
    8. ↵
      1. Kaur G,
      2. Smyth R,
      3. Williamson P
      . Developing a survey of barriers and facilitators to recruitment in randomized controlled trials. Trials 2012;13:218.
      OpenUrlCrossRefPubMed
    9. ↵
      1. Marshman Z,
      2. Innes N,
      3. Deery R,
      4. et al
      . The management of dental caries in primary teeth—involving service providers and users in the design of a trial. Trials 2012;13:143.
      OpenUrlPubMed
    10. ↵
      1. Boote J,
      2. Baird W,
      3. Beecroft R
      . Public involvement at the design stage of primary health research: a narrative review of case examples. Health Policy 2010;95:10–23.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Forbes LJL,
      2. Nicholls CM,
      3. Linsell L,
      4. et al
      . Involving users in the design of a randomised controlled trial of an intervention to promote early presentation in breast cancer: qualitative study. BMC Med Res Methodol 2010;10:110.
      OpenUrlCrossRefPubMed
    12. ↵
      1. Smith E,
      2. Ross F,
      3. Donovan S,
      4. et al
      . Service user involvement in nursing, midwifery and health visiting research: a review of evidence and practice. Int J Nurs Stud 2008;45:298–315.
      OpenUrlCrossRefPubMed
    13. ↵
      1. Hanley B,
      2. Truesdale A,
      3. King A,
      4. et al
      . Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey. BMJ 2001;322:519–23.
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Evans B,
      2. Bedson E,
      3. Bell P,
      4. et al
      . Involving service users in trials: developing a standard operating procedure. Trials 2013;14:219.
      OpenUrl
    15. ↵
      Mental Health Research Network and INVOLVE. Budgeting for involvement: Practical advice on budgeting for actively involving the public in research studies. London: Mental Health Research Network, INVOLVE: Eastleigh, 2013.
    16. ↵
      Walker D-M, Pandya-Wood R. Can research development bursaries for patient and public involvement have a positive impact on grant applications? A UK-based, small-scale service evaluation. Health Expect 2013. Early View (online version). doi: 10.1111/hex.12127.
    17. ↵
      Boote J, Twiddy M, Baird W, et al. Supporting public involvement in research design and grant development: a case study of a public involvement award scheme managed by a National Institute for Health Research (NIHR) Research Design Service (RDS). Health Expect 2013. Early View (online version). doi: 10.1111/hex.12130.
    18. ↵
      1. Wright D,
      2. Foster R,
      3. Amir Z,
      4. et al
      . Critical appraisal guidelines for assessing the quality and impact of user involvement in research. Health Expect 2010;13:359–68.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Ives J,
      2. Damery S,
      3. Redwod S
      . PPI, paradoxes and Plato: who's sailing the ship? J Med Ethics 2013;39:181–5.
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Staley K
      . There is no paradox with PPI in research. J Med Ethics 2013;39:186–7.
      OpenUrlFREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors CG conceived the idea for the research and led the development of the grant application and project. CG developed the data extraction tool, developed the database, extracted data, analysed data and wrote the manuscript. LD was involved in the development of the data extraction tool, completed data extraction and co-wrote the manuscript. PW was contributed to the project specification, commented on the data extraction items and provided comments on the manuscript. BY contributed to the project specification, commented on the data extraction items and provided comments on the manuscript. JP contributed to the project specification, commented on the data extraction items, extracted data, commented on the manuscript and co-led the coordination of the patient/public involvement advisory group. BH contributed to the project specification, commented on the data extraction items and on the manuscript and co-led the coordination of the patient/public involvement advisory group. AA, AW, HG and PB formed the patient and public advisory group. All members commented on the data extraction items, and contributed to the manuscript.

    • Funding This project was funded by INVOLVE and managed by the National Institute for Health Research (HS&DR) (project number 10/2001/29).

    • Competing interests CG, PW and BY have received research grants from the National Institute of Health Research, BH has received personal fees from TwoCan Associates, Medical Research Council Clinical Trials Unit and becandderek.co.uk, no other relationships or activities that could appear to have influenced the submitted work.

    • Ethics approval Ethical approval was sought and provided by the University of Liverpool Institutional Ethics Board. The University of Liverpool was the sponsor of the project.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.