Table 2

Study design and planning strategies to minimise the problem of missing PRO data


DesignTopicSpecific recommendationN recommendations*Potential drawbacksSource/s: first author (year). Full citations are provided as Online Supplementary Appendix C
Assessment scheduleSpecify PRO assessment time pointsSpecify the required PRO assessment time points2NoneBernhard, Gusset (1998), Beitz (1996)
Specify the minimum PRO data requirements (eg, ‘baseline, on and off treatment, and and/or end of study’ (ref. 5, p. 524)3May create impression that additional PRO assessments are not importantBernhard, Cella (1998)
PRO assessment schedule if treatment schedule is disrupted (ie, will the PRO assessment schedule be altered if the treatment schedule is altered?)1NoneFairclough (2010)
 Time point selection (guidance on how to select PRO assessment time points)Align PRO assessments to clinic visits so that data may be captured while the patient visits the clinic16Clinic visits may not be most informative to capture particular treatment effects (eg, chemotherapy toxicity)Bernhard, Cella (1998), Moinpour (1998), Movsas (2003), Aaronson (1990), Land (2007), Walker (2003), Calvert (2004), Sprague (2003), Revicki (2005), Fairclough (2010), Kyte (2013), Blazeby (2003), Simes (1998)
May be burdensome to participants to attend clinic for regular assessments
Align assessment schedule to a fixed reference point (for ease of calculating when PRO assessments are due)1May be burdensome to participants to attend regular assessmentsBernhard, Cella (1998)
Allow sufficient breaks between PRO assessments1May not be feasible if investigators wish to capture acute disease/treatment effects or their frequency via PROsSherman (2005)
Assess PROs of palliative care patients weekly4Does not consider when PRO assessment would be most meaningfulTang (2002)
Balance the number of required PRO assessments (not too few, not too many)4NoneRevicki (2005), Fairclough (2010)
Consider patient treatment and expected survival when planning assessment schedule (added note: avoid PRO assessments beyond the point of expected median survival)3NoneKaasa (2002), Hahn (1998), Atherton (2006)
Select clinically meaningful time points (ie, ensure that PRO assessments will be taken at clinically informative times, ie, to capture the trajectory of treatment and recovery)4Clinically meaningful PRO assessment time points may not align with clinic visits, which may require alternative modes of administrationGanz (2007), Jordhoy (1999), Tang (2002)
Event-driven PRO assessment for a subsample (ie, rather than subjecting entire sample to detailed PRO assessments if they experience certain clinical events, it may minimise staff effort and resources to restrict these additional assessments to a subsample only)2Event-driven PRO assessment can be logistically challenging to implementBernhard, Cella (1998), Simes (1998)
Focus on short-term outcomes in patients with advanced disease (focusing on long-term outcomes in such samples will lead to high rates of missing PRO data, and uninformative data)1May not be clinically meaningful to assess short-term outcomes in all studiesGanz (2007)
Justify chosen PRO assessment time points1NoneGanz (2007)
Minimise PRO assessment time points (select fewer time points to minimise burden and resource usage)3May sacrifice important information by omitting time points, for example, differences between treatment arms5Bernhard, Cella (1998), Macefield (2013), Cella (1995)
Shorter follow-up duration (avoid following up patients for a longer period of time as participants are more likely to drop out over time)1May sacrifice important information by ceasing PRO assessment too early in some studies. Some studies may be interested in long-term follow-up/survival outcomes.Little, Cohen (2012)
Treatment failure/cessationContinue PRO assessments after treatment failure6May be difficult to engage or contact participants beyond point of treatment failureHao (2010), Little, D'Agnostino (2012), Sprangers (2002), Chassany (2002), Cella (1995), Cella (1994)
Specify procedures for contacting participants after treatment cessation3NoneCella (1994), Revicki, Hao (2010)
Specify the PRO assessment stopping rule (ie, under what circumstances should PRO assessments discontinue)3NoneBell (2014), Kaasa (1992), Young, de Haes (1999)
Time windowsDefine PRO assessment time windows (ie, baseline assessment time window should always end before the intervention/treatment commences. Follow-up assessment time windows should border the period in which treatment effects of interest are anticipated, for example, if the time point is 1 week postsurgery, a valid assessment may occur anytime between 4 and 12 days postsurgery).12NoneBernhard, Cella (1998), Cella (1994), Wisniewski (2006), Blazeby (2003), Hopwood (1996), Bernhard, Peterson (1998), Fayers (1997), Hopwood (1998), Revicki (2005), Fairclough (2010), Cella (1995)
Flexible/large time windows (very narrow time windows may be logistically infeasible to implement and so risk of missing PRO data may be reduced by setting larger time windows)3Not all time windows can be flexible, particularly when assessing acute effects of treatmentBernhard, Cella (1998), Little, Cohen (2012), McMillan (2003)
Collect additional/supporting data (which can be used during PRO data analysis and interpretation)Auxiliary data (to assist interpretation if there are some missing PRO data). Suggestions of types of auxiliary data in the next columnAdditional information about non-responders (type of additional information unspecified)1Requires prespecification, and additional time and resources to collectKim (2004)
Clinical data1Requires additional time and resources to collectNewgard (2010)
Health status (clinician-rated quality of life index, Karnofsky or ECOG performance status)6Requires additional clinician timeCoates (1998), Bell (2014), Bernhard, Cella (1998), Simes (1998), Revicki (2005), Fairclough (2010)
Comorbidity data1Requires additional time and resources to collectBernhard, Cella (1998)
Concomitant medications1Requires additional time and resources to collectBeitz (1996)
Observation data1Requires additional time and resources to collectKaasa (2002)
Participant clinical data1NoneNewgard (2010)
Participant demographics2NoneAltman (2007), Newgard (2010)
Proxy† reports when participant is no longer able to self-complete21Proxy reports are not always concordant with participant self-reports. Care must be taken when interpreting proxy data. This is a specialist subject and additional reading is recommended for investigators considering to use proxy assessment.64Bernhard, Cella (1998), Chassany (2002), Fayers (1997), Jordhoy (2010), Kleinpell-Nowell (2000), Kyte (2013), Machin (1998), Moynihan (1998), Peruselli (1997), Revicki (2005), Rock (2007), Simes (1998), Sprangers (2002), Stewart (1992), Taphoorn (2010), Walker (2003)
Toxicity data2Requires additional time and resources to collect, if not already being collected as part of the studyFairclough (2010), Revicki (2005)
Unspecified (use an alternative to PRO in final weeks of life)1Requires additional time and resources to collect. Additional drawbacks may be apparent depending on specific alternative measure/s used.Jordhoy (2010)
Collect reasons for missing PRO dataSee ‘cover sheet’ section in administration procedures in table 3
Eligibility criteria for PRO study (suggestions of specific eligibility or inclusion criteria)Consider the participants' ability to complete PROsInclude—‘participant must be able to complete PROs’ as an inclusion criterion2Ability to complete PRO assessments may change over the course of treatment. Results may not be generalisable to all patients.Bernhard, Cella (1998), Huntington (2005)
Exclude patients with language/cognitive barriers from the PRO study only (ie, these participants are able to take part in other aspects of the trial, but will not be included in the PRO study)2May reduce the sample size/power of PRO study. Results may not be generalisable to all patients.Hopwood (1998), Sprague (2003)
Baseline PRO completion (some sources recommended include baseline PRO completion as an eligibility criterion)29NoneBernhard, Cella (1998), Bernhard, Peterson (1998), Calvert (2004), Cella (1994), Cella (1995), Chassany (2002), Conroy (2003), Fayers (1997), Hayden (1993), Hopwood (1998), Hurny (1992), Kaasa (1998), Movsas (2003), Osoba (1992), Osoba (2007), Sadura (1992), Simes (1998), Sprangers (2002), Walker (2003), Young, Maher (1999), Young de Haes (1999)
 Include patients with minimal level of impairment (as per baseline PRO) to ensure inclusion of patients with severe disease 1May lead to selection biasChassany (2002)
May impact generalisability of results
Surviving long enough to complete PROs (palliative care)3Difficult to estimate in some cases, so prognostic cues predictive of death may be more practical; may introduce selection bias.Bakitas (2009), Jordhoy (1999), Chassany (2002)
Participants’ willingness to complete PROs3May result in selection bias; patients more willing to take part in PRO study may differ systematically from non-participants.Fayers (1997), Sprague (2003)
Feasibility issues of PRO studiesPilot studyDetermine feasibility of PRO study (potential issues, resources required and/or sample size), and acceptability by conducting a pilot study9Requires time and resourcesCella (1994), Cella (1995), Groenvold (1999), Hurny (1992), Moinpour (1989), Kleinpell-Nowell (2000), Young, de Haes (1999), Sherman (2005), Wisniewski (2006)
Determine compliance targets by conducting a pilot study1Requires a long pilot study to determine; significant time and resourcesHahn (1998)
Conduct a pilot study to determine average time to complete PRO measures1Requires time and resourcesKleinpell-Nowell (2000)
Use the PRO pilot study as a training opportunity for less experienced staff1Requires time and resourcesCella (1995)
PRO resourcesEnsure there is sufficient funding for the PRO study and that the PRO study is included in study budget5Funding can be difficult to obtain; however, it is possible to minimise costs of PRO studies at no cost to high-quality PRO researchBernhard, Cella (1998), Cella (1995), Coates (1998), Gotay (2005), Moynihan (1998)
Resource allocation—ensure recruiting sites are sufficiently resourced for the PRO study15Funding can be difficult to obtain across all sites especially if recruiting internationally or trans-nationally.Bernhard, Cella (1998), Bernhard, Peterson (1998), Hayden (1993), Hopwood (1998), Hopwood (1996), Kaasa (1992), Moinpour (1998), Moynihan (1998), Revicki (2005), Scott (2004), Sprague (2003), Walker (2003), Wisniewski (2006), Young, de Haes (1999)
Ensure adequate staff at potential sites2Funding to employ new staff can be difficult to obtainRevicki (2005), Scott (2004)
Minimise resources required for the PRO study4Care must be taken not to sacrifice quality of data or performanceBernhard, Cella (1998), McMillan (2003)
Selection of recruiting sitesSelect sites with good compliance record1May limit the number of participants recruited; may overly burden particular sites; potential for selection bias65Bernhard, Cella (1998)
  Select sites with adequate resources2May limit the number of participants recruitedHurny (1992)
Sites with adequate resources may not necessarily be sites with best compliance record.
Provide PRO-specific guidance for the research teamPRO administration guidance (for site staff)General administration guidance aiming to standardise administration of PROs27NoneBernhard, Peterson (1998), Calvert (2004), Cella (1994), Cella (1995), Fayers (1997), Friedman (1998), Ganz (1988), Hahn (1998), Hayden (1993), Hopwood (1998), Kaasa (1998), Kaasa (1992), Land (2007), Newgard (2010), Osoba (1996), Osoba (1992), Sprangers (2002), Taphoorn (2010), Vantongelen (1989), Walker (2003), Wisniewski (2006)
Flexible processes across sites (There may be local variations in who is responsible for PRO data collection at different sites; therefore, procedures should be flexible to accommodate such differences.)1May introduce bias if procedures differ too much between recruiting sitesBernhard, Peterson (1998)
Importance of complete data must be stressed in PRO administration guidance1NoneFayers (1997)
Instructions to give to participants must be specified in PRO administration guidance1NoneWisniewski (2006)
Procedures for missed assessments must be specified in PRO administration guidance1NoneCalvert (2004)
Staff roles must be specified in PRO administration guidance3NonePoy (1993), Young de Haes (1999)
Procedures for handling special situations must be specified in PRO administration guidance5Not all difficult situations can be predicted in advanceHahn (1998), Hopwood (1998), Hopwood (1996), Revicki (2005)
Protocol guidance60 61 63Follow PRO protocol guidance (investigators)2NoneBernhard, Cella (1998), Osoba (2007)
Develop protocol guidance for investigators (trials groups)1NoneOsoba (1996)
MOA (ie, is the questionnaire administered in hardcopy (pen and paper), electronically, over the phone, etc)Choice of MOAConsider costs involved with each MOA1NoneMacefield (2013)
Consider impact of MOA on participants’ willingness to disclose information1The most acceptable MOA for participants may not be the most cost-effective or feasibleHallum-Montes (2014)
Consider potential impact MOA on response rate3NoneHallum-Montes (2014), Cantrell (2007)
Consider inclusion of remote participants (web-based modes may be more accommodating to remote patients than face-to-face administration)1NoneCantrell (2007)
Mode preferred by sample1Requires additional pilot work to gauge participant preferences. Requires additional staff time and costs. Need to ensure equivalence of modes66Basch (2012)
Electronic modes of administration ‘e-PROs’, for example, using a computer, tablet, smart phone, etcAllow participants to complete on their preferred electronic device1Requires resources to ensure compatibility of database across many types of electronic devicesJansen (2013)
Allows real-time compliance monitoring1NoneBasch (2012)
Avoid fancy layouts1NoneCantrell (2007)
Avoid mandating completion of all items2May lead to missing item-level data if questions are of sensitive nature67Cantrell (2007), Hanscom (2002)
Present items one at a time1May be burdensome for participants considering cumulative time required to click between screensHanscom (2002)
Avoid question presentation one at a time (to reduce response burden)2NoneCantrell (2007), Hallum-Montes (2014)
Dialogue boxes for missed items1May be costly to developWisniewski (2006)
Electronic dictation of questions1May be costly to developHallum-Montes (2014)
Email PRO assessment reminders to participants1Requires time/resources to implementCantrell (2007)
  e-PROs encouraged1e-PRO assessment may not be acceptable to some patient populations.Basch (2012)
May be subject to technical fault/data protection/connectivity issues
Keep assessment simple to reduce risk of technical fault1NoneHjermstad (2012)
Make all items mandatory1May lead to incomplete questionnaires if questions are of a sensitive natureCantrell (2007)
Flexible MOAFollow-up missed assessments with alternate mode (eg, if participant misses a face-to-face visit in which hardcopy PRO assessment was scheduled, consider calling the participant to complete PRO over the phone, or posting the questionnaire to their home address with reply-paid envelope to return completed questionnaire)4Requires additional staff time and resourcesBernhard, Cella (1998), Blazeby (2003)
Interview-administered questionnaires for very sick participants4Requires additional staff timeKaasa (1998), Stewart (1992), Moynihan (1998), Chassany (2002)
Offer more than one MOA2May complicate data entry procedures or procedures for returning PRO dataBernhard, Cella (1998), Gotay (2005)
Negotiate with the site as to their preferred MOA1May be infeasible to implement different modes between sites—some sites may have to compromiseSimes (1998)
Interview-administered MOAInterview-administered MOA may improve response rates.1Requires additional staff time and resourcesFowler (1996)
Postal MOAComplete the baseline assessment in clinic and subsequent assessments by post1NoneKaasa (1998)
Include postage-paid, self-addressed envelope for easy return of completed questionnaires (when using postal MOA)3Requires additional staff time and postage costs. May be burdensome for participants to send questionnaires back to researchers.Kleinpell-Nowell (2000), Poulter (1997)
Patient burden—minimiseMinimise patient burden (general statement)8NoneAaronson (1990), Hahn (1998), Little, D’Agostino (2012), Macefield (2013), McMillan (2003), Revicki (2005), Walker (2003)
Offer assistance to participants to complete PROs (to reduce burden PRO completion)Additional assistance—childcare (offer to provide child care for participants’ children so that participants can attend clinic visits in which PRO assessments are scheduled)1Requires additional resourcesBell (2014)
Additional assistance—travel (offer to arrange or fund travel of participants to the clinic for scheduled PRO assessments)1Requires additional resourcesBell (2014)
Avoid the need for a clinic visit where possible1May be difficult to engage participants away from the clinicLittle, Cohen (2012)
Offer assistance to complete questionnaire if needed1Requires additional staff time and resourcesSprague (2003)
ContentClear/simple content and instructions of questionnaires1NoneYoung, de Haes (1999)
Reduce overlap in questionnaire items3NoneFallowfield (1998), Walker (2003), Young, de Haes (1999)
Collect relevant PRO data only2NoneBernhard, Cella (1998), Little, Cohen (2012)
FormatAvoid using multiple questionnaires1NoneChassany (2002)
Avoid written (free text) answers1NoneFriedman (1998)
Clear/simple format6NoneConroy (2003), Little Cohen (2012), Kleinpell-Nowell (2000), Bernhard, Cella (1998), Revicki (2005), Sloan (2007)
Large/clear font1May increase printing costs if larger font adds pages to the questionnaire bookletFairclough (2010)
Professional format (eg, use study letterhead on printed questionnaires, use consistent formatting, etc)3NoneKleinpell-Nowell (2000), Revicki (2005), Sloan (2007)
Single-sided printing (some reports suggest that participants are more likely to overlook the underside of questionnaires printed double-sided)2Environmental burden. May increase printing costs due to additional pages in the questionnaire bookletFairclough (2010), Revicki (2005)
Uniform presentation format (a consistent formatting approach appears more professional and may be easier for participants to follow, potentially reducing risk of participants skipping items inadvertently or due to lack of understanding)2May not be possible if using more than one questionnaireBernhard, Peterson (1998), Hurny (1992)
Length of assessmentsConsider participant health—sicker participants will not be able to complete long PRO assessments3NoneMoinpour (1989), Stewart (1992), Young, de Haes (1999)
Fewer assessment time points (ie, PRO assessments that occur regularly may be overly burdensome)10May sacrifice important information by assessing PRO less oftenBernhard, Cella (1998), Little, Cohen (2012), Chassany (2002), Ganz (1988), Jansen (2013), Revicki (2005), Fallowfield (1998), Hurny (1992), Hao (2010), Steinhauser (2006)
Fewer pages in e-PROs (eg, minimising the number of clicks between pages may reduce burden)1NoneCantrell (2007)
Shorter questionnaire18Limits the amount of information that can be assessed using PROsBasch (2012), Basch (2014), Bell (2014), Bernhard, Cella (1998), Bernhard, Peterson (1998), Chassany (2002), Fairclough (2010), Hjermstad (2012), Hurny (1992), Moinpour (1989), Revicki (2005), Rock (2007), Sadura (1992), Siddiqui (2014), Young, de Haes (1999)
Use CAT/screening questions (allows for targeted question content and fewer items, to minimise burden)1Requires additional set-up costs. Can be difficult to introduce a second, non-electronic MOA if using CAT as questions administered will differ between participantsHjermstad (2012)
Use validated questionnairesQuestionnaire items or formatting that participants find burdensome may be addressed in response to feedback obtained during questionnaire validation process1NoneKaasa (1992)
Participant education and engagement (also see table 3)Continued participant engagement—use strategies to keep participants engaged throughout the life of the study/trialAdapt procedures to participant cultural group—conduct background research about the cultural groups involved2Requires time and resourcesWilcox (2001)
Participant incentives for participating/completing PRO questionnairesOffer participants access to care via/after trial/study3Requires time and resourcesBlazeby (2003), Little, Cohen (2012), Little D'Agnostino (2012)
Offer participants financial incentives13Requires time and resources. Conflicting evidence about the effectiveness (in general population samples)68 and ethical issues in patient populationsDykema (2012), Gates (2009), Jansen (2013), Kleinpell-Nowell (2000), Little, Cohen (2012), Meyers (2003), Sherman (2005)
Offer participants non-financial incentives8Requires time and resources. Conflicting evidence about the effectiveness (in general population samples)68 and ethical issues in patient populationsDykema (2012), Little, Cohen (2012), Sherman (2005), Hellard (2001)
Reimburse participants for their time/costs involved in participating (factor into study budget)3Requires time and resourcesHellard (2001), Little, Cohen (2012), Senturia (1998)
Selecting a PRO measureAcceptable measures for participants5NoneChassany (2002), Jordhoy (2010), Kaasa (1992), Revicki (2005)
Clinically relevant measures (select PRO measures that are clinically appropriate, that is, include questions about relevant issues to specific disease/treatment)7NoneBernhard, Cella (1998), Friedman (1998), Ganz (2007), Gheorghe (2014), Hahn (1998), Revicki (2005)
Features to avoid in prospective PRO measuresAvoid overlapping content/highly correlated items2NoneBeitz (1996), Taphoorn (2010)
Avoid sensitive item content (ie, participants are more likely to skip items addressing sensitive issues such as sexuality or finances; so by avoiding such items you may minimise risk of missing PRO data)4Participants may have different views about what constitutes sensitive data. Some key issues for particular studies are considered sensitive, for example, sexual functionFallowfield (1998), Jansen (2013), Pijls-Johannesma (2005), Simes (1998)
Translated (validated) questionnaires2Complicates trial set up and implementation, particularly when using e-PROsKaasa (1998), Kleinpell-Nowell (2000)
Validated measures (these are likely to be more clinically relevant and acceptable to patients)6NoneBernhard, Cella (1998), Blazeby (2003), Fallowfield (1998), Kaasa (1992), Siddiqui (2014)
OtherOrdering questionnaire items chronologically may speed up completion time and be easier for patients to complete1We strongly recommend that researchers do not change the item order of validated questionnaires. Questionnaires should be administered in the exact format as validated.Dunn (2003)
Strategies for measuring sensitive issues (please see Chassany 2002 for a description of various strategies)1NoneChassany (2002)
PROs part of trial/larger studyResearch team should commit to the PRO substudy (eg, when part of larger trial)11Requires time and resourcesBernhard, Cella (1998), Bernhard, Peterson (1998), Cella (1994), Cella (1995), Chassany (2002), Hayden (1993), Kiebert (1998), Moynihan (1998)
Incorporate PROs in trial/main study designPROs should be a mandatory/integral part of the trial/ larger study (ie, PRO data are not an optional extra)10NoneAaronson (1990), Bernhard, Cella (1998), Hayden (1993), Hurny (1992), Kaasa (1992), Movsas (2004), Osoba (2007), Sadura (1992), Siddiqui (2014), Young, de Haes (1999)
Consider logistic factors when designing PRO study4NoneChassany (2002), Little, D’Agostino (2012), Wisniewski (2006), Young, de Haes (1999)
PRO content in the study protocol60 61 63Define end points/hypotheses (ensure PRO end point is scientifically compelling)5NoneCella (1994), Fallowfield (2005), Little, Cohen (2012), Taphoorn (2010), Walker (2003)
Specify how missing data will be handled1May not be possible to fully plan how missing data will be handled prospectivelyCalvert (2004)
Specify the importance of PRO assessment compliance1NoneFayers (1997)
Include/plan PRO aspects of the study carefully13NoneBell (2014), Fayers (1997), Ganz (2007), Hahn (1998), Hao (2010), Land (2007), Moinpour (1998), Movsas (2003), Poy (1993), Revicki (2005), Sloan (2007),Walker (2003)
Specify plans for minimising missing data (such as those listed in this review) in the protocol11NoneBeitz (1996), BIQSFP (2012), Calvert (2004), Fairclough (2010), Kaasa (1998), Moinpour (1998), Revicki (2005), Simes (1998), Young, de Haes (1999)
Specify PRO assessment schedule2NoneHopwood (1996), Moinpour (1998)
Specify the rationale for PRO assessment (understanding why PROs are being measured and the value the information will bring to the trial is useful for all members of the trial team, and reinforces the importance of high-quality PRO data collection)11NoneAaronson (1990), Bell (2014), Cella (1994), Cella (1995), Conroy (2003), Fayers (1997), Hopwood (1998), Sadura (1992)
Include PROs in the SAP†Specify potential problems with PRO analysis in SAP2May not be possible to predict and prepare for all potential problems with PRO analysis when developing the SAPTaphoorn (2010), Walker (2003)
Plans for addressing missing data in SAP2May not be possible to fully plan how missing data will be handled prospectivelyBell (2014), Bernhard, Peterson (1998)
PROs in other trial/study documentsInclude PRO study in relevant sections of procedural documents1NoneLand (2007)
QAQA—planning aheadConsider logistic factors when designing PRO study1NoneFallowfield (2005)
Create study databases with QA in mind (ie, consider how PRO data completion rates will be monitored using the database)5Requires time and resourcesBernhard, Cella (1998), Land (2007), Moinpour (1998), Wisniewski (2006)
Manage PROs with other trial/study end point data (ie, in a single database)2Data managers will require additional training for PROs—which requires additional time and resourcesBernhard, Peterson (1998), Hurny (1992)
Describe QA procedures in protocol3NoneCella (1995), Gheorghe (2014), Revicki (2005)
Specify QA procedures in a manual2NoneCella (1994),Cella (1995)
Establish target PRO compliance rates (ie, quotas that must be achieved, eg, a target of 95% indicates that no more than 5% of missing PRO questionnaires will be tolerated)6NoneHahn (1998), Little, Cohen (2012), Little, D’Agostino (2012), McMillan (2003), Sloan (2007)
Sample (for PRO data collection)PRO subsample (if study power permits and if the study budget or logistics limit capacity to collect PROs from all participants, consider collecting PROs from a subsample only)PRO data from representative subsample of the trial population2May be difficult administratively, particularly for site staff to implementSimes (1998)
Do not collect PROs from patients with advanced disease1QOL issues are often of very important in patients with advanced disease.Bernhard, Cella (1998)
  Allow patients/sites to opt in to the PRO study1May lead to selection bias if sites or participants opt-in to PRO studySimes (1998)
May also lead to impression that PRO study is of lesser importance than other study outcomes
Recruit motivated patients only2May lead to selection bias if only motivated participants take part in PRO studyBernhard, Cella (1998), Simes (1998)
Separate (additional) consent for PRO study1Requires additional time and resourcesSimes (1998)
Sample sizeIncrease sample size to allow for attrition7The rate of missing data is important, regardless of whether the available data meet sample size requirements. Although increasing sample size will improve study power in the case of low PRO completion rates, the outcomes of participants with missing PRO data may differ to those with complete PRO data—which may lead to bias.Altman (2007), Kaasa (2002), Little, D’Agostino (2012), Sherman (2005), Stewart (1992), Tang (2002), Jordhoy (2010)
Team—design/protocol developmentInvolve committees (to review PRO study)Ethics review1NoneMovsas (2003)
PRO committee (ie, some trials groups have a dedicated PRO committee, comprised of PRO research specialists who review and provide feedback on PRO aspects of trials)6Requires access to a trials group with resources for a PRO committeeHahn (1998), Osoba (1992), Osoba (2007), Revicki (2005)
Multidisciplinary team involved in design/protocol development (each party brings unique and complementary expertise and experiences to improve the design of the PRO study)Involve a multidisciplinary team in PRO study design6NoneBernhard, Cella (1998), Cella (1994), Cella (1995), Kiebert (1998), Moinpour (1998)
Involve experienced investigators in PRO study design (to offer strategies for maximising compliance, selection of informative measures and time points, and other key aspects of study design)2NoneLittle, Cohen (2012), Little, D’Agostino (2012)
Involve nurses in PRO study design (to offer expertise about patient experiences and relevant QOL issues, clinic environment, data collection, etc)1NoneHayden (1993)
Involve patients in PRO study design (to comment on the acceptability and relevance of PRO questionnaires, suitability of assessment time points in capturing desired outcomes, patient burden, strategies to educate and engage participants, and many other important aspect of study design)3NoneBernhard, Peterson (1998), Hurny (1992), Moynihan (1998)
Involve PRO experts in PRO study design (to offer strategies for maximising compliance, selection of informative measures and time points, analysis and interpretation strategies and other key aspects of study design)3NoneFallowfield (1998), Kiebert (1998), Basch (2014)
Involve site coordinators in PRO study design (to offer expertise about logistics of PRO assessment, patient experiences and relevant QOL issues, data collection strategies, etc)4NoneBernhard, Cella (1998), Hayden (1993), Larkin (2012), Moinpour (1998), Cella (1995)
Support the site staffMinimise institution/staff burden (an overly burdensome PRO assessment schedule or procedure for site staff is likely to lead to high rates of missing data)6NoneAaronson (1990), Young, de Haes (1999)
  • *Some sources may have provided a recommendation more than once.

  • †This review only covers proxy reporting as a strategy to facilitate interpretation of missing PRO data. If considering using proxies, please consult the literature for a review of additional challenges and implementation strategies.

  • CAT, computer-adaptive testing; ECOG, Eastern Cooperative Oncology Group; ePRO, PROs administered electronically; MOA, mode of administration; PRO, patient-reported outcome; QA, quality assurance; QOL, quality of life; SAP, statistical analysis plan.