You are here

  • Home
  • Archive
  • Volume 12, Issue 2
  • Patient engagement partnerships in clinical trials (PEP-CT): protocol for the systematic development and testing of patient partner and investigator decision aids

Article Text

Article menu
Patient-centred medicine
Protocol
Patient engagement partnerships in clinical trials (PEP-CT): protocol for the systematic development and testing of patient partner and investigator decision aids
  1. Monica Parry1,
  2. Tina Ceroni2,
  3. David Wells3,
  4. Dawn P Richards2,
  5. Karine Toupin-April4,5,6,
  6. Hafsa Ansari1,
  7. Ann Kristin Bjørnnes7,
  8. Heather Burnside1,
  9. Sabrina Cavallo8,
  10. Andrew Day9,
  11. Anne Ellis10,
  12. Debbie Feldman8,
  13. Ian Gilron11,
  14. Adhiyat Najam3,
  15. Zoya Zulfiqar1,
  16. Susan Marlin2
  1. 1Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Ontario, Canada
  2. 2Clinical Trials Ontario, Toronto, Ontario, Canada
  3. 3Diabetes Action Canada, Toronto, Ontario, Canada
  4. 4School of Rehabilitation Sciences, University of Ottawa, Ottawa, Ontario, Canada
  5. 5Research Institute, Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada
  6. 6Institut du Savoir Montfort, Ottawa, Ontario, Canada
  7. 7Department of Nursing and Health Promotion, Oslo Metropolitan University, Oslo, Norway
  8. 8School of Rehabilitation, Faculty of Medicine, Université de Montréal, Montreal, Quebec, Canada
  9. 9Department of Community Health and Epidemiology and CERU, Queen’s Unversity, Kingston, Ontario, Canada
  10. 10Department of Medicine, Queen’s University, Kingston, Ontario, Canada
  11. 11Departments of Anesthesiology and Perioperative Medicine, Biomedical and Molecular Sciences and School of Policy Studies, Queen’s University, Kingston, Ontario, Canada
  1. Correspondence to Dr Monica Parry; monica.parry{at}utoronto.ca

Abstract

Introduction Building capacity to improve sex/gender knowledge and strengthen patient engagement in clinical trials requires training and support. The overall goal of this 2-year project is to refine, translate and evaluate two web-based open-access patient and investigator decision aids aimed to improve patient engagement partnerships in clinical trials.

Methods and analysis Two decision aids were designed in Phase 1 of this programme of research and this protocol describes a subsequent sequential phased approach to refine/translate (Phase 2A) and conduct alpha/usability (Phase 2B) and beta/field (Phase 3) testing. Decision aid development is guided by the International Patient Decision Aid Standards, User-Centred Design, Ottawa Decision-Support Framework and the Ottawa Model of Research Use. We have integrated patient-oriented research methods by engaging patient partners across all phases of our programme of research. Decision aids will first be refined and then translated to French (Phase 2A). Eight iterative cycles of semistructured interviews with 40 participants (20 patient partners and 20 investigators) will be conducted to determine usability (Phase 2B). A pragmatic pre/post pilot study design will then be implemented for field/beta testing using another purposive sample of 80 English-speaking and French-speaking participants (40 patients and 40 investigators). The samples are purposive to ensure an equal representation of English-speaking and French-speaking participants and an equal representation of men and women. Since sex and/or gender differences in utilisation and effectiveness of decision aids have not been previously reported, Phase 3 outcomes will be reported for the total sample and separately for men and women.

Ethics and dissemination Ethics approval has been granted from the University of Toronto (41109, 28 September 2021). Informed consent will be obtained from participants. Dissemination will include co-authored publications, conference presentations, educational national public forums, fact sheets/newsletters, social media sharing and videos/webinars.

  • information technology
  • quality in health care
  • health informatics
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

http://dx.doi.org/10.1136/bmjopen-2021-060267

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Purposive sampling to assist in recruiting participants of English-speaking and French-speaking patients and investigators who are representative of men and women from academic and community settings within urban and rural locations in Canada.

    • Reliable and valid measures to assess acceptability and the core attributes of decision-making processes and decision choice to minimise information/measurement bias.

    • Although the Decisional Conflict Scale is most widely used, there is potential for increased bias in patients with limited reading skills.

    Introduction

    Patient-oriented research (POR) aims to actively engage patients and their caregivers throughout the research process to better facilitate the prioritisation and research of patient-identified problems. Patient engagement in research has mostly been limited to providing feedback, rather than more active collaborative processes to design and governance, data analysis and knowledge dissemination.1 2 Patient engagement in research is influenced by three core values: developing best practices, improving research impact and quality and building strong relationships.3 Building genuine partnerships guided by collaboration, honesty and trust are important for all stakeholders; including patient partners, healthcare providers, researchers, industry and policy makers.3 Recommendations for developing patient partnerships suggest that the following are essential in building capacity for POR: (1) positive researcher attitudes grounded in shared goals and strong communication practices, (2) supportive institutional policies, (3) values of trust, respect and co-learning, (4) tools/resources for effective patient engagement, (5) necessary training for team members and (6) value for patient partners in all stages of research.4

    A 2015 report commissioned by Clinical Trials Ontario provided information regarding patient engagement in clinical trials. One of the report’s recommendations was to offer resources/tools and best practices for patient engagement (eg, decision aids).5 Decision aids are interventions that increase knowledge surrounding expectations,6 advantages and disadvantages,6 and choices and outcomes.7 Users can develop skills in assessing uncertainties and highlight personal priorities using the advantages and disadvantages of participating7 8 in a particular decision.7 8 Research evidence supports the benefits of decision aids in increasing knowledge9–11 and improving expectations concerning priorities and choice.10 A 2017 Cochrane Review found that patient decision aids reduce decisional conflict related to feeling uninformed and decrease the proportion of individuals who are passive in decision-making.11 At a minimum, decision aids improve the: (1) quality of the decision-making processes, and (2) quality of the choice that is made.12 However, potential sex and/or gender differences in utilisation and effectiveness of decision aids were not reported. Core attributes of the quality of decision-making processes include: (1) recognition that a decision needs to be made, (2) being informed about options and benefits/risks, (3) value clarity, (4) discussion about goals/preferences and (5) involvement in treatment decision-making.12 The quality of choice is the extent to which end-users are informed and make decisions that reflect their goals and preferences.7 The decision aids developed in Phase 1 of this programme of research are not designed to assist patients in making treatment decisions; they are designed to assist patients and investigators in making decisions about engaging as/with patient partners in research.

    In addition to limited sex and/or gender differences in utilisation and effectiveness of decision aids, there has been limited sex and gender reporting in clinical trial research.13–15 There continues to be poor uptake of sex and gender in clinical trial research in Canada; of the trials published between January 2013 and July 2014, 6% (n=6) conducted a subgroup analysis of sex, 4% (n=4) reported sex-disaggregated data and none defined sex or gender or conducted a sex- or gender-based analysis.16 Many data collection instruments fail to incorporate variables associated with sex and gender, such as income and household responsibilities. The terms sex and gender also continue to be used interchangeably and incorrectly in research, suggesting a lack of understanding that these are distinct concepts.14 17

    Objectives

    The overall goal of this 2-year project is to refine, translate and evaluate two web-based open-access patient and investigator decision aids designed in Phase 1 of this programme of research (online supplemental file 1). These decision aids aim to improve sex and gender knowledge and patient engagement partnerships in clinical trials, which we refer to as ‘PEP-CT’. The objectives for the next phases (2A, 2B and 3) are as follows: (1) to refine content and functionalities of the decision aids and translate them to French as the second most commonly spoken language in Canada (Phase 2A), (2) to further refine the patient partner and investigator decision aids through usability testing (Phase 2B) and (3) to assess patient and investigator decisional conflict related to PEP-CT (Phase 3). Secondary objectives of Phase 3 are to evaluate: (1) sex/gender and POR knowledge, and (2) patient and investigator acceptability and engagement with the respective decision aid. Additional exploratory objectives of Phase 3 are to: (1) conduct a formative evaluation of the use of the decision aids to assess the predisposing, enabling and reinforcing factors18 that may impact the ability of each aid to support informed decision-making and (2) evaluate adoption and impact (eg, uptake by end-users) of each decision aid.

    Methods and analysis

    Decision aid functionalities were developed through a scoping review with input from health charity and patient organisations, research administrators and industry via a full-day POR consultation workshop (Phase 1A) and results have previously been published.19 The two decision aids were developed for initial dissemination and feedback via a New/Early Investigator Training Day and disseminated for feedback in our Building Capacity for Patient-Oriented Research (POR) in Clinical Trials, Translating the Evidence into Practice, Policy and Outcomes (POR STEPP) Digital Health Project in Ontario (Phase 1B). Knowledge gained from the POR STEPP Digital Health Project will be used to further refine the decision aids and translate them to French (Phase 2A). Alpha/usability testing and further refinements of the decision aids will then take place via iterative cycles (Phase 2B), followed by beta/field testing of the patient and investigator decision aids for large-scale implementation (Phase 3).

    Patient and public involvement

    The PEP-CT patient partners will continue to collaborate throughout Phases 2 and 3 of this programme of research. In Phase 1, patient partners informed research priorities and identified search terms. They extracted and collated data from the scoping review and co-presented at the consultation workshop, New/Early Investigator POR Training Day and through online webinars and a conference presentation. Patient partners co-led the development of the Patient Decision Aid and co-authored the Phase 1 manuscript.19 Patient partners helped identify priorities for Phases 2 and 3 and will collaborate in alpha and beta testing and knowledge dissemination of the revised Patient and Investigator Decision Aids.

    Phase 2A: refinement and translation

    Procedures

    Phase 2A focuses on refining and translating both the patient partner and investigator decision aids. Refinements identified in Phase 1 align with both the Integrate, Design, Assess and Share20 and the WHO21 frameworks for disseminating and scaling up innovations. General refinements will include increasing font size, incorporating more white space and visuals/videos and ensuring language/content reflects diversity in race and ethnicity. Racial and ethnic diversity will be demonstrated using acceptable literacy, visuals/videos that promote equity, diversity and inclusion, and by translating both decision aids to French. Further refinements include the addition of a glossary, hover-over text, bookmarks, hyperlinks to existing resources, adapting print-friendly sections and ensuring access to decision aids on all devices (ie, iPAD). Hyperlinks to existing organisational information/resources will be added preceding My Decision to better guide individuals to find a patient partner or a clinical trial project. Usefulness of the investigator decision aid for researchers already interested in POR and patient partner training (eg, screening, data extraction) will be highlighted. Integration of broader language to demonstrate the application of the patient and investigator decision aids to research projects broader than clinical trials will be incorporated. The patient decision aid text will be reviewed to ensure a grade 5–6 reading level.22 We are currently completing all refinements and then both decision aids will be translated to French.

    Phase 2B: alpha (usability) testing

    Study design

    A qualitative approach using semistructured, audiotaped interviews and user observation will be undertaken by a trained observer in iterative cycles to determine usability of the patient partner and investigator decision aids. The iterative rapid design in Phase 2B will focus on user performance (ease of use, efficiency, ease of learning and errors) and satisfaction with decision aid content and functionality (resources, web-based design).23

    Sample

    A single coordinating centre will recruit a purposive sample of 20 English-speaking and French-speaking patients (men and women) and 20 English-speaking and French-speaking investigators (men and women) through Clinical Trials Ontario, Network of Women, Canadian Cancer Clinical Trials Network, Canadian Arthritis Patient Alliance, Cystic Fibrosis Canada, Canadian Skin Patient Alliance, Brain Tumour Foundation of Canada, Huntington Society of Canada, Sickle Cell Awareness Group of Ontario, Myeloma Canada and the SPOR Support Units and Chronic Disease Networks. Based on previous experience24–26 and recommendations that usability testing by 3–5 users finds approximately 85% of interface usability problems,27 28 each usability cycle will include five end-users.

    Eligibility criteria

    Patient partners and investigators will be greater than 18 years of age and be fluent in either English or French. Access to a computer or another device with internet will also be mandatory.

    Study setting

    Participants will engage in one-on-one observation for 60–90 min via audio/video conferencing using ZOOM. Informed consent (online supplemental file 2) for participation and audiotaping will be done prior to the interview, along with completion of a Demographic and Clinical Information Form.

    Procedures

    Eight usability cycles in total are planned: 2 cycles of patients (English, men and women), 2 cycles of patients (French, men and women), 2 cycles of investigators (English, men and women) and 2 cycles of investigators (French, men and women). After completion of the first cycle for each group, changes will be made to the respective decision aid (ie, patient and investigator). The revised decision aids will then be evaluated in subsequent cycles. These iterations usually require 2–3 testing cycles with each end-user group until no further comments are identified.26 27 29 We will provide four testing cycles for each decision aid to accommodate for an equal number of men and women, and English-speaking and French-speaking end-users (figure 1). Travel reimbursement and compensation will be offered to participants based on time (ie, $50 for an estimated 2 hours to use each decision aid).

    Figure 1
    Figure 1

    Alpha and beta testing. PEP-CT, patient engagement partnerships in clinical trials; POR STEPP, Building Capacity for Patient-Oriented Research (POR) in Clinical Trials, Translating the Evidence into Practice, Policy and Outcomes.

    Each participant will be provided with a brief explanation of the decision aid in their language and then asked to move through the required features: (1) Introduction (get facts on POR/PEP-CT), (2) My Priorities (where in the research process patient partners can be engaged, including levels of engagement), (3) Learn More (to plan, engage and evaluate PEP-CT (including sex/gender)), (4) My Readiness (comparing priorities with perceived benefits/risks) and (5) My Decision (decision and next steps, such as finding a patient partner or finding a clinical trial). We will employ a ‘think aloud’ approach30 to gather insight into the way users move through the decision aids. Comments will be recorded, and the research coordinator will make field notes about any problems encountered on the Usability Testing Error and Efficiency Documentation Form. At the end of the session, participants will be asked to complete the System Usability Scale.31 32 Ten 5-point Likert questions will be scored to provide a point estimate of usability. In addition, four semistructured questions will be asked to determine users’ overall impression of the decision aid; what they liked and why, what could be improved, and if anything was missing.26

    Analysis

    Quantitative data from the Demographic and Clinical Information Form, Usability Testing Error and Efficiency Documentation Form (adapted for use in this study)33 and the System Usability Scale will be analysed using descriptive statistics in SAS V.9.2.34 The 10 5-point Likert questions scores will be analysed to provide a point estimate of usability with a reported reliability of 0.85.32 The semistructured interview audiotapes will be transcribed and translated to English. Transcribed data will be managed and imported into NVivo.35 Two members of the investigative team will use simple content analysis to obtain an understanding and develop codes after each iterative cycle. These codes will be used to generate themes. Disagreements about themes will be handled through consensus and a third member of the investigative team.36 37 Raw data will be revisited on a regular basis to ensure codes and resulting themes are grounded in the data.38 In addition, we will collect information on sex and gender to provide recommendations on any sex and gender differences in usability, which could inform Phase 3 of this project.

    Outcomes

    This iterative user-centred design and the semistructured interview questions will assist us to understand the proposed requirements of each decision aid and whether each is suited to both men and women end-users. Patients and investigators will be able to assess the appropriateness and ease of use of each decision aid prior to the pragmatic pre/post pilot study (Phase 3).

    Phase 3: beta (field) testing

    Study design

    Beta/field testing of the refined decision aids will be conducted using a pragmatic pre/post pilot study design. Field testing will be guided by the Ottawa Model of Research Use.39 The Ottawa Model of Research Use provides a framework for evaluating knowledge translation innovation implementation using six key elements: (1) change agents and resources, (2) evidence-based innovation, (3) environmental barriers and facilitators, (4) awareness and skills/training, (5) adoption and (6) impact (eg, uptake by end-users). In the context of the decision aids, the potential adopters include patient partners and investigators interested in clinical trial research.

    Sample

    We will recruit 40 English-speaking and French-speaking patients (20 men, 20 women) and 40 English-speaking and French-speaking investigators (20 men, 20 women). Participants will be recruited through the same process as Phase 2B. The sample size was chosen based on Hertzog’s recommendation40 of a minimum of 20–30 participants for single sample pre/post studies. In addition, based on Cochrane data41 and other recommendations,42 for a level of significance of alpha=0.05 (two-sided), power=0.80, a pre and post SD of 0.81 and a correlation between pre and/post scores of 0.80, a pre and post change of 0.34 in decisional conflict scores will be able to be detected with a sample size of 20 participants. This effect size can distinguish between being ready and not being ready to make a decision. However, in order to do a sex-based analysis with the same precision within each sex, 40 English-speaking and French-speaking patients (20 men and 20 women) and 40 English-speaking and French-speaking investigators (20 men and 20 women) will be recruited.

    Eligibility criteria

    Patients will be eligible if they have not participated in Phase 2B and have had no previous experience in participating as a patient partner on a research team. If patients have previously been a participant (ie, not a patient partner) in a clinical trial, they will be eligible to participate in this phase. Patient partners and investigators will be greater than 18 years of age. As the decision aids are web-based, access to a computer and/or other device with internet will be mandatory.

    Procedures

    Interested participants will contact the research coordinator by telephone or email to express their interest using a decision aid and participating in the study. Eligibility criteria will be confirmed, and informed consent obtained (online supplemental file 2). Participants will complete an online Demographic Form and baseline measures (ie, sex/gender and POR knowledge). Participants will then review either the patient or investigator web-based open-access decision aid. Engagement will be assessed using Google Analytics/Google Tag Manager (eg, event tracking and heatmap tools). Choice predisposition has been incorporated into the design features of each of the decision aids. After using their respective decision aid, patient partners and investigators will be asked to mark along a 5-point choice predisposition scale anchored by ‘engage’ or ‘not engage’ as a patient partner (patient decision aid) or with a patient partner (investigator decision aid). Response options in the centre indicate that the participant is ‘undecided’. Test–retest reliability of various iterations of the choice predisposition scale in various populations has exceeded 0.90, values and expectations have also been consistently correlated with choice predisposition.43 44 Decisional conflict will be assessed at post-test using the 16-item Decisional Conflict Scale,45 which measures personal perceptions of: (1) uncertainty in choosing options, (2) modifiable factors contributing to uncertainty (eg, feeling uninformed or unclear about priorities/values) and (3) effective decision-making (eg, feeling the choice is informed, values-based and satisfied with the choice). Five subscales (informed, values clarity, support, uncertainty and effective decision) contribute to a total score that ranges from 0 (no decisional conflict) to 100 (extremely high decision conflict). Knowledge of sex/gender and POR will be assessed at pretest and post-test using two separate Sex/Gender and Patient-Oriented Research Knowledge Scales, one for patients and one for investigators, developed for use in this study. Based on Cochrane data,11 the knowledge scale questions will be based on information contained in the decision aids. The proportion of correct responses will be converted to a percentage scale ranging from 0% (no correct responses) to 100% (all correct responses). Decision support acceptability will be assessed using a modified Acceptability E-Scale (AES)46 at post-test only. Lastly, telephone interviews and a brief semistructured interview guide at 6 months will assist to assess predisposing, enabling and reinforcing factors of adoption and uptake in context of each decision aid based on the Ottawa Model of Research Use for evaluating knowledge translation innovation implementation and the WHO21 framework for disseminating and scaling up innovations. English and French interviews will be conducted by two team members experienced in conducting interviews. Field notes will be taken.

    Analysis

    The focus of the analyses is on descriptive statistics rather than formal tests of hypothesis (ie, we are not testing for statistical significance).47 Since sex/gender differences in utilisation and effectiveness of decision aids have not been previously reported, we will report outcomes for the total sample and separately for men and women. Quantitative data will be analysed using version SAS V.9.2.34 Since decisional conflict is our primary outcome, criteria for success will be defined as low decisional conflict (80% of scores <25). All differences between pretest and post-test knowledge scale scores will be assessed using McNemar’s test for binary variables and the paired t-test for (pseudo) continuous variables. We also anticipate high acceptability (80% of AES scores >24), and moderate to high post-test and 6-month engagement, defined as: (1) 80% of patients will access the PEP-CT Patient Partner decision aid at post-test and 6 months, (2) 80% of investigators will access the PEP-CT Investigator decision aid at post-test and 6 months, (3) end-users will access >80% or 4/5 functionalities (Introduction, My Priorities, Learn More, My Readiness and My Decision) at post-test and 50% of the functionalities at 6 months. Audiotapes and field notes from the interviews will be transcribed, translated and imported into NVivo.35 Further analyses incorporating codes, content analysis and disagreement processes will be conducted in a similar way to phases Phase 2B. In addition, we will provide recommendations on any sex/gender differences that may impact knowledge translation.

    Outcomes

    Results of the pragmatic pre/post pilot study of the bilingual decision aids will establish the extent to which each decision aid is feasible in terms of implementation (acceptability, engagement and fidelity). Phase 3 field/beta testing will also enable us to finalise, disseminate and evaluate adoption of these open-access web-based innovations, with an anticipated planned end date of Spring 2023.

    Ethics and dissemination

    Ethics approval has been granted from the University of Toronto (41109, 28 September 2021). Informed consent (online supplemental file 2) will be obtained from all participants engaging in Phase 2B and Phase 3 of the study. We will disseminate knowledge of the decision aids through co-authored publications, conference presentation, educational national public forums, fact sheets/newsletters, social media sharing and videos/webinars.

    Ethics statements

    Patient consent for publication

    References

    1. ↵
      1. Price A,
      2. Albarqouni L,
      3. Kirkpatrick J, et al
      . Patient and public involvement in the design of clinical trials: an overview of systematic reviews. J Eval Clin Pract 2018;24:240–53.doi:10.1111/jep.12805pmid:http://www.ncbi.nlm.nih.gov/pubmed/29076631
      OpenUrlCrossRefPubMed
    2. ↵
      1. Bird M,
      2. Ouellette C,
      3. Whitmore C, et al
      . Preparing for patient partnership: a scoping review of patient partner engagement and evaluation in research. Health Expect 2020;23:523–39.doi:10.1111/hex.13040pmid:http://www.ncbi.nlm.nih.gov/pubmed/32157777
      OpenUrlPubMed
    3. ↵
      1. Haywood K,
      2. Lyddiatt A,
      3. Brace-McDonnell SJ, et al
      . Establishing the values for patient engagement (PE) in health-related quality of life (HRQoL) research: an international, multiple-stakeholder perspective. Qual Life Res 2017;26:1393–404.doi:10.1007/s11136-016-1465-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/27933429
      OpenUrlPubMed
    4. ↵
      1. Kirwan JR,
      2. de Wit M,
      3. Frank L, et al
      . Emerging guidelines for patient engagement in research. Value Health 2017;20:481–6.doi:10.1016/j.jval.2016.10.003pmid:http://www.ncbi.nlm.nih.gov/pubmed/28292494
      OpenUrlCrossRefPubMed
    5. ↵
      1. Willison D,
      2. Richards D
      . Landscape review of strategies for recruitment and retention of research participants into clinical trials, 2015.
    6. ↵
      1. Coulter A,
      2. Stilwell D,
      3. Kryworuchko J, et al
      . A systematic development process for patient decision AIDS. BMC Med Inform Decis Mak 2013;13.doi:10.1186/1472-6947-13-S2-S2
    7. ↵
      1. Elwyn G,
      2. O'Connor A,
      3. Stacey D, et al
      . Developing a quality criteria framework for patient decision AIDS: online international Delphi consensus process. BMJ 2006;333:417. doi:10.1136/bmj.38926.629329.AEpmid:http://www.ncbi.nlm.nih.gov/pubmed/16908462
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Volk RJ,
      2. Coulter A
      . Advancing the science of patient decision AIDS through reporting guidelines. BMJ Qual Saf 2018;27:337–9.doi:10.1136/bmjqs-2017-007657pmid:http://www.ncbi.nlm.nih.gov/pubmed/29371413
      OpenUrlFREE Full Text
    9. ↵
      1. Feldman-Stewart D,
      2. O’Brien MA,
      3. Clayman ML, et al
      . Providing information about options in patient decision AIDS. BMC Med Inform Decis Mak 2013;13.doi:10.1186/1472-6947-13-S2-S4
    10. ↵
      1. Stacey D,
      2. Légaré F,
      3. Col N
      . Decision aid for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2014;1.
    11. ↵
      1. Stacey D,
      2. Légaré F,
      3. Lewis K, et al
      . Decision AIDS for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2017;4:CD001431. doi:10.1002/14651858.CD001431.pub5pmid:http://www.ncbi.nlm.nih.gov/pubmed/28402085
      OpenUrlCrossRefPubMed
    12. ↵
      1. Sepucha KR,
      2. Borkhoff CM,
      3. Lally J, et al
      . Establishing the effectiveness of patient decision AIDS: key constructs and measurement instruments. BMC Med Inform Decis Mak 2013;13:1–12.doi:10.1186/1472-6947-13-S2-S12
      OpenUrlCrossRefPubMed
    13. ↵
      1. Clayton JA,
      2. Tannenbaum C
      . Reporting sex, gender, or both in clinical research? JAMA 2016;316:1863–4.doi:10.1001/jama.2016.16405pmid:http://www.ncbi.nlm.nih.gov/pubmed/27802482
      OpenUrlCrossRefPubMed
    14. ↵
      1. Rajakannan T,
      2. Fain K,
      3. Williams R
      . Reporting of sex and gender in clinical trial protocols and published results. Chicago, USA: International Congress on Peer Review and Scientific Publication, 2017.
    15. ↵
      1. Norris CM,
      2. Tannenbaum C,
      3. Pilote L, et al
      . Systematic incorporation of sex-specific information into clinical practice guidelines for the management of ST-Segment-Elevation myocardial infarction: feasibility and outcomes. J Am Heart Assoc 2019;8:e011597. doi:10.1161/JAHA.118.011597pmid:http://www.ncbi.nlm.nih.gov/pubmed/30929545
      OpenUrlPubMed
    16. ↵
      1. Welch V,
      2. Doull M,
      3. Yoganathan M, et al
      . Reporting of sex and gender in randomized controlled trials in Canada: a cross-sectional methods study. Res Integr Peer Rev 2017;2:15. doi:10.1186/s41073-017-0039-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/29451565
      OpenUrlPubMed
    17. ↵
      1. Day S,
      2. Mason R,
      3. Lagosky S, et al
      . Integrating and evaluating sex and gender in health research. Health Res Policy Syst 2016;14:75. doi:10.1186/s12961-016-0147-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/27724961
      OpenUrlPubMed
    18. ↵
      1. Logan JO,
      2. Graham IAND
      . Toward a comprehensive interdisciplinary model of health care research use. Sci Commun 1998;20:227–46.doi:10.1177/1075547098020002004
      OpenUrlCrossRefWeb of Science
    19. ↵
      1. Parry M,
      2. Bjørnnes AK,
      3. Toupin-April K, et al
      . Patient engagement partnerships in clinical trials: development of patient partner and investigator decision AIDS. Patient 2020;13:745–56.doi:10.1007/s40271-020-00460-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/33026639
      OpenUrlPubMed
    20. ↵
      1. Mummah SA,
      2. Robinson TN,
      3. King AC, et al
      . Ideas (integrate, design, assess, and share): a framework and toolkit of strategies for the development of more effective digital interventions to change health behavior. J Med Internet Res 2016;18:e317. doi:10.2196/jmir.5927pmid:http://www.ncbi.nlm.nih.gov/pubmed/27986647
      OpenUrlPubMed
    21. ↵
      1. WHO
      . Nine steps for developing a scaling-up strategy. France, 2010.
    22. ↵
      1. Cotugna N,
      2. Vickery CE,
      3. Carpenter-Haefele KM
      . Evaluation of literacy level of patient education Pages in health-related journals. J Community Health 2005;30:213–9.doi:10.1007/s10900-004-1959-xpmid:http://www.ncbi.nlm.nih.gov/pubmed/15847246
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Holzinger A
      . Usability engineering methods for software developers. Commun ACM 2005;48:71–4.doi:10.1145/1039539.1039541
      OpenUrl
    24. ↵
      1. Jibb LA,
      2. Stevens BJ,
      3. Nathan PC, et al
      . A smartphone-based pain management APP for adolescents with cancer: establishing system requirements and a pain care algorithm based on literature review, interviews, and consensus. JMIR Res Protoc 2014;3:e15. doi:10.2196/resprot.3041pmid:http://www.ncbi.nlm.nih.gov/pubmed/24646454
      OpenUrlPubMed
    25. ↵
      1. Stinson J,
      2. Gupta A,
      3. Dupuis F, et al
      . Usability testing of an online self-management program for adolescents with cancer. J Pediatr Oncol Nurs 2015;32:70–82.doi:10.1177/1043454214543021pmid:http://www.ncbi.nlm.nih.gov/pubmed/25037173
      OpenUrlCrossRefPubMed
    26. ↵
      1. Breakey VR,
      2. Warias AV,
      3. Ignas DM, et al
      . The value of usability testing for Internet-based adolescent self-management interventions: "Managing Hemophilia Online". BMC Med Inform Decis Mak 2013;13:113. doi:10.1186/1472-6947-13-113pmid:http://www.ncbi.nlm.nih.gov/pubmed/24094082
      OpenUrlPubMed
    27. ↵
      1. Macefield R
      . How to specify the participant group size for usabiltiy studies: A practitioner’s guide. J Usability Stud 2009;5:34–45.
      OpenUrl
    28. ↵
      1. Nielsen J,
      2. Landauer T
      . A mathematical model of the finding of usability problems. paper presented at: proceedings of ACM INTERCHI'93 conference; 24-29 April 1993. Amsterdam, The Netherlands, 1993.
    29. ↵
      1. Kushniruk A
      . Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med 2002;32:141–9.doi:10.1016/S0010-4825(02)00011-2pmid:http://www.ncbi.nlm.nih.gov/pubmed/11922931
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Jaspers MWM
      . A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform 2009;78:340–53.doi:10.1016/j.ijmedinf.2008.10.002pmid:http://www.ncbi.nlm.nih.gov/pubmed/19046928
      OpenUrlCrossRefPubMed
    31. ↵
      1. Brooke J
      . A "quick and dirty" usabilty scale. In: Jordan P, Thomas B, Weerdmeester B, et al., eds. Usability evaluation in industry. London: Taylor & Francis, 1996.
    32. ↵
      1. Bangor A,
      2. Kortum PT,
      3. Miller JT
      . An empirical evaluation of the system usability scale. Int J Hum Comput Interact 2008;24:574–94.doi:10.1080/10447310802205776
      OpenUrlCrossRefWeb of Science
    33. ↵
      1. Jibb LA,
      2. Cafazzo JA,
      3. Nathan PC, et al
      . Development of a mHealth Real-Time Pain Self-Management App for Adolescents With Cancer: An Iterative Usability Testing Study [Formula: see text]. J Pediatr Oncol Nurs 2017;34:283–94.doi:10.1177/1043454217697022pmid:http://www.ncbi.nlm.nih.gov/pubmed/28376666
      OpenUrlPubMed
    34. ↵
      1. SAS
      . SAS: the power to know. Available: http://www.sas.com/ [Accessed 15 Oct 2012].
    35. ↵
      NVivo [computer program]. Australia. QSR International
    36. ↵
      1. Sandelowski M
      . Qualitative analysis: what it is and how to begin. Res Nurs Health 1995;18:371–5.doi:10.1002/nur.4770180411pmid:http://www.ncbi.nlm.nih.gov/pubmed/7624531
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Sandelowski M
      . Whatever happened to qualitative description? Res Nurs Health 2000;23:334–40.doi:10.1002/1098-240X(200008)23:4<334::AID-NUR9>3.0.CO;2-Gpmid:http://www.ncbi.nlm.nih.gov/pubmed/10940958
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Kvale S
      . Interviews: an introduction to qualitative research interviewing. California: Sage Publications, 1996.
    39. ↵
      1. Graham ID,
      2. Logan J
      . Innovations in knowledge transfer and continuity of care. Can J Nurs Res 2004;36:89–103.pmid:http://www.ncbi.nlm.nih.gov/pubmed/15369167
      OpenUrlPubMed
    40. ↵
      1. Hertzog MA
      . Considerations in determining sample size for pilot studies. Res Nurs Health 2008;31:180–91.doi:10.1002/nur.20247pmid:http://www.ncbi.nlm.nih.gov/pubmed/18183564
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      1. Stacey D,
      2. Bennett C,
      3. Barry M
      . Decision AIDS for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2011;10.
    42. ↵
      1. McGillion MH,
      2. Carroll SL,
      3. Metcalfe K, et al
      . Development of a patient decision aid for people with refractory angina: protocol for a three-phase pilot study. Health Qual Life Outcomes 2014;12:93. doi:10.1186/1477-7525-12-93pmid:http://www.ncbi.nlm.nih.gov/pubmed/24920518
      OpenUrlPubMed
    43. ↵
      1. Metcalfe KA,
      2. Poll A,
      3. O'Connor A, et al
      . Development and testing of a decision aid for breast cancer prevention for women with a BRCA1 or BRCA2 mutation. Clin Genet 2007;72:208–17.doi:10.1111/j.1399-0004.2007.00859.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/17718858
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      1. O'Connor AM,
      2. Tugwell P,
      3. Wells GA, et al
      . A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns 1998;33:267–79.doi:10.1016/S0738-3991(98)00026-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/9731164
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      1. O'Connor AM,
      2. O’Connor A
      . Validation of a decisional conflict scale. Med Decis Making 1995;15:25–30.doi:10.1177/0272989X9501500105pmid:http://www.ncbi.nlm.nih.gov/pubmed/7898294
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      1. Tariman JD,
      2. Berry DL,
      3. Halpenny B, et al
      . Validation and testing of the acceptability E-scale for web-based patient-reported outcomes in cancer care. Appl Nurs Res 2011;24:53–8.doi:10.1016/j.apnr.2009.04.003pmid:http://www.ncbi.nlm.nih.gov/pubmed/20974066
      OpenUrlPubMed
    47. ↵
      1. Thabane L,
      2. Ma J,
      3. Chu R, et al
      . A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010;10:1. doi:10.1186/1471-2288-10-1pmid:http://www.ncbi.nlm.nih.gov/pubmed/20053272
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @parryresearch

    • Contributors The PI (MP) and Co-PI (TC) conceived the study. ZZ drafted and revised the manuscript prior to submission. Co-authors (HA, AKB, HB, SC, AD, AE, DF, IG, AN, DPR, KT-A, DW, ZZ, SM) contributed to formulating the study design and will be involved in the execution of the planned sequential phases (recruitment, dissemination activities). All authors are grant holders. Day provided quantitative methodological expertise and will lead the sex-based analyses. AKB and KT-A provided qualitative methodological expertise and will lead all qualitative analyses. Three patient partners (TC, AN, DW) form the PEP-CT PAC and are Co-PIs. MP and HB finalised the Research Ethic Board (REB) submission. The PI (MP) and Co-PI (DW) will provide day-to-day oversight of the project. Most authors (HA, AKB, HB, SC, TC, AD, AE, DF, IG, AN, MP, DPR, KT-A, DW, SM) assisted to build and approve content for the funding application. All authors (HA, AKB, HB, SC, TC, AD, AE, DF, IG, AN, MP, DPR, KT-A, DW, ZZ, SM) approved the final manuscript prior to submission and are accountable for all aspects in ensuring accuracy and integrity of work across all phases of the study.

    • Funding This work was supported by the Canadian Institute of Health Research Project Grant Fall 2019 and Spring 2020 (CIHR; 436272).

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; peer reviewed for ethical and funding approval prior to submission.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.