Article Text
PDF
Abstract
Introduction Despite being more than two decades of research, mesenchymal stromal cell (MSC) treatments are still struggling to cross the translational gap. Two key issues that likely contribute to these failures are (1) the lack of clear definition for MSC and (2) poor quality of reporting in MSC clinical studies. To address these issues, we propose a modified Delphi study to establish a consensus definition for MSC and reporting guidelines for clinical trials of MSC therapy.
Methods and analysis We will conduct a three-round international modified Delphi survey. Findings from a recent scoping review examining how MSCs are defined and reported in preclinical and clinical studies were used to draft the initial survey for round 1 of our Delphi. Participants will include a ‘core group’ of individuals as well as researchers whose work was captured in our scoping review. The core group will include stakeholders from different research fields including developmental biology, translational science, research methods, regulatory practices, scholarly journal editing and industry. The first two survey rounds will be online, and the final round will take place in person. Each participant will be asked to rate their agreement on potential MSC definition characteristics and reporting items using a Likert scale. After each round, we will analyse the data to determine which items have reached consensus for inclusion/exclusion, and then develop a revised questionnaire for any new items, or items that did not reach consensus.
Ethics and dissemination This study received ethical approval from the Ottawa Health Research Network Research Ethics Board. To support the dissemination of our findings, we will use an evidence-based ‘integrated knowledge translation’ approach to engage knowledge users from the inception of the research. This will allow us to develop a tailored end-of-project knowledge translation plan to support and ensure dissemination and implementation of the Delphi results.
- protocols & guidelines
- cell biology
- therapeutics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
We proposed to address the current limitations in mesenchymal stromal cell experimental and clinical research with a rigorous and methodological consensus building method (Delphi method) that will allow for structured communication on controversial issues.
To support dissemination and implementation of our results, we will engage stakeholders and end users from the inception of the project—such as patient partners—and will develop a tailored end-of-project knowledge translation plan (integrated knowledge translation approach) in order to overcome historical issues related to community uptake.
To address the main limitations of a Delphi method (eg, lack of participation, no in-person interaction or information exchange), we use a modified Delphi survey with a core group of stakeholders and a face-to-face meeting.
Introduction
Since their original description1 and their first use as a therapeutic agent in humans,2 interest for mesenchymal stromal cells (MSC) from the scientific and patient community keeps growing exponentially: a PubMed search with the query MSCs between 1995 and 2021 found 73 876 results; more than half of the results were from the last 5 years. From a clinical research perspective, over 1300 MSC clinical trials have been registered on ClinicalTrials.gov.3 Despite promising results of MSC in different preclinical diseases models, clinical trials using MSC in various medical conditions failed to deliver encouraging results.4 5 Many potential explanations for this disparity between preclinical and clinical results exist, including MSC characteristics, cell manufacturing processes, administration protocols or study participants’ characteristics such as disease severity and associated comorbidities.6 In addition, MSC characteristics (eg, definition, characterisation, immune compatibility, cell viability and dose) raise two main issues of (1) lack of consensus definition for MSC and (2) broad variability in MSC characteristics reported. For example, a report from the Food and Drug Administration showed a significant heterogeneity in MSC products used in the clinical trials, with important differences in cell surface marker characterisation, product bioactivity assessment, as well as tissue sourcing and product manufacturing.7 Regarding reporting of MSC characteristics, members of our group have performed systematic reviews of published MSC clinical trials and have found extremely poor (ie, incomplete) reporting of cell products used.8–10 These observations have pushed some scientists to call to ‘clear up this stem cell mess’.4
The first important initiative to develop a consensus definition for MSC was provided by the International Society for Cell & Gene Therapy (ISCT), which in 2006 established the minimal criteria for human bone marrow-derived MSC and these were updated in 2019.11 12 These criteria were determined through a group decision-making method (eg, informal consensus of a small number of experts), which is a consensus development method with several limitations.13 However, implementation of this MSC definition has been inconsistent and several researchers have expressed scepticism and highlighted limitations (eg, cell surface markers phenotype, misinterpretation of differentiation assays, limitations in functionally defining stromal cells, etc).4 11 14 For example, a scoping review conducted by our group to describe how MSCs are defined and characterised in both preclinical and clinical research showed that only 18% of the articles from our sample explicitly referred to the ISCT criteria, and only 7% of clinical studies selection reported the three minimal criteria from the 2006 definition. The uncertainty and lack of consensus on this crucial issue of how to define MSC has also allowed questionable for-profit business to sell unproven and unlicensed ‘stem cell’ therapies to patients and public.15 16
Therefore, there is an urgent need to develop a consensus definition of MSC and reporting guidelines to create standards for complete and transparent reporting of both cell characteristics and clinical trial/manufacturing details. A clear definition and reporting guidelines will be the cornerstones of a more robust, reproducible and transparent research, and are mandatory to better understand underlying factors that may contribute to efficacy (or a lack of efficacy) of MSC. Moreover, for patients and members of the public, having a clear definition of MSC will allow them to make informed decisions into which related treatments, clinical trials or products they elect to partake in.
To address limitations of previous attempts to define MSC and support the dissemination and implementation of MSC definition and reporting guidelines, our research protocol is based on a modified Delphi method combined with an integrated knowledge translation (iKT) approach.
The objectives of the current study are to: (1) develop, disseminate and implement an updated consensus definition of minimal criteria to define MSC; and (2) develop reporting guidelines for the clinical trials of MSC therapy. The work is not hypothesis testing, and therefore we have no a priori predictions related to study outcomes.
Methods and analysis
Project overview
We will conduct a three-round Delphi survey. The Delphi method is a process that uses several rounds of surveying in order to reach consensus on a topic.13 17 Between rounds, responses to survey items are aggregated—items that reach a priori threshold for inclusion/exclusion are omitted, and any remaining or new items are shared with participants in the subsequent round to reconsider and vote on again. The Delphi method addresses group decision-making method’s limitations such as (1) group influences on individual performance, (2) adequately accounting for divergent opinions, (3) limiting the number of participants, (4) meeting organisation and costs, (5) time limits for very complicated decisions, (6) peer pressure and influence in the decision-making process, and (7) logistical issues.13 The Delphi method has been successful in the past in solving contentious issues.18–20
Participants
Participants for this Delphi study will be identified in two ways:
First, we will invite members of a ‘core group’ of experts that have been identified for the purpose of planning and implementing this research programme. The core group members are diverse in their make-up, including in vitro, preclinical and clinical researchers, and include member with backgrounds in developmental biology, translational science, research methods, regulatory practices, scholarly journal editing and industry.
Second, we will invite a ‘researcher panel’ to participate in the Delphi. This panel will comprise 311 researchers that we identified in our previous scoping review of MSC research conducted between March and May 2020 (submitted). Our scoping review (https://osf.io/3dsqx/) was developed to describe how MSCs are currently defined in preclinical and clinical research, which will provide valuable themes to consider as items for the current Delphi survey. We extracted the corresponding author’s name, email and country of primary affiliation from all original articles included in this scoping review. These authors will be contacted by a member of the research team and invited to participate in the Delphi survey.
Recruitment
Participants on the core group and on the researcher panel will be approached for participation in the study using a standardised email recruitment script (see online supplemental appendix 1). This script will provide participants with more information about the study and provide a link to access the informed consent form and the round 1 online Delphi survey. The core group will take part in all three rounds of the Delphi. The researcher panel will be invited to the initial round only. This approach was taken to maximise the initial reach of our survey while maintaining a manageable number of individuals to facilitate our round 3 in-person consensus meeting.
Supplemental material
Participants will be given 3 weeks to complete each round of the Delphi survey. Reminder emails will be sent 7 and 14 days following the dissemination of each online questionnaire.
Study design
Our modified Delphi study will involve three rounds of surveying; the first two will take place via an online survey, while the third round will take place via an in-person consensus meeting. The study design is presented in figure 1. The online Delphi surveys will be administered using Surveylet (https://calibrum.com), a cloud-based platform specifically designed for Delphi surveys.
Study design. MSC, mesenchymal stromal cell.
Round 1 (online): completed by the researcher panel and the core group.
Round 2 (online): completed by the core group only.
Round 3 (virtual consensus meeting): all items will be discussed by the core group members during a virtual meeting in 2022.
Delphi survey
We have generated an initial survey (https://osf.io/u579g/) which has three sections: (1) participant demographics, (2) items for MSC definition, and (3) items for reporting guidelines. Based on participants’ self-identified field of expertise, they will access different parts of the survey: MSC definition, reporting guidelines or both. This survey was piloted initially by six members of our research group. Then, the questionnaire was sent to eight leaders of opinion in MSC research (core group members and leadership within international stem cell scientific societies) to ensure relevance and completeness of our items for definition and reporting guidelines.
After completing the demographics section, participants will be asked to rate each survey item’s importance for MSC definition and/or as a minimal reporting criterion (eg, ‘A description of MSC capacity to adhere to a plastic surface when maintained in standard culture condition, is essential to define them’) on a scale of 1 (strongly disagree) to 9 (strongly agree). A free text response option will accompany each listed item to allow participants to elaborate on their rating. At the end of the survey, participants will also have the opportunity to suggest any new relevant items to define MSC or report clinical trials of MSC therapy that were not already listed on the survey. All the additional input will be analysed and added to the new questionnaire for the next round. This process allows any additional items (eg, haemocompatibility of MSC), to be discussed and rated by all the participants. After each round, data from the survey will be aggregated. A revised survey will be developed for the subsequent rounds that incorporates any newly suggested items, and items that did not reach consensus. Items that did not reach consensus will be presented with verbatim feedback participants made in the previous rounds.
Study conceptualisation and survey development began in the second and third quarters of 2021. We are currently refining and piloting the survey, and plan to administer the three rounds in the fourth quarter of 2021. The study should be completed by the first quarter of 2022.
Data analysis
Data analysis
Data will be analysed using Excel. Data will be collected anonymously. The total number of participants, item rating scores and demographic data will be summarised using frequencies and percentages. For each Likert scale item, we will report the mean and range. Mean rating scores for each item will be categorised into three groups (unessential, potentially essential, essential). A median rating of 1–3 means an item is deemed unessential, 4–6 implies that an item is potentially essential and a median rating of 7–9 is deemed essential to report.
Consensus will be achieved when at least 80% of Delphi participants rate the item in the same category of importance: unessential or essential. This threshold was described as the most commonly used in a systematic review of consensus in Delphis.21 Items that achieve consensus, as being ‘essential or unessential’, will be labelled as such, and in subsequent rounds participants will have the opportunity to comment on these items. Items that did not achieve consensus, achieved consensus as ‘potentially essential’, had major wording changes or were added by Delphi participants will automatically move on to the subsequent rounds. For each round, response rate will be recorded and reported.
First round analysis
Any items from round 1 with disagreement between the core group and researcher panel will be provided to the core group in round 2 to revote on. For transparency, all the results will be reported separately in a final manuscript so that any discrepancies between the core group and the research group will be clearly stated.
Second round analysis
After the second round with the core group, items that reach consensus (>80% agreement) in the essential category will be considered for inclusion in the MSC definition and the reporting guidelines. Items that reach consensus in the unessential category will be considered for exclusion. All remaining items that have not yet reached consensus will be presented and discussed at the final face-to-face virtual meeting where they will then be electronically voted on anonymously using Surveylet software (https://calibrum.com).
Potential limitations and mitigation strategies
One feature of the traditional Delphi method is that the participants are isolated from each other, with no in-person interaction or information exchange to limit group influences and peer pressure. Face-to-face interaction can be useful to identify reasons for any disagreements. To address this limitation, we plan to organise the third round as face-to-face virtual meeting, after which participants again vote anonymously. A lack of participation overall is another concern for any survey-based study. We have mitigated this risk by involving thought leaders from MSC science and other stem cell areas who have committed to contributing. Furthermore, our previously conducted scoping review captured 311 MSC researchers, and as such if not all researchers respond we still anticipate sufficient recruitment numbers. Another potential limitation of the Delphi method is respondent attrition between rounds. We have addressed this by engaging our core group from project inception. During the rounds, reminder emails will be sent to solicit greater responses, and to limit participant attrition between rounds, we plan to analyse the results of each round and provide feedback and a new questionnaire rapidly to maintain the interest and engagement of participants. Respondent attrition per round will be recorded and reported.
Ethics and dissemination
Ethics
Ethics approval was obtained from the Ottawa Health Research Network Research Ethics Board (REB protocol ID 20210187-01K).
iKT approach
The iKT will ensure that the MSC research community is aware of the recommendations described in our consensus definition and reporting guidelines in order to maximise the impact of the work. This entails the early inclusion of key stakeholders who have the ability to implement our recommendations, including patient partners. This participation begins from project inception and continues through to dissemination and implementation of study findings. The goal of the iKT approach is to ensure that the needs and preferences of stakeholders are considered throughout the project, with the idea being that successful partnership upstream will facilitate improved implementation and uptake downstream.22
Finally, to further support the dissemination of our findings (definition and reporting guidelines), we will use a structured approach to end of project, modelled on Canadian Institutes of Health Research’s Knowledge Translation planning guide (https://cihr-irsc.gc.ca/e/45321.html). This essentially will involve considering the specific project findings and determining the dissemination goal (eg, to increase awareness of the findings, to increase knowledge, to influence practice or policy), identifying key audiences, crafting messages tailored to specific audiences and using strategies and media to reach each audience. A written knowledge translation plan will help ensure study outputs are effectively used. Our knowledge users will help interpret the Delphi results and craft dissemination messages. We plan to create an extension to existing clinical study reporting guidelines (eg, Consolidated Standards of Reporting Trials) that will focus on aspects unique to MSC therapy trials (eg, manufacturing, characterisation, storage and delivery of cells). To support its implementation, the guideline draft will be published as a preprint to obtain feedback from the scientific community and associated explanatory documents such as a guidance development statement and an Explanation and Elaboration document will be provided.
Registration and data availability
This study protocol was registered using the Open Science Framework (OSF) (https://osf.io/3dsqx/). Data and study materials will be made publicly available at the time of publication using OSF.
Patient and public involvement
A patient partner is involved in our project since inception. The patient partner will provide feedback on the consensus definition of MSC and reporting guidelines development and on knowledge translation strategies. They will help codevelop a communication plan for the general public by supporting the development of non-technical summaries for the scoping review and the Delphi findings as well as their dissemination to general public via patients’ associations and social media.
Ethics statements
Patient consent for publication
Acknowledgments
The authors want to thank the following members from the Thebaud lab for their valuable feedbacks on the initial Delphi questionnaire: Pauline Bardin, Chanele Cyr-Depauw, Flore Lesage, Marissa Lithopoulos, Ivana Mizikova and Liqun Xu.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @MaxLeOTT, @manojlalu
Contributors LR: conceptualisation, methodology and writing (original draft preparation). MML, KDC, BT: conceptualisation, funding acquisition, methodology, supervision and writing (review and editing). ML (patient partner): manuscript review and editing.
Funding This work is supported by the Canadian Stem Cell Network, ‘Translation and Society Team Awards’ 2020 grant. MML is supported by The Ottawa Hospital Anesthesia Alternate Funds Association and holds a University of Ottawa Junior Research Chair in Innovative Translational Research.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.