Article Text
PDF
Abstract
Introduction Cerebral palsy (CP) is the most common childhood physical disability. Accurate diagnosis before 6 months is possible using predictive tools and decision-making skills. Yet diagnosis is typically made at 12–24 months of age, hindering access to early interventions that improve functional outcomes. Change in practice is required for physicians in key diagnostic behaviours. This study aims to close the identified research–practice gap and increase accurate CP diagnosis before 6 months of age through tailored web-based implementation interventions. This trial will determine whether adaptive e-learning using virtual patients, targeting CP diagnostic behaviours and clinical decision-making skills, effectively changes physician behaviour and practice compared with non-adaptive e-learning instructional design or control.
Methods and analysis This study is a 3-arm parallel superiority randomised controlled trial of two tailored e-learning interventions developed to expedite physician CP diagnosis. The trial will compare adaptive (arm 1) and non-adaptive (arm 2) instructional designs with waitlist control (arm 3) to evaluate change in physician behaviour, skills and diagnostic practice. A sample size of 275 paediatric physicians enables detection of small magnitude effects (0.2) of primary outcomes between intervention comparators with 90% power (α=0.05), allowing for 30% attrition. Barrier analysis, Delphi survey, Behaviour Change Wheel and learning theory frameworks guided the intervention designs. Adaptive and non-adaptive video and navigation sequences utilising virtual patients and clinical practice guideline content were developed, integrating formative key features assessment targeting clinical decision-making skills relative to CP diagnosis.
Physician outcomes will be evaluated based on postintervention key feature examination scores plus preintervention/postintervention behavioural intentions and practice measures. Associations with CP population registers will evaluate real-world diagnostic patient outcomes. Intervention costs will be reported in a cost–consequence analysis from funders’ and societal perspectives.
Ethics and dissemination Ethics approved from The University of Sydney (Project number 2021/386). Results will be disseminated through peer-reviewed journals and scientific conferences.
Trial registration number Australian New Zealand Clinical Trials Registry: ACTRN 12622000184774.
- developmental neurology & neurodisability
- medical education & training
- protocols & guidelines
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The use of theoretical frameworks and evidence-centred instructional design processes underpinned the tailored interventions.
Head to head comparison of two theory-guided design features to avoid confounding in e-learning interventions containing multiple active learning components.
Study sample size powered to detect meaningful changes between comparator interventions and control.
Application of contemporary validity theory strengthens the evaluation framework and implications evidence for real-world practice.
The consent process for patient outcome measures may impact recruitment and attrition.
Introduction
Cerebral palsy (CP) is the most common physical childhood disability worldwide, with a rate of 1.4 per 1000 live births in Australia.1 Typically, a ‘wait and see’ diagnosis is made at 12–24 months of age in high-income countries2 and up to 5 years of age in low-income and middle-income countries, as indicated on CP Registers.3 Incontrovertible evidence now supports accurate, early CP diagnosis under 6 months of age through pooled predictive assessments and clinical decision-making skills, as outlined in a clinical guideline.2 Nonetheless, there is a demonstrable research–practice gap for CP diagnosis internationally. The Australian Cerebral Palsy Register currently indicates that only 21% of infants with CP are diagnosed under 6 months of age, with severe presentations diagnosed earlier.4
Congruent with animal studies demonstrating the importance of the critical drivers of neuroplasticity,5 6 groundswells of targeted early intervention studies in the early months of life are demonstrating improved functional outcomes for children and families.7–9 Early diagnostic-specific therapy and support can optimise infant motor and cognitive outcomes,7 10 limit musculoskeletal complications11 and foster protective factors for parent mental health.12 Conversely, a delay in diagnosis may deny access to early targeted intervention at the most critical time for infant motor and cognitive development, and is known to increase anger and depression in parents and caregivers, negatively impacting parental mental health.13 A fundamental change in health professional diagnostic practice is urgently needed to support and enable an earlier clinical diagnosis of CP.
The variable uptake of guidelines and time lag of up to 17 years14 15 to change historical practice is well documented and has driven research fields to target the research–practice gap.16 17 Implementation science studies the ‘methods to promote the systematic uptake of research findings and other evidence-based practice into routine practice’.18 Implementation interventions aim to facilitate change in clinical practice, behaviour or policy and can be targeted at different health system levels.18 Tailoring such interventions,19 to their context and identified barriers, is recommended to improve effectiveness.20 Testing and evaluating theories that underpin the development of implementation interventions and their evaluation are critical to demonstrating effectiveness in practice.20–26 In addition, it is recommended that evaluations adopt a comprehensive and structured approach to validity testing of outcome measures for the context of their use.27–29
To focus our work, we applied principles from the Behaviour Change Wheel (BCW), a framework that guides theory-informed implementation studies.30 Through research prioritisation methods in the Australian context,31 we established target audiences and behaviours to increase the diagnosis of CP under 6 months of age. Paediatric physicians are the main providers of a clinical CP diagnosis in the Australian setting and were ascertained as a primary implementation target audience. Priority physician clinical behaviours were identified using the BCW. Education, training, modelling, persuasion and enablement interventions were identified as means of changing physician behaviour, skills and diagnostic practice in the early diagnosis of CP. Web-based formats were determined as accessible and potentially cost-effective for these interventions.
As an educational intervention mediated electronically via the internet, web-based learning and education can be broadly referred to as e-learning in health professional literature.32 Multiple systematic reviews establish e-learning as effective in improving health professional knowledge outcomes; however, a paucity of high-quality evidence exists to support e-learning effectiveness in changing behaviour and practice or patient outcomes.32–35 Advancing the empirical evidence base for web-based educational interventions beyond knowledge acquisition and demonstrating real-world effectiveness is a priority for implementation fields of research36 and e-learning.32 Field leaders in health professional e-learning implore rigorous design methods supported by theory and robust evaluations controlling for confounders to enable replication.37 38 High-quality reporting of design and development costs associated with health professional e-learning is lacking.39 Evaluations exploring the costs and consequences associated with comparative e-learning designs will progress the evidence base.39
The development of theory and evidence-based e-learning necessitates an understanding of instructional design principles.40 Instructional design involves task analysis and specification of instructional strategies based on appropriate theory.41 Design choices specific to e-learning need to first consider the overall mode of instruction (eg, simulation)42 and in the second place consider the individual design features (eg, interactivity and feedback), which are the active ingredients.33 43 Guiding instructional design frameworks specific to health professional e-learning are rare.42 In addition, reporting of theory-based insructional design is sparse in the health professional e-learning literature.44 Therefore, after determining outcomes and active ingredients for behaviour change using the BCW, we subsequently focused on the process of aligning theory and evidence-based instructional modalities and design features for our tailored e-learning intervention.
Video-based instructional modalities using clinical simulation to facilitate engagement and authentic immersion are a growing health professional research area.45 46 Video simulations of clinical cases, known as virtual patients, have demonstrated large effect sizes on health professional skills compared with traditional education, highlighted in a recent meta-analysis (standardised mean difference (SMD)=0.90, 95% CI: 0.49 to 1.32, I2=88%, n=897, low-quality evidence).45 The strength and quality of the established systematic review evidence in virtual patients supports the selection of this instructional modality for education and training interventions for health professional skills, including clinical decision-making.45 46 However, there is an identified gap in the evidence base for individual design features within virtual patient modalities.45 46 This evidence gap in instructional design limits the understanding of how different design features affect learning outcomes and, subsequently, the evidence-informed development of virtual patient interventions. Narrative synthesis of design feature comparisons within a systematic review of virtual patients suggests cognitive interactivity (the degree of learners’ cognitive engagement), mastery learning and feedback design features may be associated with improved learning outcomes.46 Systematic review evidence within the broader simulation field supports these findings and identifies other effective design features to affect behaviour, skills and patient outcomes: variation of clinical cases, task difficulty and learning strategies, repeated and spaced practice, longer time-frame and individualised learning approaches.43 Further comparative effectiveness research is necessary within virtual patient instructional modalities, comparing theory-guided active ingredients for a given learning outcome.45
Individualised learning design features have more recently been referred to as adaptive e-learning environments.47 Adaptive environments consider each learner’s interactions and performance level to adapt the content, navigation, multimedia or learning strategies of instructional design.47 Individual tailoring and user-centred approaches align with contemporary principles of web-based behaviour change.48 In addition, the alignment of design features to learning approaches is congruent with the core tenets of instructional design.41 Adaptive instructional design features have demonstrated large pooled effect sizes on health professional skills (SMD=1.19, 95% CI: 0.59 to 1.79, p<0.00001, low-quality evidence47), including clinical decision-making, warranting further investigation in high-quality studies.47 As such, the individual instructional design feature of adaptivity was further considered in our tailored intervention and comparative research study design to progress the evidence base in web-based learning.
The optimal use of instructional design features to support web-based learning and tailoring of our intervention was also considered through cognitive and learning theories. The application of learning theory is fundamental to health professional education design, yet is infrequently reported in e-learning interventions.41 44 49 While cognitive science fields have identified a vast array of evidence-based learning strategies to improve knowledge acquisition, retention and transfer that is applicable to health professionals,50–57 practical application to e-learning instructional design remains underutilised.44
Learning approaches can vary in complexity and may not apply uniformly across all subject domains, cognitive tasks and levels of expertise.58 For example, the learning processes required to acquire domain knowledge aspects of clinical decision-making are grounded in simple memory-based processes.58 Yet the application of knowledge through clinical decision-making and active problem-solving skills draws on more complex learning processes of induction and refinement, understanding and sense-making.59 60 To manage this variation in learning approaches in our intervention development, we repurposed the Knowledge-Learning-Instruction (KLI) framework60 used in education research. The KLI integrates knowledge structures, cognitive learning processes and instructional methods.60 When applied with the KLI framework, adaptive learning environments can be individualised and align learning processes and instructional design features. For example, multiple-choice questions with immediate feedback can be used for simple factual knowledge content for novice learners or key feature virtual patient cases can be used for higher-order clinical decision-making skills61 requiring greater expertise.
The key features assessment methodology has strong validity evidence supporting its use to assess health professionals’ clinical decision-making, rather than their knowledge.61 62 Key feature cases are based on a clinical scenario with 1–3 related questions on the unique critical elements essential to clinical decision-making or that are often performed incorrectly in practice.63 Presenting formative key feature virtual patients within an adaptive e-learning instructional design harnesses opportunities for immediate, tailored feedback on the most critical aspects of patient care when an error is made. Additionally, key feature examinations as summative assessments have demonstrated sensitivity to measure the impact of educational interventions.61 62 The established validity evidence for the use of key features examination scores to measure outcomes of physician clinical decision-making skills61 in our randomised controlled trial (RCT) was explored in our evaluation framework.
Conceptualisations of an evidence-based validity argument for the proposed use of test scores appropriate to context rather than an intrinsic property of an assessment have set the standards for contemporary validation research.64 65 Yet, despite reference guides by field leaders,66 there remains a paucity of application of validity theory or argument-based validity frameworks in health professional education.67 In addressing this gap, we utilised Kane’s contemporary validity framework to underpin our evaluation design.68 69 Kane’s framework guided the synthesis of priority evidence to build a validity argument for the use of identified assessment measures for the purposes of our RCT. Key feature cases for summative purposes targeting clinical decision-making skills in the early diagnosis of CP have been developed with a group of experts in CP, and piloted with practising physicians. The constructed validity argument supports the use of examination scores as an outcome measure of clinical decision-making skills and the plausibility of association of examination scores with real-world performance measures and patient outcome implications. The predictive validity of key feature examination scores on clinical performance, including patient outcomes, has been demonstrated through correlation studies with medical regulatory bodies70 71 and patient adherence to antihypertensive treatment.72 Further investigation of key feature performance scores posteducational interventions and association with real-world measures is warranted. Within our RCT, patient data transfer to population CP Registers and the Australian National Disability Insurance Scheme (NDIS) will measure patient outcomes and the strength of association with physician examination performance scores. This study will progress the evidence base in health professional e-learning and implementation science in measuring real-world diagnostic practice following tailored theory and evidence-based interventions. The study design has potential for replication in other country contexts (high-income, middle-income and low-income) to increase CP diagnosis under 6 months of age and adherence to clinical guidelines.
This paper describes: (1) the development of theory and evidence-based virtual patient e-learning tailored for physicians to enable early diagnosis of CP; (2) a 3-arm RCT comparing adaptive and non-adaptive e-learning instructional designs with a control group to evaluate the effectiveness on change in physician behaviour, skills, diagnostic practice and patient outcomes; and (3) a within-trial cost–consequence analysis alongside the RCT. A conceptual overview of the study’s use of theoretical frameworks and evidence-based design is presented in figure 1.
Conceptual overview of study using theoretical frameworks and evidence-based design
Objectives
The primary objective of this RCT is to evaluate the effectiveness of adaptive (arm 1) versus non-adaptive e-learning (arm 2) interventions and control (arm 3) on physician postintervention key feature examination scores and change in pre–post intervention target diagnostic behaviours and self-reported CP diagnostic practice.
The secondary objective is to evaluate the effect of physician clinical decision-making and behaviour on real-world patient outcomes: (1) age and severity of CP on referrals to CP Registers,1 (2) age and severity of CP on access requests to NDIS and (3) NDIS eligibility and funding outcomes.
The tertiary objectives of this study are to evaluate: (1) physician experience and satisfaction and (2) intervention costs and consequences of the e-learning resource.
Primary hypotheses
We hypothesise that:
Adaptive e-learning will produce superior scores on the key features examination by an effect size of Cohen’s d equal to 0.2 compared with physicians who receive the non-adaptive e-learning and an effect size of Cohen’s d equal to 0.8 compared with the control group.
Effect size assumptions on key feature test primary outcome measure considered: (1) pilot study average test item score across 11 test items of 0.54 with a SD of 0.28, (2) sensitivity of key features examination to detect change post educational interventions61 62 73 and (3) systematic reviews of pooled effect sizes on health professionals’ skills for adaptive e-learning47 and virtual patients.45
Adaptive e-learning will be more effective in improving target physician diagnostic behaviours measured by physician behavioural intention scores and self-report audit of practice behaviours pre–post intervention by an effect size of Cohen’s d equal to 0.2 compared with physicians who receive the non-adaptive e-learning and an effect size of Cohen’s d equal to 0.4 compared with the control group.
Effect size assumptions for primary outcomes of physician behavioural intentions and self-report audit of practice behaviours considered: (1) Continuing Professional Development (CPD)-Reaction questionnaire effect sizes post online educational interventions in other fields74 75; and (2) current Australian practice from a group of advisory experts.
Secondary hypotheses
We hypothesise that:
Adaptive and non-adaptive e-learning will be more effective than control in increasing physician diagnosis of CP under 6 months of age for milder phenotypes of CP severity (predictive of ambulant CP) measured by retrospective self-report audit and prospective referrals to the CP Registers by an effect size of Cohen’s d equal to 0.2.
Adaptive and non-adaptive e-learning will be more effective than control in increasing physician access requests for CP diagnosis under 6 months of age to the NDIS by an effect size of Cohen’s d equal to 0.2.
Methods and analysis
Trial design
This study is a 3-arm parallel superiority assessor-blinded RCT. This protocol has been informed by the Standard Protocol Items for Randomised Trials 2013,76 and the Consolidated Standards of Reporting Trials (CONSORT)-EHEALTH checklist V.1.6.1.77 The design is developed following the Medical Research Council for standards for evaluating the effectiveness of complex interventions.78 The cost–consequence analysis follows recommendations from the Consolidated Health Economic Evaluation Reporting Standards statement.79
Participants
The study population is paediatric physicians. During the study period, if a physician participant provides a clinical diagnosis of CP to an infant in their regular clinical practice, they are asked to invite the parents of the infant to participate in the study to measure patient outcomes (see Participant Information Statement in online supplemental file 1). The study design with practising physicians involves pre–post intervention self-reported physician behaviours and postintervention self-audit of physician skills and referral practice. The study design with parents of infants diagnosed with CP involves consent for data transfer to CP Registers1 and the NDIS (see Patient Consent Form online supplemental file 2).
Supplemental material
Study endpoints
The physician study endpoints are:
Key feature examination scores 6 months post-baseline (T3).
CPD Reaction Questionnaire at baseline (T1), immediately postintervention (T2) and 6 months post intervention (T3).
Clinical practice self-audit at baseline (T1) and 6 months post-intervention (T3).
Patient access requests to the NDIS (T1–T3).
Patient referrals to state and territory CP Registers1 (T1–T3).
The patient study endpoints are:
Age at time of access request to the NDIS and NDIS patient funding outcomes.
Age and predicted severity at time of referral to CP Registers.
A within-trial cost–consequence analysis will be conducted alongside this RCT. Resources (and associated costs) required to develop, implement and evaluate the interventions to the target population (paediatric physicians) will be recorded and accounted for in the cost side of the analysis. The consequence described will include selected endpoints of physicians and patients.
Study setting
The study is web-based and open for national recruitment in Australia.
Randomisation, allocation and blinding
After completing baseline assessments, physician participants will be randomised using central concealed random allocation with 1:1:1 allocation to intervention groups or control. The study statistician will generate the allocation table. Randomisation will be computer generated using REDCap Electronic Data Capture Tools (REDCap) hosted at The University of Sydney.80 81 Study participants in intervention groups will be naive to the e-learning intervention they receive, and both interventions contain interactive video design features. Scorers of key features examinations and researchers analysing the data will be blinded to group assignment.
Eligibility criteria
To participate, paediatric physician participants must identify as working in a clinical setting in Australia. In addition, paediatric physicians can identify as a consultant paediatrician or in a paediatric subspecialist practice field. In the Australian context, consultant paediatricians have completed dedicated paediatric medicine training components to meet standards for fellowship with the Royal Australasian College of Physicians within the Paediatrics and Child Health Division. General practitioners and allied health professionals are not eligible. In the Australian healthcare system, general practitioners are family physicians who provide universal care for individuals of all ages.
Patient participants eligible for this trial must comply with the following eligibility criteria: (1) infant with a clinical diagnosis of ‘cerebral palsy’ or ‘high-risk of cerebral palsy’ and (2) written consent from parent or person responsible for sharing of infant information with CP Registers1 and the NDIS.
Interventions
Intervention development
The e-learning intervention was developed in two phases: (1) design of the behaviour change intervention and (2) design of the web-based evidence-centred education and training intervention. Intervention design and development principles considered the evidence base and theoretical frameworks of behaviour change, learning processes and e-learning instructional design. The following section provides an in-depth description of the intervention and development process and demonstrates adherence to the Template for Intervention Description and Replication (TIDieR) reporting tool.82
Behaviour change intervention
The BCW was used in the intervention development following user guidelines.30 A guideline barrier analysis was completed in Australia, underpinned by the Theoretical Domains Framework (TDF),20 21 identifying paediatric physicians as a primary implementation target audience to expedite an early clinical CP diagnosis.31 Barriers to and facilitators of physician early diagnostic behaviours were explored through expert advisory committee meetings (n=20); parent focus groups (n=2); evaluation poll of members of the Australasian Academy of Cerebral Palsy (n=459); and evaluation of conference workshop and presentation feedback (n=10). Individual-level behaviour change interventions are a priority in diagnosing CP, as the clinical encounter between an individual health professional and patient remains fundamental to the delivery of diagnosis.
A Delphi study identified priority target behaviours for paediatric physicians to increase CP detection and diagnosis under 6 months of age in Australia.31 Consensus was reached on six target behaviours. Ranking identified the top three priorities: (1) referral for the General Movements Assessment, a standardised motor test with established predictive validity evidence in the early detection of CP2 83 84; (2) referral for the Hammersmith Infant Neurological Examination, a standardised neurological examination with established predictive validity evidence in the early detection of CP2 84 85; and (3) communication of the diagnosis with parents.2 31 86 The determinants of behaviour were considered using the Capability, Opportunity and Motivation-Behaviour domains framework and TDF.30 Comments on the Delphi surveys were organised into subthemes around the six ranked physician behaviours, which became the themes in the data. The themes and subthemes31 were then mapped onto the BCW framework to facilitate a behavioural analysis to direct intervention development, as displayed in table 1.
Target behaviours: results of thematic analysis linked with TDF and COM-B30 31
The BCW contains nine categories describing how interventions may change behaviour, referred to as intervention functions.30 Education, training, modelling, persuasion and enablement interventions were selected by the research team for this study using the BCW user guide and Capability Opportunity and Motivation Behaviour subcategories. Behaviour change techniques for paediatric physicians were selected using the BCW user guidelines.30 Fourteen behaviour change techniques were selected, with agreement from the research team and expert stakeholder group. An intervention strategy was drafted and refined by an iterative review process with the research team and expert stakeholder advisory group. Evaluation of the APEASE criteria (‘affordability, practicability, effectiveness, cost-effectiveness, acceptability, safety and equity’) is also recommended to guide intervention content and delivery.30 87
A web-based model of delivery met the APEASE criteria, was supported by guideline barrier analysis findings and was reached by consensus by the research team. Table 2 provides an overview of the web-based behaviour change intervention mapped to identified behaviour change techniques.30
Behaviour change intervention linked to behaviour change techniques30
Web-based evidence-centred education and training intervention
A cognitive task analysis88 with an expert advisory committee identified physician clinical decision-making skills and application of knowledge as priority constructs for education and training. Following cognitive task analysis, learning objectives were developed by the research team around the subtasks of clinical decision-making skills, identified target behaviours and knowledge in the early diagnosis of CP. Video virtual patients were selected as the instructional modality for the intervention, with an established evidence base for clinical decision-making skill outcomes in health professionals. Instructional design features with supporting evidence for change in behaviour, skills and patient outcomes in systematic reviews of simulation43 and virtual patients were utilised.45 46 The KLI framework60 guided the application of instructional design features to relevant knowledge and skills components, the complexity of learning processes and targeted instructional principles. Simple constant knowledge components,60 such as CP and predictive assessment definitions, may imply instructional approaches to optimise memory and fluency learning processes, for example, immediate feedback, recall and spacing. Instructional design features to facilitate simple learning processes included multiple-choice questions with immediate feedback, access to additional electronic knowledge and downloadable content resources. More elaborate and variable clinical decision-making incorporates more complex learning processes.60 Complex clinical decision-making is not developed through factual knowledge alone but by experience with context and case-specific mixed practice.59 Instructional design features to facilitate these complex learning processes included the use of variable mixed repeated-practice questions, worked examples, key feature problem-solving questions with immediate feedback, and a spaced virtual patient bank via monthly email reminders.
Cognitive theories were further considered within our education and training instructional design. Cognitive load principles were utilised to reduce extraneous load for users and to optimise intrinsic load for novice learners.41 89 Problem-Based Learning90 and concepts from Situated Learning Theory91 were also considered in creating an authentic web-based context of virtual patients.
Structure
The e-learning module is self-paced with an estimated completion time of 30–60 min (depending on electronic knowledge resources accessed). The first 8 min section is comprised of an opinion leader video introduction and a statement of objectives. In addition, interactive video features of knowledge questions, immediate feedback and optional links to an e-book and fact sheet resources are available. The second section comprises a narrative virtual patient92 clinical case using an interactive video format. Videos were recorded in a clinical setting using actors as patients to represent the real-life context of an outpatient clinic appointment. Interactive problem-solving and key feature questions, with immediate feedback, are used throughout the virtual patients. Questions are multiple-choice. Immediate feedback via text onscreen is provided. A virtual patient bank of mixed practice key feature cases comprises multimedia images, interactive videos and text onscreen.
An accessible menu is available at all times throughout the e-learning module and virtual patient bank, comprising downloadable resource fact sheets, an e-book and link to a web-based library of electronic knowledge resources: opinion leader videos, parent experience videos, podcasts, and lecture series.
Development
A development team was formed from Australian independent consultants and the research team. A user experience flow chart was developed to integrate data management, learning management system and evaluation requirements. Video production development involved storyboarding, actor auditions for patient actors and script copywriting with the research team. Two e-learning interventions were developed for the RCT to further the e-learning evidence base for comparing different instructional design. The instructional design features of the two e-learning modules vary in terms of adaptive content, navigation and multimedia domains.
Adaptive e-learning
Adaptive designs can facilitate a learner-centred approach using technology as tools to support the tailoring of instructional design. The complexity of instructional design is aligned with the complexity of the learning process, identified by cognitive task analysis and the participant’s expertise. Prior reviews of efficacy of adaptive e-learning, recommendations for health professional adaptive e-learning,47 end-user engagement and guidance from e-learning industry experts guided the tailoring of adaptivity method, types and timing.
In our adaptive instructional design of interactive videos and key features virtual patients, we utilised designed adaptation throughout the training according to participants’ responses to reflective questions, knowledge questions, key feature cases and problem-solving tasks. We used adaptivity of content through adapted text onscreen information and links to curriculum content. Adaptive navigation is used within the interactive video content, with an enforced path determined by participants’ responses. Adaptive tools with interactive multimedia are used, for example, with hotspot interactive videos and with the scoring of video assessments (General Movements Assessment and components of Hammersmith Infant Neurological Examination) for more expert participants who respond that they have undertaken training in these assessments. Direct instruction and modelling videos are shown for participants who have not undertaken training in these assessments, with text onscreen explanations of scores and scoring systems. Feedback to participant responses includes text onscreen knowledge information and adaptive video content displaying authentic, emotive patient reactions to their responses.
Non-adaptive e-learning
The non-adaptive intervention is linear in sequence, with no adaptivity of content, navigation, multimedia or tools. The non-adaptive e-learning module will look similar to the adaptive e-learning design with regards to video content and access to electronic resources; however users in the non-adaptive group will not be given choices in the videos or be provided with e-learning content determined by their prior responses (prior training and level of expertise or performance on e-learning questions).
Control
The waitlist control arm will not receive any e-learning intervention during the study period. However, participants who receive the control arm will be emailed a link to the adaptive e-learning and virtual patient bank at the end of the study.
Access
Access control is via REDCap.80 Availability is restricted to study participants. Participants enter the e-learning intervention via fixed URL encrypted with transport layer security (TLS), directed from REDCap on completion of baseline surveys. REDCap enrols participants in Moodle and receives completion information via their respective Application Programming Interfaces (APIs). Participants can access the e-learning module via their choice of desktop or tablet, from home or work. The e-learning utilises the Moodle V.3.8 platform (Moodle Pty Ltd, Perth, Australia) hosted by Amazon Web Services for the duration of the study.
The intervention is entirely web-based and asynchronous. The research team will be available to provide technical support via email or telephone.
Participants are given the option to enter the e-learning module immediately post completion of baseline REDCap surveys or at a later time through a link emailed to them. Participants will be asked to complete the e-learning module within 1 month. Email reminders will be sent up to three times at 1-week intervals if the e-learning module is not completed.
After 1 month, participants will be emailed a link to the bank of key feature cases. After that, participants receive a reminder email every month for 5 months, providing access to the bank of key feature cases.
Learning analytics
Both intervention groups have equal access to the e-learning module, a menu of electronic knowledge resources and a virtual patient bank of key feature cases. Participants can repeat the e-learning module and virtual patients as many times as they choose to. The total number of key feature virtual patients is 15.
Moodle collects information about each user with a timestamp for each action and resources that are accessed in the e-learning package (eg, e-book and resource downloads). The duration of each e-learning session (module and virtual patient bank) and the number of links accessed in a session will be recorded. As long latency periods may also indicate absence from the platform, we define time on a page of greater than 15 min as a threshold that would be deducted from the total session time, as described by Fontaine et al.93 The number of reminders sent to participants to complete the e-learning module or virtual patients is recorded in REDCap.
Adherence and fidelity
Participants in all groups will be asked to complete a survey to provide information on any early diagnosis of CP training or continuing professional development they accessed during the study period.
Tables 3 and 4 provide an overview of the two phases of intervention development, linking instructional design features with behaviour change techniques and identified learning processes.
Intervention development linking instructional design features to behaviour change techniques and intervention functions for modelling, persuasion and enablement
Intervention development linking instructional design features to complexity of learning process, cognitive task analysis for education and training interventions and differentiation of adaptive and non-adaptive designs
Outcomes
Physician participant data will be collected at three time points: baseline (T1), immediately postintervention if randomised to an intervention group (T2) and 24 weeks postintervention (T3). Patient outcome measures will be collected between (T1) and (T3) on return of patient consent Forms.
Primary outcome measures
Key-features examination of clinical decision-making
A web-based key features examination in the early diagnosis of CP has been developed by experts in CP and the key features methodology and piloted with practising physicians for psychometric reliability and acceptance. The examination target domains were mapped to priority behaviours and cognitive task analysis and comprise topic areas of CP risk factors; early detection using Prechtl’s General Movements Assessment, the Hammersmith Infant Neurological Examination and Neuroimaging; differential diagnosis; early intervention; and communication skills when communicating a diagnosis. The examination comprises 11 cases and 27 key feature questions with demonstrated reliability with Cronbach’s alpha 0.82 and mean item discrimination 0.34.
Early diagnosis CP CPD Reaction Questionnaire
The CPD-Reaction Questionnaire is a self-report instrument underpinned by a theoretical model combining the Theory of Planned Behaviour and Triandis Theory.94 CPD-Reaction scores are used to measure health professional clinical behavioural intentions in the context of CPD interventions.94 95 Constructs are evaluated using a 12-item Likert scale for ‘(1) intention, (2) social influence, (3) beliefs about capabilities, (4) moral norms and (5) beliefs about consequences.’94 CPD-Reaction developers have demonstrated score reliability (Cronbach’s coefficient for constructs ranging from 0.77 to 0.85) and moderate test–retest reliability (weighted kappa values 0.4–0.6).94 Developer guidelines were followed to adapt the CPD-Reaction Questionnaire for the early diagnosis of CP priority clinical behaviours targeted in this study.
De-identified clinical behaviour physician self-report
A self-assessment audit of clinical practice behaviours will provide evidence of physician clinical performance. A questionnaire has been developed (and piloted with three physicians) as a checklist against key priority physician clinical diagnostic behaviours retrospectively over the 6 months prior to study commencement. Responses are banded percentages (0%, 1%–20%, 21–40%, 41–60%, 61–80%, 81–100%), count numbers, yes/no, multi-choice questions and open text box.
Secondary outcome measures
Associations of postintervention key features examination scores, clinical-self audit and behavioural intentions with referrals to real-world CP databases will be explored with consenting patients. Patient outcomes data collected via physician participants and through data transfer methodology with real-world referrals was considered over other forms of patient outcome measures to decrease the burden on patients and associated distress around the time of diagnosis.
Access requests to NDIS
The Australian NDIS is a social and economic reform agenda supporting people with disabilities. The NDIS recognises children with a diagnosis of CP or high-risk of CP. Data are collected on all access requests made to the NDIS and data on participant demographics and outcomes for those eligible for the NDIS. For consenting parents, access requests made by physician study participants, NDIS eligibility and amount of funding provided per package will be evaluated, controlling for the physician intervention group, age at the time of access request and severity of CP. No current data are available about how many infants meet NDIS eligibility under 6 months of age.
Referrals to CP Registers
Each Australian state and territory has a CP Register. The Australian CP Register1 contains de-identified data uploaded from each state and territory Register. For consenting parents, data transfer of electronic notification of referrals to each state and territory CP Register will be evaluated, controlling for the physician intervention group, age at the time of CP diagnosis and severity of CP. Estimations from CP register data4 are indicative of approximately 126 new Australian babies receiving a diagnosis under 6 months of age within a 12-month period, predominantly involving severe forms of CP. However, with 600 new babies born with CP in Australia each year and with established high-risk infant follow-up (and pathways for referral for the General Movements Assessment and the Hammersmith Infant Neurological Examination),96 we anticipate capturing milder forms of CP earlier and >50% of infants eligible for a high-risk of CP diagnosis by 6 months corrected age.
Tertiary outcome measures
Evaluation of Technology-Enhanced Learning Materials: Learner Perceptions Short Form (ETELM)97
The ETELM is an evaluation instrument to assess learner perceptions of key quality web-based learning domains identified by educational frameworks and instructional design.97 Participants report their learning experience on a 7-point Likert scale (strongly agree to strongly disagree) in addition to free-text responses.97 Validity evidence is not available for the ETELM. Field leaders urge the use of established forms of evaluation over new forms to allow study comparisons to progress the field.97
Cost–consequence analysis
A cost–consequence analysis was considered an appropriate economic evaluation for this study and will be conducted with primary and secondary outcomes.98 The within-trial cost analysis is conducted from a funder’s perspective (including physicians and organisations) in the Australian context. A societal perspective will also be considered to understand external economic benefits of the interventions to patients and their families. We will use a 12-month time horizon to coincide with the follow-up period of the RCT. Costs and outcomes will be presented separately in their natural units in a tabular format. A reference group of stakeholders will be used to determine cost items. Costs will be analysed for the two e-learning interventions.
The cost ingredients method will be utilised to determine intervention costs.99 Prospective logs will be used for quantification of personnel costs of stakeholders. Cost measures include direct costs of intervention components, intervention development, personnel costs, information and communication technologies and website costs. All resource use associated with the intervention development and delivery will be identified.100 Any in-kind contributions will be identified. Research costs will not be included in the analysis.100 Cost items related to physician practice change, including direct and indirect health costs, will not be included in the evaluation. Costs will be inflated to Australian dollars according to study completion year.100 A 5% annual discount rate will be used in the base case, and 3% and 7% in the sensitivity analysis.100
Participant timeline
The schedule of assessments and interventions are provided below in table 5, and the CONSORT101 study flow diagram is provided in figure 2.
Schedule of assessments for early diagnosis Cerebral Palsy study
Flow diagram of study
Sample size
A total sample size of N=275 physicians (approximately 13% of practising physicians working in Paediatrics and Child Health102) enables detection of small magnitude effects (0.2) on primary outcome measure key feature examination scores between intervention comparators, both containing active learning strategies45 and large effects (0.8) compared with control45 47 with 90% power (α=0.05), allowing for 30% attrition. Sample size calculation was performed with software R103 and packages pwr104 and pwr2ppl.105 Sample size is based on a non-stratified sample. We intend to use sample size re-estimation at the interim analysis after 50% recruitment. Patients are recruited to this study through their physician. There is variability in potential physician participant clinical location and role which limits the modelling of expected number of patients recruited.
Recruitment
Recruitment will commence in 2022 and is expected to span a 6-month period. Multifaceted strategies to achieve adequate physician participant enrolment to reach the target sample size include social media campaigns, conference presentations, paediatric physician peak body newsletters; and email distribution by opinion leaders. Both consultant paediatricians with mixed caseloads and high-risk of CP subspecialists will be targeted.
Physician participants will recruit potential patients as part of routine clinical practice. There will be no recruitment procedure or advertisements for potential patients. Parents will receive a downloadable $50 gift certificate to thank them for their participation and time in completing the online Patient Consent Form.
Retention
The following strategies will be used to promote physician participant retention: (1) provision of supporting evidence for CPD credits with the peak body, Royal Australasian College of Physicians (completion certificate, summary reports of key features examination, adherence to guidelines and de-identified patient outcomes); (2) REDCap reminders; and (3) new key feature cases added to the virtual patient bank for each month of the study.
Participant withdrawal
Participants can withdraw from the study at any time until the analysis of data, at which point data will be pooled and it will not be possible to identify individual participants’ data. Participants who choose to withdraw from the study will not be penalised in any way.
Data management and access
All study data will be collected and stored using REDCap (Research Electronic Data Capture) hosted at The University of Sydney.80 81 REDCap forms will use forced completion. Data will be encrypted when exported from REDCap, and a data transfer team from the University of Sydney, blinded to the study, will conduct all data transfer processes with the NDIS and CP Registers. A research agreement between the research group and the National Disability Insurance Agency has been developed for the purposes of measuring outcomes of this study. After project completion, datasets will be exported and stored in The University of Sydney Research Data Store.
Statistical methods
SPSS Version 28.0.0.0 will be used. The cohort will be described and analysed according to demographic and baseline questionnaires. Descriptive statistics will summarise characteristics and factors measured. A comparison of intervention and control group outcomes between will be made. An analysis of covariance will be used to test the primary hypotheses. Subgroup analysis will examine the effects modified by the intervention arms. The predictors of behaviour and intervention effects will also be reviewed through exploratory analysis.
Generalised estimating equations will estimate intervention effectiveness on primary and secondary outcome measures and emerging patterns.106 Ordinal interval responses will be analysed using non-parametric methods (Mann-Whitney U test) for between-group comparisons. Multiple imputation approaches will be utilised for missing outcomes data.107 108 Analysis of the primary outcomes will be carried out according to intention to treat.109 All participants will be analysed in the group they were randomised to.
Data monitoring and safety
The risk of any adverse event is low for this study. However, any untoward occurrence will be considered an adverse event and recorded on an adverse event case report form and reported to The University of Sydney Human Research Ethics Committee and the Clinical Trials office as trial sponsor. A decision will be made about the safety to continue the trial following any adverse event by the chief investigators (IN, LM, KS, RNB) at the earliest convenience. Monthly online monitoring meetings will be conducted by a monitor external to the research group and monitoring reports sent to The University of Sydney Clinical Trials office. Standard clinical trials insurance for this trial is held by The University of Sydney.
Patient and public involvement
Parents of children diagnosed with CP have participated in all phases of intervention development, study design and evaluation framework.
Informed consent process
Potential physician participants will be provided with a Participant Information Statement and at the beginning of the REDCap preintervention survey will indicate that they have read the Participant Information Statement and consent to participate in the study.
Physician participants are provided with instructions on how to obtain parental consent to participate in this study: (1) physician participants will be emailed a downloadable Patient Information Statement and a link to an online Patient Consent Form; (2) physician participants will be asked to share information about this study with parents that they have been notified with a diagnosis of CP during the study period; and (3) potential parent participants will be invited to complete the online Parent Consent Form.
Potential patient participants will not be subject to coercion or pressure in deciding whether or not to participate in the study. The research group will answer questions from potential participants regarding the study or consent process via email or telephone.
Ethics and dissemination
Early Diagnosis CP is registered on the Australian New Zealand Clinical Trials Registry (ACTRN 12622000184774). The study received ethical approval from The University of Sydney Human Research Ethics Committee (Project number 2021/386). Protocol updates will be reported in the trial registration database and within publication of results.
Results of the study will be disseminated through: (1) the trial registration database, (2) conference abstracts and presentations, (3) peer-reviewed articles in scientific journals, (4) organisation and institution newsletters and media releases and (5) as per the Australian National Statement 3.1.65, directly to participants in a format that is appropriate and accessible to them.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors Conceptualisation: LM, IN, KS and RNB. Intervention development: LM, IN, KS, RNB and MB. Evaluation framework: LM, IN, KS, RNB, EF and MB. Writing-original draft preparation: LM. Statistical analysis plan design: LM and AW. Cost–consequence analysis: LM and KHN. Eligibility criteria and recruitment plan: LM, IN, KS, RNB and MB. Writing-review and editing: LM, IN, KS, RNB, EF, AW, MB and KHN. Supervision: IN, KS and RNB. All authors designed the study and have read, edited and approved the final manuscript and supplementary files.
Funding LM received NHMRC Postgraduate Scholarship (APP1169076) funding and scholarship support from The Australasian Cerebral Palsy Clinical Trials Network (AusCP-CTN, NHMRC Number 1116442). The e-learning intervention development is funded with support from The Cerebral Palsy Alliance.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.