Process evaluation details including process data, outcome data, data type and source
| CFIR and RE-AIM constructs | Research aim | Process or outcome data | Data type and source |
| Outer setting | |||
| School contextual characteristics | What was the broad context of the schools in which the SPARX intervention was delivered? (aim 2) | School socioeconomic index School location (metropolitan/regional) | Publicly available information (ICSEA, GPS) |
| Inner setting | |||
| School organisational characteristics | What were the characteristics of the delivery environments (schools)? (aim 2) What were the barriers and facilitators that affected buy-in, delivery and student uptake? (aim 2) | School size, type, composition, funding School culture Implementation climate, relative priority, competing demands, leadership, school counsellor availability and level of support, networks and communication, school culture and climate, school readiness for implementation | Publicly available information (school size, type, funding) SSPESH (assesses school culture) Interviews with school staff Implementation Climate measures (staff and students) Relative Priority measure Competing Demands measure Interviews with school staff Checklists completed by trial manager Other administrative data including number of staff allocated to assist with delivery and consent process, level of communication with the research team |
| School leadership | How supportive of delivering SPARX were school principals, deputy principals and executives? (aim 2) | Level of support and buy-in from school leaders | Interviews with school staff |
| Individual characteristics | |||
| School staff | What were the characteristics (including attitudes, beliefs, traits) of school staff supporting the delivery of the intervention? (aim 2) How supportive of delivery were school staff who were involved on the ground? (aim 2) | Age, gender, current employment, role, etc Leadership, skills, motivations, expectations, self-efficacy, expectations, time available, knowledge and beliefs about the intervention, study buy-in | Demographics questionnaire Implementation Leadership Scale Interviews with school staff and study facilitators |
| Study facilitators | How well did study facilitators attending schools support the delivery of the intervention? (aim 2) | Age, employment Skills, self-efficacy or confidence, motivations and expectations | Demographics questionnaire Self-confidence measure Interviews with study facilitators |
| Students | What were the characteristics of young people that affected intervention uptake and effectiveness? (aim 2) | History of mental illness | Reported by year 8 students as part of the online FPS survey |
| Intervention characteristics | |||
| SPARX | Were there any barriers to intervention use? (aims 2 and 3) What do staff think about the efficacy and advantage of using the intervention? (aim 2) | Technical issues Evidence strength and quality, relative advantage | Logs of technical issues sent through schools, parents and participants IT data pertaining to technical problems Informal feedback provided by schools and research staff attending schools Relative Advantages measure Anticipated Benefits measure Interviews with school staff |
| Implementation processes | |||
| Normalisation and integration | How did school staff perceive the implementation processes? (aim 2) | Coherence, cognitive participation and collective action | NoMAD |
| Fidelity to the implementation strategy | To what extent was the intervention implemented as planned? (aim 2) | School delivery of the FP programme, including changes to the plan | Completed implementation checklists, emails and feedback forms Interviews with school staff |
| Implementation outcomes | |||
| Reach | What was the extent to which those who were eligible to receive SPARX used it? (aim 1) | Proportion of eligible participants who consented to participate; proportion who opened, used and completed the SPARX intervention Representativeness of the student sample | Administrative data about consent Digital analytic data including usage (app downloads, installs, opens), completion rate (number of modules completed) and time spent using SPARX Reported by year 8 students as part of the online FPS survey |
| Uptake | How many eligible schools participated in the study? Within those schools, how many staff supported the delivery of SPARX? (aim 1) | Proportion of eligible schools that were onboarded to the study; proportion of school staff (in intervention schools) who supported SPARX Representativeness of the sample | Administrative data Publicly available information about schools and self-report demographic data from school staff |
| Acceptability/appropriateness | How satisfied were participants with the intervention? (aim 1) How satisfied were school staff with supporting the intervention the FP programme (including SPARX)? (aim 1) | Acceptability/appropriateness of the intervention, expectations Acceptability/appropriateness of the FP programme (including SPARX) | Reported by year 8 students as part of the online FPS survey Informal conversations and feedback provided by year 8 students Implementation Appropriateness measure Interviews with school study staff Informal conversations and feedback provided by school staff |
| Across domains | |||
| How might the relationship between the intervention, the staff supporting the programme and context of each school shape variation in outcomes (implementation strength metric)? (aims 2 and 3) How might the school-level variation (in implementation fidelity and outcomes) affect clinical effectiveness outcomes (eg, self-reported depression)? (aim 3) What key lessons emerge from this study that can be generalised to the implementation of digital mental health programmes in schools more broadly? | |||
The process data and outcomes are mapped onto figure 2.
CFIR, Consolidated Framework for Intervention Research; FPS, Future Proofing Study; GPS, Global Positioning System; ICSEA, Index of Community Socio-Educational Advantage; IT, information technology; NoMAD, Normalisation Measure Development questionnaire; RE-AIM, Reach, Effectiveness, Adoption, Implementation, Maintenance; SSPESH, Survey of School Promotion of Emotional and Social Health.