Article Text

Download PDFPDF

Protocol for process evaluation of SMART Mental Health cluster randomised control trial: an intervention for management of common mental disorders in India
  1. Ankita Mukherjee1,
  2. Mercian Daniel1,
  3. Sudha Kallakuri2,
  4. Amanpreet Kaur1,3,
  5. Siddhardha Devarapalli2,
  6. Usha Raman4,
  7. Graham Thornicroft5,
  8. Beverley M Essue6,
  9. D Praveen2,7,
  10. Rajesh Sagar8,
  11. Shashi Kant9,
  12. Shekhar Saxena10,
  13. Anushka Patel11,
  14. David Peiris11,
  15. Pallab K Maulik1,7
  1. 1The George Institute for Global Health, New Delhi, India
  2. 2The George Institute for Global Health, Hyderabad, India
  3. 3Jindal School of Psychology & Counselling, O P Jindal Global University, Sonipat, Sonipat
  4. 4Department of Communication, University of Hyderabad, Hyderabad, India
  5. 5Centre for Global Mental Health Centre for Implementation Science, Health Service and Population Research Department, King's College London Institute of Psychiatry Psychology and Neuroscience, London, UK
  6. 6Institute of Health Policy, Management and Evaluation, University of Toronto Dalla Lana School of Public Health, Toronto, Ontario, Canada
  7. 7University of New South Wales, Sydney, New South Wales, Australia
  8. 8Department of Psychiatry, All India Institute of Medical Sciences, New Delhi, India
  9. 9Centre for Community Medicine, All India Institute of Medical Sciences, New Delhi, India
  10. 10Department of Global Health and Population, Harvard University T H Chan School of Public Health, Boston, Massachusetts, USA
  11. 11The George Institute for Global Health, UNSW Sydney, SydneyAustralia
  1. Correspondence to Dr Pallab K Maulik; pmaulik{at}


Introduction In India about 95% of individuals who need treatment for common mental disorders like depression, stress and anxiety and substance use are unable to access care. Stigma associated with help seeking and lack of trained mental health professionals are important barriers in accessing mental healthcare. Systematic Medical Appraisal, Referral and Treatment (SMART) Mental Health integrates a community-level stigma reduction campaign and task sharing with the help of a mobile-enabled electronic decision support system (EDSS)—to reduce psychiatric morbidity due to stress, depression and self-harm in high-risk individuals. This paper presents and discusses the protocol for process evaluation of SMART Mental Health.

Methods and analysis The process evaluation will use mixed quantitative and qualitative methods to evaluate implementation fidelity and identify facilitators of and barriers to implementation of the intervention. Case studies of six intervention and two control clusters will be used. Quantitative data sources will include usage analytics extracted from the mHealth platform for the trial. Qualitative data sources will include focus group discussions and interviews with recruited participants, primary health centre doctors, community health workers (Accredited Social Health Activits) who participated in the project and local community leaders. The design and analysis will be guided by Medical Research Council framework for process evaluations, the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework, and the normalisation process theory.

Ethics and dissemination The study has been approved by the ethics committee of the George Institute for Global Health, India and the Institutional Ethics Committee, All India Institute of Medical Sciences (AIIMS), New Delhi. Findings of the study will be disseminated through peer-reviewed publications, stakeholder meetings, digital and social media platforms.

Trial registration number CTRI/2018/08/015355.

  • Clinical trials
  • Protocols & guidelines
  • Suicide & self-harm
  • Depression & mood disorders

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • A strength of our study is its use of implementation science theories and guidelines for process evaluation to frame the study design.

  • This study combines data from an open-source medical record system with qualitative methods to understand trends, patterns and differences in outcome.

  • One limitation could be the overlap between the implementation team and the evaluation team.


India has a significant burden of mental disorders with an estimated 115 million people in need of mental healthcare.1 The National Mental Health Survey of India (2015–2016) found substance use, depressive disorders and anxiety disorders to be prevalent in about 10% of the population.1 Despite the significant burden, access to mental health services is severely limited and it is estimated that nearly 95% of individuals with common mental disorders (CMDs) are unable to access care in India2 leading to large treatment gaps. Studies report that in low-income and middle-income countries, the treatment gap for any mental disorder is between 75% and 85%.3 One study found that in low-resource settings such as India, only 1 in every 27 individuals with depression who recognised need for treatment, could access minimally adequate treatment from a trained mental health professional.4

This large treatment gap is due to several factors, on both demand and supply sides. Low awareness about mental health in the community and high level of stigma related to mental illness are key demand side factors for poor help-seeking for CMDs.5 On the supply side, several systemic barriers limit access to mental health services. Among these are the lack of a trained mental health workforce and absent/minimal mental health services at the primary care level, inadequate supply of psychotropic drugs at primary healthcare facilities and limited budget for mental healthcare.6

Our formative research has demonstrated that addressing both supply and demand side factors by conducting a community-based anti-stigma campaign and implementing a technology-enabled mental health services delivery model by primary health workers, has the potential to increase access to mental healthcare for those at risk of CMDs and reduction in depression and anxiety scores.7–10 In this research, task sharing by primary health workers helped facilitate the process, and technology was seen as an enabling factor in streamlining delivery of mental healthcare.10

Based on these findings, we developed Systematic Medical Appraisal, Referral and Treatment (SMART) Mental Health—a hybrid effectiveness-implementation cluster randomised controlled trial (cRCT) that is being implemented in two Indian states. The cRCT protocol is available elsewhere.11 The goal of SMART Mental Health is to reduce psychiatric morbidity due to psychological stress, depression and risk of self-harm (collectively referred to here as CMDs for the project) in individuals identified at high risk of these conditions. The coprimary outcomes are:

(1) The mean difference in Patient Health Questionnaire-9 (PHQ-9) scores at 12 months in people identified at high-risk of CMDs and (2) the difference in mean behaviour scores at 12 months in the total population.

In this paper, we outline the protocol for a process evaluation of the SMART Mental Health. Process evaluations provide important insights into how an intervention is implemented, leading to understanding what strategies either worked or did not work, explaining differences in outcome, and to gain insights into the experience of the target population for whom the intervention was designed. The aims of the process evaluation are to:

  1. Assess implementation fidelity and understand how the intervention was implemented.

  2. Understand perceptions about effectiveness and acceptability of intervention components by different stakeholders.

  3. Identify and explain facilitators of and barriers to implementation of the intervention.

  4. Explain variations in outcomes and unexpected consequences across sites.

  5. Explain any adaptations to the intervention during the study and their possible impact on the outcomes.

Methods and analysis

Theoretical framework

The process evaluation has been integrated into the cRCT design with an early formative study conducted to understand the feasibility of implementing the project components. It draws on multiple theories and frameworks (table 1). The Medical Research Council guidelines for process evaluation will provide an overall conceptual framework.12 According to this framework, the three broad areas of enquiry in a process evaluation are ‘ implementation’ (what is implemented and how); ‘mechanism of impact’—(how intervention produces change) and ‘context’—(how context affects implementation and outcomes). The framework also emphasises the need to spell out the key causal assumptions made in the programme theory.

Table 1

Theories to be used in the study

We will also use the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework13 to understand and describe the reach, effectiveness, adoption, implementation and maintenance of the intervention. The normalisation process theory (NPT)14 will help to understand the factors that influence integration and routinisation (becoming part of routine practice) of novel interventions in specific settings. NPT is grouped into four broad sub-constructs which influence normalisation or routinisation of novel interventions (coherence, cognitive participation, collective action and reflexive monitoring). RE-AIM and NPT will be used to evaluate how the programme was implemented to understand barriers to and facilitators of its routine use by primary health centre (PHC) doctors, and community health workers commonly known as ASHAs (abbreviation for Accredited Social Health Activists) and community participants.

Broad thematic areas of inquiry will include the context, implementation and mechanism of impact (table 2). Under the theme ‘context’, social, political, cultural and health system level factors impacting on implementation of the intervention will be explored. Differences between the sites, programme adaptations that were a result of change in context (eg, the COVID-19 pandemic), and site-specific barriers and facilitators that impacted the programme implementation and outcome will be enquired into. Under ‘implementation’ the process evaluation will assess the implementation of the two intervention components—anti-stigma campaign and mHealth based service delivery—using the RE-AIM parameters. It will also investigate the experiences of end users of the intervention. Finally, the process evaluation will explore the ‘mechanism of impact’ by critically examining any variations in outcomes or unexpected outcomes.

Table 2

Conceptual framework for process evaluation

Study setting

SMART Mental health is being implemented in 133 villages serviced by 44 randomly selected PHC in West Godavari district of Andhra Pradesh (South India) and Palwal and Faridabad districts of Haryana (North India).

Study design

The process evaluation will use a mixed-method multiple case study design with PHC clusters constituting a ‘case’. Up to eight case studies will be included. Each case will be selected purposively based on the principle of maximum variation in terms of health service delivery context, implementation challenges and outcomes.

Intervention description

The intervention comprises two key components; an antistigma campaign, and a technology-enabled mental health service intervention delivered through task sharing. The capacities of community health workers known as ASHAs and PHC doctors will be enhanced, by providing training in identifying and managing stress, depression, or suicide risk using a technology enabled electronic decision support system (EDSS).

In the preintervention phase, ASHAs will be trained to use the EDSS to screen individuals at high risk of stress, depression, self-harm or suicide using digital hand-held tablets. The tablets have two preinstalled, standardised screening and assessment tools—the PHQ915 16 and the Generalised Anxiety Disorder-7 (GAD-7)16 17 questionnaire. The screening process classifies whether participants are at high risk of CMDs based on the PHQ-9 and GAD-7 scores. Because a substantial proportion of people at risk of CMDs undergo natural remission over a period of time18 a second screening of all people initially identified at high risk is undertaken by the ASHAs within 6 months of the first screening to identify those who remain at ‘high risk’.

Additionally, a Knowledge Attitude Behaviour19 scale is administered to assess levels of stigma associated with mental disorders in the community, a Barrier to Access to Care Evaluation-Treatment Stigma20 questionnaire to assess stigma perceptions related to help-seeking for mental disorders and the EuroQol 5-Dimension-3 Level scale21 to assess quality of life. Questions related to history of psychiatric morbidity, availability of social network/support, treatment history and costs incurred in treatment (which will be used for economic evaluation) are also asked.

In the intervention phase, the two major intervention components will be implemented to those PHCs randomised to receive SMART Mental Health (figure 1). The logic model for how the intervention strategy is hypothesised to meet its aims has been provided (figure 2).

Figure 1

Study schema for smart mental health.11 ASHA, Accredited Social Health Activists; CMD, common mental disorder; GAD7, Generalised Anxiety Disorder−7; PHQ9, Patient Health Questionnaire-9.

Figure 2

Logic model of smart mental health. ASHAs, Accredited Social Health Activists; EDSS, electronic decision support system; IEC, Institutional Ethics Committee; PHCs, primary health centres.

The anti-stigma campaign uses audio-visual and print material tailored to the local community and delivered to both high-risk and non-high-risk individuals, with the aim of reducing negative knowledge, attitudes and behaviours related to mental disorders. The second component of the intervention is a technology enabled mental health service delivery model. An mHealth platform will be used for screening, diagnosis, referral and management of CMDs by community level health workers (ASHAs) and PHC doctors. Health workforce capacity building is a crucial input which will be embedded throughout the intervention. The ASHAs will follow-up individuals at high-risk of CMDs to support access to care from the PHC doctors. When the patient reaches the PHC, the doctors will use an EDSS based on WHO’s Mental Health Gap Action Programme Intervention Guide.22 Clinical data will be shared between the ASHAs and doctors using a secure cloud-based server. For follow-up care, the ASHAs will have an algorithm enabled priority listing that will provide them with a traffic-light system to prioritise and track the progress of individuals in her village. They will use this to follow-up patients, paying particular attention to the highest priority individuals and enquiring about their treatment adherence and mental well-being.

Following the intervention, a post-intervention phase up to 9 months will assess the sustainability of the intervention without external influence of the trial team. In this phase, the components of the intervention will be rolled out in the control arm too. Support for ASHAs and doctors by project staff will be minimal. Staff will assist ASHAs and doctors to resolve any technical problems with the tabs and provide initial support and troubleshoot any issues.

Control arm

In the control arm, ASHAs will be provided with the names of individuals at high risk of CMDs and they will support those individuals to seek care and provide them with relevant information of mental healthcare providers. PHC doctors in the control arm will be informed that there may be patients who may seek care for CMDs. The ASHAs and the doctors in the control arm will not be provided with access to the EDSS. The anti-stigma campaign will be delivered in a less extensive manner. Besides pamphlets and brochures, all the other anti-stigma components will be shared with the study participants. The live drama shows however, will not be conducted. Only videos of the drama will be shown. The ASHAs will draw on their existing training and experience on mental health to support individuals as needed.

Data collection

Quantitative data source includes analysis of the usage analytics extracted from the mHealth platform. This includes (1) user metrics from each tablet used by ASHAs and PHC doctors; (2) screening and treatment data about each high-risk individual in the intervention cohort; (3) data from the priority listing application (used by ASHAs) which provide information on treatment status and high-risk individuals who need to be followed up and (4) data from the interactive voice recorded system used to send messages to ASHAs and high-risk individuals (to facilitate treatment adherence and follow-up). These data will be used to assess reach, effectiveness, adoption, maintenance and service utilisation of the intervention.

Qualitative data will include key informant interviews and focus group discussions (FGDs) with PHC doctors, ASHAs, hospital administrators, service users and any other relevant stakeholders such as family members of service users and community leaders. The qualitative study data will explore perceptions of key stakeholders about the effectiveness and acceptability of intervention components and challenges in implementation. A detailed data collection plan has been discussed in table 3.

Table 3

Qualitative data collection plan

At the end of the post-intervention phase, a detailed comparative case study of two PHCs with be undertaken. It will include one PHC with high utilisation of EDSS and one with low utilisation. The case study will provide insights into barriers and facilitators in adoption and routinisation of EDSS and explain differences in levels of utilisation of mHealth in different PHC clusters. Interviews with all key stakeholders (including PHC doctors, ASHAs, supervisors associated with the PHC) will be used to develop the case study.

Data analysis

For quantitative data, basic descriptive analyses will be conducted. For qualitative data analysis interview transcripts will be read independently by two persons. A priori codes based on the conceptual framework (table 2) will be used to code the data. Additional thematic findings emerging through the data will be added to the coding framework. Data will be coded using NVivo V.12.0. Both qualitative and quantitative data across case studies will be triangulated to arrive at the findings.

Patient and public involvement

In the formative phase, community feedback was sought through FGDs, to make antistigma content culturally and contextually relevant. The study findings will be shared with the public. Findings will be disseminated through publication in peer reviewed journals, meetings, digital and social media platforms.

Ethics and dissemination

SMART Mental Health cRCT was approved by the George Institute for Global Health, India and the Institutional Ethics Committee (IEC), All India Institute of Medical Sciences (AIIMS), New Delhi. The trial has been registered (CTRI/2018/08/015355) with the Clinical Trial Registry-India, National Institute of Medical Statistics, Indian Council of Medical Research (ICMR). The project has received requisite approval the Health Ministry’s Screening Committee (HMSC), ICMR.

Trial status

At the time of writing this paper, the intervention phase of the trial had begun in both sites. Clinical Trials Registration was completed on 16 August 2020. Randomisation of clusters in Haryana was done on 21 September 2020 and in Andhra Pradesh on 4 December 2020. Key intervention components were being delivered in Andhra Pradesh and postintervention activities and follow-up surveys were being planned in Haryana.

Ethics statements

Patient consent for publication



  • Twitter @drpraveend

  • Contributors DPe, PKM and AP conceptualised the SMART Mental Health Trial with a process evaluation in mind. AM developed this process evaluation protocol with extensive and significant inputs from PKM and DPe. PKM and DPe commented on multiple drafts before sending a prefinal version to everyone listed as authors. The manuscript was reviewed and critical comments were provided by MD, SKl, AK, SD, UR, GT, BME, DPr, RS, SK, SS and AP. All authors provided critical intellectual inputs and comments to the draft. PKM leads the implementation of the trial in India along with MD, SD, SKl, AK and AM. All authors contribute to science or operationalising of the project as coinvestigators, or as steering committee members or researchers helping in the implementation of the trial. All authors have critically reviewed, commented and approved the final manuscript.

  • Funding This research is supported by an Australian National Health and Medical Research Council (NHMRC) Global Alliances for Chronic Disease Grant (APP1143911).

  • Disclaimer There is no role of the funding body in the design of the study and the conceptualisation and writing of the manuscript. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

  • Competing interests The George Institute has a part-owned social enterprise, George Health Enterprises, which has commercial relationships involving digital health innovations. PKM is partially supported through NHMRC/GACD grant (SMART Mental Health-APP1143911) and UKRI/MRC grant MR/S023224/1-Adolescents' Resilience and Treatment nEeds for Mental health in Indian Slums (ARTEMIS). MD, SD, SKl, AK, AM and DPe are partially or wholly supported through the SMART Mental Health NHMRC/GACD grant. DPe is supported by fellowships from the National Health and Medical Research Council of Australia (1143904) and the Heart Foundation of Australia (101890). AK is partially supported by Indigo Partnership (MR/R023697/1) award. GT is supported by the National Institute for Health Research (NIHR) Applied Research Collaboration South London at King’s College London NHS Foundation Trust, and by the NIHR Asset Global Health Unit award. GT is also supported by the Guy’s and St Thomas’ Charity for the On Trac project (EFT151101), and by the UK Medical Research Council (UKRI) in relation to the Emilia (MR/S001255/1) and Indigo Partnership (MR/R023697/1) awards.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.