Article Text

Protocol
Mixed-methods study to examine the response of opioid addiction treatment programmes to COVID-19: a study protocol
  1. Sugy Choi1,
  2. Rhea Naik1,
  3. Kamila Kiszko1,
  4. Charles Neighbors1,2,
  5. Thomas D’Aunno2
  1. 1Department of Population Health, New York University Grossman School of Medicine, New York City, New York, USA
  2. 2New York University Robert F Wagner Graduate School of Public Service, New York City, New York, USA
  1. Correspondence to Dr Sugy Choi; sugy.choi{at}nyulangone.org

Abstract

Introduction The COVID-19 pandemic is forcing changes to clinical practice within traditional addiction treatment programmes, including the increased use of telehealth, reduced restrictions on methadone administration (eg, increased availability of take-home doses and decreased requirements for in-person visits), reduced reliance on group counselling and less urine drug screening. This paper describes the protocol for a mixed-methods study analysing organisational-level factors that are associated with changes in clinic-level practice changes and treatment retention.

Methods and analysis We will employ an explanatory sequential mixed-methods design to study the treatment practices for opioid use disorder (OUD) patients in New York State (NYS). For the quantitative aim, we will use the Client Data System and Medicaid claims data to examine the variation in clinical practices (ie, changes in telehealth, pharmacotherapy, group vs individual counselling and urine drug screening) and retention in treatment for OUD patients across 580 outpatient clinics in NYS during the pandemic. Clinics will be categorised into quartiles based on composite rankings by calculating cross-clinic Z scores for the clinical practice change and treatment retention variables. We will apply the random-effects modelling to estimate change by clinic by introducing a fixed-effect variable for each clinic, adjusting for key individual and geographic characteristics and estimate the changes in the clinical practice changes and treatment retention. We will then employ qualitative methods and interview 200 key informants (ie, programme director, clinical supervisor, counsellor and medical director) to develop an understanding of the quantitative findings by examining organisational characteristics of programmes (n=25) representative of those that rank in the top quartile of clinical practice measures as well as programmes that performed worst on these measures (n=25).

Ethics and dissemination The study has been approved by the Institutional Review Board of NYU Langone Health (#i21-00573). Study findings will be disseminated through national and international conferences, reports and peer-reviewed publications.

  • substance misuse
  • organisation of health services
  • COVID-19
  • public health
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • This is an explanatory sequential mixed-methods study that focuses on organisational factors that affect treatment practices and retention during COVID-19.

  • The post hoc analyses will examine the racial disparities across clinics that have a large representation of black and/or Latinx patients.

  • The data linkage between the state treatment registry and Medicaid claims is one of the strengths of the study.

  • One of the limitations is that this study does not assess the impact of COVID-19 on primary care delivered services.

  • This study will provide best practices and methods at the organisational level to prepare for responding to future epidemics.

Background

The COVID-19 pandemic has revealed that public health capacity is critical and exposed the vulnerability of the addiction treatment services for patients who have opioid use disorder (OUD).1 2 The USA was already experiencing the opioid crisis before the pandemic, which has exacerbated the catastrophic effects of the pandemic.3

During the pandemic, the federal government agencies temporarily changed the regulatory policies regarding the medication for opioid use disorder (MOUD) administration. The Substance Abuse and Mental Health Services Administration (SAMHSA) relaxed regulations for take-home methadone at opioid treatment programmes (OTPs) when a patient was suitably stable.4 SAMHSA also lifted the mandatory in-person medical evaluation before buprenorphine initiation.5 The Drug Enforcement Administration allowed prescribers to initiate buprenorphine for new patients via telephone and/or audio-visual examination.6

The COVID-19 public health emergency measures also inevitably changed the treatment practices. Due to social distancing, in-person group counselling sessions were replaced with individual counselling sessions.7 Social distancing and the US Centers for Medicare and Medicaid Services’ approval for reimbursing telehealth services changed the standard OUD treatment delivery model to accommodate new forms of outpatient visits, including virtual behavioural therapies, medical screening and medication management for Medicaid-insured patients with OUD.8 9 As a result, treatment through individualised and group meetings via telehealth increased, reliance on drug screening decreased and new strategies for treatment engagement emerged.10 These care delivery changes have impacted revenues and, in many cases, organisational viability.11

Multiple public health measures have impacted treatment availability and accessibility as well, threatening the foundations of an already fragmented and fragile treatment system for OUD. Almost half of providers reported that they have reduced capacity for treatment and are contending with closing programmes.12 These challenges are affecting the more than 4 million people who are in treatment.13 In New York State, lockdowns and social distancing measures have changed the delivery of OUD treatment.14 The impact of the pandemic on racial and ethnic disparities cannot be underestimated for patients with OUD.15 There is concern that black and Latinx (Latino/Latina) patients—who are more likely to receive care in more poorly resourced clinics—received poorer quality care and have worse outcomes after the pandemic.16–18 Black and Latinx individuals often lack social resources (eg, residential stability) to weather communal shocks from a disaster and are more likely to suffer from challenges in access and retention in a treatment system under duress.19 20 Expansion of telehealth services during the pandemic raises concerns about disparities in access to technology for telehealth and adequate broadband services.21 22

The pandemic provides an opportunity to study organisational and clinical factors associated with substantial changes in treatment for OUD. Using a mixed-methods approach, we will study organisational-level factors that are associated with changes in treatment practices and retention, including treatment programmes that primarily serve black and Latinx patients. There are three goals of this study: first, to identify variation in clinical care practices—for example, telehealth frequency, retention—across 580 outpatient clinics that serve OUD patients throughout New York during the COVID-19 pandemic; second, to identify organisational factors that allow for rapid and effective clinical care adaptation during the pandemic; and third, to identify variation in treatment practices and retention associated with organisational factors affecting clinics that primarily serve black and Latinx patients with OUD.

Methods

Study design and aims of the study

We propose an explanatory sequential mixed-methods design to study treatment practices for OUD patients in New York State (figure 1). We employ a socioecological framework to study multiple factors that affect organisational responses to the exogenous shock caused by the COVID-19 pandemic. Under this model, clinics are subject to an interplay between external contingencies and internal capacity for responding to external demands. External demands can come from payers, regulators, communities, clients or environmental shocks, for example, an opioid epidemic or viral pandemic. Figure 1 depicts the framework that guides our study questions and analytic plan. Our study emphasises the clinic-level characteristics and the extent to which clinical practices and treatment retention may depend on key features of clinics (eg, telehealth, take-home medications) and the communities (eg, income, racial composition) in which clinics are embedded.

Figure 1

Explanatory sequential mixed-methods design in studying clinic-level organisational factors that affect outpatient addiction treatment practices and outcomes. CDS, Client Data System.

Aim 1: examine clinic-level variation in clinical care for OUD among outpatient addiction clinics during the COVID-19 pandemic

We will analyse Client Data System (CDS) and Medicaid claims from New York State to identify variation in treatment practice across 580 outpatient addictions treatment clinics between March 2020 and February 2021. CDS is an Office of Addiction Services and Supports (OASAS) treatment registry that all licenced providers of substance use disorder (SUD) treatment in the state of New York use to enter admission and discharge data. Data include demographics (eg, age, race/ethnicity and marital status), level of functioning (eg, housing status, health status and comorbid mental health), criminal justice status, recent history and frequency of substance use, and recent SUD treatment history. We will examine the following specific changes to treatment practices in outpatient addiction clinics: clinical delivery (eg, use of telehealth, a mix of group vs individual treatment, use of urine drug testing, changes related to MOUD administration such as remote initiation, greater flexibility in methadone self-administration), MOUD use and retention in treatment. Aim 1 will inform aim 2.

Aim 2: identify clinic-level factors associated with clinical practice changes and outcomes

We will examine organisational factors associated with treatment practices and retention during this period, including clinical practice changes and retention. We will also examine what factors predict the adoption of new clinical care approaches. Based on MOUD use and retention in treatment measures, we will rank clinics based on their performance during COVID-19. We will interview treatment clinic staff to contexualise the quantitative findings and understand which organisational factors and themes appear to discriminate between high, improving and low-performing clinics during the pandemic.

Aim 3: examine racial disparities in clinic-level measures

We will examine if there are differences in treatment retention between March 2020 and February 2021 among clinics that have a large proportion of black and/or Latinx patients with OUD. Similarly, we will explore the organisational characteristics of clinics that serve a large proportion of black and/or Latinx patients with OUD using qualitative data.

Setting

The New York State OASAS-regulated addiction treatment system serves more than 230 000 people per year in more than 900 detoxification, inpatient, residential and outpatient programmes. Approximately 580 outpatient clinics provide psychosocial and medical treatment (eg, buprenorphine for the treatment of OUD), of which 110 OTPs have special licences to provide methadone for the treatment of OUD. These clinics are spread across a variety of demographic areas from urban (eg, New York City, Buffalo) to suburban, exurban and rural.

Quantitative study methods

We adopted the SPIRIT 2013 Statement: Defining standard protocol items for clinical trials23 to summarise our approach (online supplemental material 1). We will conduct clinic-level analyses between March 2020 and February 2021. The proposed study period for the quantitative analysis is from September 2021 to April 2022. We will analyse and rank the clinics on clinical practice and retention in treatment. Table 1 lists sample measures and data sources. We will use a difference-in-differences approach to estimate changes in: (A) clinical practice changes (ie, telehealth frequency by patient, MOUD use by clinic) and (B) retention in treatment by clinic, while adjusting for individual (except race/ethnicity) and geographic characteristics. We will use a four-step method. First, we will model pre-to-within pandemic change through use of random-effects modelling to estimate change by clinic by introducing a fixed-effect variable for each clinic, adjusting for key individual and geographic characteristics. Second, we will estimate marginal effects by clinic to derive adjusted estimates of changes in the aforementioned clinical practice changes and outcome variables (ie, telehealth, MOUD, retention). Third, we will calculate cross-clinic Z scores for each of the clinic practice variables (ie, telehealth, type of counselling, etc) and then sum these by clinic to create composite rankings. Fourth, we will select clinics from the top and bottom quartiles for the composite rankings, ensuring equal representation among clinics that serve majority black and Latinx patients. This approach will enable us to address qualitative research aims.

Table 1

Measures and data sources

Qualitative study methods

We used the Consolidated Criteria for Reporting Qualitative Studies24 tool to summarise our approach (online supplemental material 2). The proposed start date for site recruitment is March 2022, the end date for interviews is September 2022, and the qualitative analysis will be finished by December 2022.

We will draw potential sites from high-performing and low-performing clinics as computed in the analyses described in the quantitative aim. We plan to sample poorer performing programmes because they may help draw organisational-level comparisons and point to the development of clinic-level interventions to address the needs. We will rank and select 50 programmes for interviews: 25 from higher performing programmes (programmes with longer retention rates) and 25 from lower performing programmes (programmes with shorter retention rates). We will purposefully sample to achieve diversity in geographic location (areas that serve predominantly black and Latinx communities vs white patients) and programme structure (ownership; staff size). Last, we will conduct interviews with four key informants (programme director, clinical supervisor, counsellor and medical director) at each of the selected programmes. We will recruit staff who work on-site at the location in some capacity (vs a larger executive director for a group of treatment centres who are not working on-site). We will interview a small number of medical directors (compared with the programme director, clinical supervisor and counsellor roles) working with the selected programmes. Medical directors typically do not work full time on-site at any one clinic. Instead, they may work for a larger parent organisation or split their time between multiple clinics. Recruitment and analysis for these interviews will therefore be treated differently than that of the programme directors, clinical supervisors and clinic staff given that the medical directors are unlikely to have had the same manner of ‘on the ground’ experience during the pandemic. We will create a list of ‘primary contacts’ for sampling within target programmes using the OASAS provider directory.25 We will explain the goals and procedures of the study and request programme participation and ask for staff members who can best answer questions related to the COVID-19 response at the clinic. Examples of key informants who may possess relevant knowledge of strategies used by treatment clinics to improve outcomes include medical directors, doctors, other clinicians and other treatment clinic staff. We will emphasise the importance of gaining knowledge about effective strategies for improving outcomes for patients in the COVID-19 era. As an incentive to participate, we will offer programmes a report describing the overall study findings and lessons learnt. Additionally, we will provide a gift card of $40 value to each of the interviewees. If there is no response after 2 weeks, the research coordinator will call each contact by phone to confirm their receipt of the email. If we receive no response from the primary contacts by email or phone, we will reattempt to get in touch with them using a combination of email or phone. Because our primary contacts are high-level managers or directors within clinics, they are incredibly busy, hard to reach and used to a high number of contact attempts. Due to these particular difficulties reaching such high-level contacts, we will not limit the number of attempts per clinic. We will try contacting other staff if the primary contact does not answer after three attempts.

First, interview participants will be informed that their participation in this study is voluntary and that they may withdraw at any time. All participants will have the right to refuse participation without any compromise of their employment status or damaging impact on their reputation or relationships. We will obtain verbal consent prior to conducting the interview. We will use verbal consent to minimise the administrative burden for the interviewees. Participants are always given permission to not answer questions with which they feel uncomfortable. The confidentiality of the participants will be protected in several ways. All participants will receive an identifying number for coding and analysing data. Identifying code numbers will be kept in a separate file from completed transcripts. Data will be reported in clinic-level format only. Any other potentially identifying information will also be removed (eg, specific information about unusual characteristics or events that may be known in the community). All data will be kept in a locked file and will be accessible only to the research team. Participants’ names and identifying characteristics will not be used outside of the NYU research team.

We will conduct interviews using online conferencing technologies as in-person research activities continue to be suspended due to the COVID-19 pandemic. If audio-recording verbal consent is given, the interviews will be audio-recorded and transcribed. We anticipate that interviews will last between 45 min and 60 min. Participants will be encouraged to direct the discussion as much as possible, but the interviewer will use prompts and probes contained in the guide to clarify concepts, elicit detail and extend the narrative. After a few preliminary questions to assess the individual’s role within the organisation, we will begin each interview with an open-ended question, such as, ‘Please describe your program’s strategies for maintaining or improving care for patients since COVID-19’. Since interviews will occur after the period during which outcomes were calculated, we will ask participants to describe the care environment and specific strategies employed over the prior 1–2 years. We will also ask them to describe changes in care that have occurred during the intervening period (online supplemental material 3).

Immediately after each interview with clinic staff members, the researchers will make notes documenting the conversation and reflecting on key concepts and themes. The first few interviews will be used to develop the coding scheme, and additional transcripts will be compared with previously coded transcripts to ensure the consistent assignment of codes. As emerging concepts are identified, the existing coding structure will be adapted. Using socioecological model,26 data collection and analysis will be iterative, with the interview guide adapted to reflect emerging themes as noted previously. The research team will review the structure of codes to ensure that it is logical and comprehensive. After the team has reviewed the coding structure and all interviews have been reviewed in depth by two researchers, trained project staff will independently code all transcripts using the final coding scheme. Twenty per cent of the transcripts will be double coded to assess intercoder agreement. Any differences in coding will be discussed and resolved after discussion with the investigators. Data will be entered into a qualitative software programme (Atlas.ti) to facilitate organisation and retrieval. To further ensure data integrity, we will create an analysis audit trail to document all analytic decisions. Targeted analyses will examine the consistency of the data within sites and identify those themes that appear to discriminate between high-performing and low-performing OUD treatment programmes. The coding process will be conducted by at least three investigators (SC, TD and CN). When coding is complete, we will check for intercoder reliability, and the team will meet to review summaries of the qualitative results and refine hypotheses about the contextual factors and strategies that lead to better outcomes and those that might be barriers to success. We will use the socioecological framework to interpret the data using multilevel data collected from both quantitative and qualitative research. We will look for concordant and discordant results as well. We will conduct content analysis, detecting emergent codes, categories and themes.27 28 We will use a continuous process of coding, categorising and reviewing the raw data to reflect on the analysis at various points and make revisions (eg, recoding data). Weekly coding meetings will be held to review coding decisions, discuss discrepancies and check progress. We will use several methods to increase the trustworthiness of the analysis: sample to the point of saturation, keep an audit trail on study decisions and maintain a detailed codebook. Such a comprehensive method will allow us to detail the facilitators and barriers to implementing changes after COVID-19 at the clinic and provider levels, providing insights into how to better prepare for future pandemics.

Patient and public involvement

There will be no direct patient or public involvement in this research. However, we are addressing a critical need to understand the long-term implications of systemic changes emerging from COVID-19—particularly for clinics that primarily serve black and Latinx individuals with OUD. The clinics that will be recruited are generally highly burdened and under-resourced in relation to their mission, particularly among clinics serving the underserved populations. We have designed our data collection process to limit the burden to the director and staff. We believe that our study will inform clinic leadership as well as policymakers with targeted data on treatment services to support the need for fiscal and regulatory interventions to close these gaps.

Discussion

This proposed research will identify emerging clinic-level changes in treatment practices and retention in treatment for OUD. The study will examine short-term and intermediate-term effects of clinical changes implemented as a result of COVID-19. The pandemic presents a unique circumstance under which a fragmented and fragile system of care must adapt to unprecedented changes. By combining findings from analyses of large administrative data with lessons about adapting to an unprecedented emergency situation gleaned from qualitative interviews of successful and struggling clinics, we will build scientifically supported knowledge to inform policy at a larger systemic level. Identifying organisational factors of high-performance and low-performing clinics will help inform state, local and provider policy. This study will identify changes in quality of care for SUD, including treatment retention and clinical changes such as an increase in telehealth visits and flexibility in take-home doses. This study will inform on the organisational context in which successful practice change is likely to occur. In particular, it will highlight the variation and gaps in care for clinics that mostly serve black and Latinx patients. We plan to disseminate the results to the participants as well as the clinic directors in NYS to provide useful benchmarks and recommendations.

We hope to draw on the strengths of both quantitative and qualitative methods, but the proposed research is not without limitations. One limitation is that the study will use data from a single state. While the study aims to focus on organisational factors and the impact of COVID-19 on treatment for SUD in New York State, we will be cautious in making claims about the generalisability of findings to other states. We note that New York is geographically large, has a diverse population of approximately 19 million people and a broad mix of population density from urban to rural. Study findings regarding population and geographical characteristics will be presented so other states can tailor lessons learnt to their own circumstances.

The study focuses on the specialty SUD treatment system. The proposal examines organisational factors affecting changes in treatment in response to COVID-19 within state-regulated SUD treatment clinics. These clinics treat more than 100 000 individuals with OUD each year in New York. The study does not address treatment outside of these settings (eg, OUD treatment by office-based MOUD prescribers), which represents a smaller but growing portion of care for SUD. The specialty treatment system is critically important due to the size of the population served, the variety of SUDs treated for which there are limited alternative options and its key role in providing services to socially disenfranchised and vulnerable individuals. Additionally, recruitment and interviews are subject to selection and/or non-response bias.

Ethics and dissemination

The study was approved by the NYU Langone Institutional Review Board (approved 2021; i21-00573). This work is funded by the Foundation of Opioid Response Efforts. Funding sources had no role in the writing of this manuscript or the decision to submit it for publication. This research will provide summary reports that describe clinic characteristics (eg, retention) statewide and by region that will offer useful benchmarks for the clinic directors. We plan to publish the results in peer-reviewed journals and communicate the findings via webinars and conferences to all relevant stakeholders.

Ethics statements

Patient consent for publication

Acknowledgments

We would like to express our gratitude to Shazia Hussain and Yichuan Wang for conducting preliminary data analysis.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors SC, CN and TD'A conceptualised the project and obtained funding. SC and RN drafted an initial version of this paper. CN, TD'A and KK provided edits and contributed to all sections. All authors read and approved the final manuscript.

  • Funding This work was supported by Foundation for Opioid Response Efforts.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.