Study design and aims of the study

We propose an explanatory sequential mixed-methods design to study treatment practices for OUD patients in New York State (figure 1). We employ a socioecological framework to study multiple factors that affect organisational responses to the exogenous shock caused by the COVID-19 pandemic. Under this model, clinics are subject to an interplay between external contingencies and internal capacity for responding to external demands. External demands can come from payers, regulators, communities, clients or environmental shocks, for example, an opioid epidemic or viral pandemic. Figure 1 depicts the framework that guides our study questions and analytic plan. Our study emphasises the clinic-level characteristics and the extent to which clinical practices and treatment retention may depend on key features of clinics (eg, telehealth, take-home medications) and the communities (eg, income, racial composition) in which clinics are embedded.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Explanatory sequential mixed-methods design in studying clinic-level organisational factors that affect outpatient addiction treatment practices and outcomes. CDS, Client Data System.

Setting

The New York State OASAS-regulated addiction treatment system serves more than 230 000 people per year in more than 900 detoxification, inpatient, residential and outpatient programmes. Approximately 580 outpatient clinics provide psychosocial and medical treatment (eg, buprenorphine for the treatment of OUD), of which 110 OTPs have special licences to provide methadone for the treatment of OUD. These clinics are spread across a variety of demographic areas from urban (eg, New York City, Buffalo) to suburban, exurban and rural.

Quantitative study methods

We adopted the SPIRIT 2013 Statement: Defining standard protocol items for clinical trials23 to summarise our approach (online supplemental material 1). We will conduct clinic-level analyses between March 2020 and February 2021. The proposed study period for the quantitative analysis is from September 2021 to April 2022. We will analyse and rank the clinics on clinical practice and retention in treatment. Table 1 lists sample measures and data sources. We will use a difference-in-differences approach to estimate changes in: (A) clinical practice changes (ie, telehealth frequency by patient, MOUD use by clinic) and (B) retention in treatment by clinic, while adjusting for individual (except race/ethnicity) and geographic characteristics. We will use a four-step method. First, we will model pre-to-within pandemic change through use of random-effects modelling to estimate change by clinic by introducing a fixed-effect variable for each clinic, adjusting for key individual and geographic characteristics. Second, we will estimate marginal effects by clinic to derive adjusted estimates of changes in the aforementioned clinical practice changes and outcome variables (ie, telehealth, MOUD, retention). Third, we will calculate cross-clinic Z scores for each of the clinic practice variables (ie, telehealth, type of counselling, etc) and then sum these by clinic to create composite rankings. Fourth, we will select clinics from the top and bottom quartiles for the composite rankings, ensuring equal representation among clinics that serve majority black and Latinx patients. This approach will enable us to address qualitative research aims.

Qualitative study methods

We used the Consolidated Criteria for Reporting Qualitative Studies24 tool to summarise our approach (online supplemental material 2). The proposed start date for site recruitment is March 2022, the end date for interviews is September 2022, and the qualitative analysis will be finished by December 2022.

We will draw potential sites from high-performing and low-performing clinics as computed in the analyses described in the quantitative aim. We plan to sample poorer performing programmes because they may help draw organisational-level comparisons and point to the development of clinic-level interventions to address the needs. We will rank and select 50 programmes for interviews: 25 from higher performing programmes (programmes with longer retention rates) and 25 from lower performing programmes (programmes with shorter retention rates). We will purposefully sample to achieve diversity in geographic location (areas that serve predominantly black and Latinx communities vs white patients) and programme structure (ownership; staff size). Last, we will conduct interviews with four key informants (programme director, clinical supervisor, counsellor and medical director) at each of the selected programmes. We will recruit staff who work on-site at the location in some capacity (vs a larger executive director for a group of treatment centres who are not working on-site). We will interview a small number of medical directors (compared with the programme director, clinical supervisor and counsellor roles) working with the selected programmes. Medical directors typically do not work full time on-site at any one clinic. Instead, they may work for a larger parent organisation or split their time between multiple clinics. Recruitment and analysis for these interviews will therefore be treated differently than that of the programme directors, clinical supervisors and clinic staff given that the medical directors are unlikely to have had the same manner of ‘on the ground’ experience during the pandemic. We will create a list of ‘primary contacts’ for sampling within target programmes using the OASAS provider directory.25 We will explain the goals and procedures of the study and request programme participation and ask for staff members who can best answer questions related to the COVID-19 response at the clinic. Examples of key informants who may possess relevant knowledge of strategies used by treatment clinics to improve outcomes include medical directors, doctors, other clinicians and other treatment clinic staff. We will emphasise the importance of gaining knowledge about effective strategies for improving outcomes for patients in the COVID-19 era. As an incentive to participate, we will offer programmes a report describing the overall study findings and lessons learnt. Additionally, we will provide a gift card of $40 value to each of the interviewees. If there is no response after 2 weeks, the research coordinator will call each contact by phone to confirm their receipt of the email. If we receive no response from the primary contacts by email or phone, we will reattempt to get in touch with them using a combination of email or phone. Because our primary contacts are high-level managers or directors within clinics, they are incredibly busy, hard to reach and used to a high number of contact attempts. Due to these particular difficulties reaching such high-level contacts, we will not limit the number of attempts per clinic. We will try contacting other staff if the primary contact does not answer after three attempts.

First, interview participants will be informed that their participation in this study is voluntary and that they may withdraw at any time. All participants will have the right to refuse participation without any compromise of their employment status or damaging impact on their reputation or relationships. We will obtain verbal consent prior to conducting the interview. We will use verbal consent to minimise the administrative burden for the interviewees. Participants are always given permission to not answer questions with which they feel uncomfortable. The confidentiality of the participants will be protected in several ways. All participants will receive an identifying number for coding and analysing data. Identifying code numbers will be kept in a separate file from completed transcripts. Data will be reported in clinic-level format only. Any other potentially identifying information will also be removed (eg, specific information about unusual characteristics or events that may be known in the community). All data will be kept in a locked file and will be accessible only to the research team. Participants’ names and identifying characteristics will not be used outside of the NYU research team.

We will conduct interviews using online conferencing technologies as in-person research activities continue to be suspended due to the COVID-19 pandemic. If audio-recording verbal consent is given, the interviews will be audio-recorded and transcribed. We anticipate that interviews will last between 45 min and 60 min. Participants will be encouraged to direct the discussion as much as possible, but the interviewer will use prompts and probes contained in the guide to clarify concepts, elicit detail and extend the narrative. After a few preliminary questions to assess the individual’s role within the organisation, we will begin each interview with an open-ended question, such as, ‘Please describe your program’s strategies for maintaining or improving care for patients since COVID-19’. Since interviews will occur after the period during which outcomes were calculated, we will ask participants to describe the care environment and specific strategies employed over the prior 1–2 years. We will also ask them to describe changes in care that have occurred during the intervening period (online supplemental material 3).

Immediately after each interview with clinic staff members, the researchers will make notes documenting the conversation and reflecting on key concepts and themes. The first few interviews will be used to develop the coding scheme, and additional transcripts will be compared with previously coded transcripts to ensure the consistent assignment of codes. As emerging concepts are identified, the existing coding structure will be adapted. Using socioecological model,26 data collection and analysis will be iterative, with the interview guide adapted to reflect emerging themes as noted previously. The research team will review the structure of codes to ensure that it is logical and comprehensive. After the team has reviewed the coding structure and all interviews have been reviewed in depth by two researchers, trained project staff will independently code all transcripts using the final coding scheme. Twenty per cent of the transcripts will be double coded to assess intercoder agreement. Any differences in coding will be discussed and resolved after discussion with the investigators. Data will be entered into a qualitative software programme (Atlas.ti) to facilitate organisation and retrieval. To further ensure data integrity, we will create an analysis audit trail to document all analytic decisions. Targeted analyses will examine the consistency of the data within sites and identify those themes that appear to discriminate between high-performing and low-performing OUD treatment programmes. The coding process will be conducted by at least three investigators (SC, TD and CN). When coding is complete, we will check for intercoder reliability, and the team will meet to review summaries of the qualitative results and refine hypotheses about the contextual factors and strategies that lead to better outcomes and those that might be barriers to success. We will use the socioecological framework to interpret the data using multilevel data collected from both quantitative and qualitative research. We will look for concordant and discordant results as well. We will conduct content analysis, detecting emergent codes, categories and themes.27 28 We will use a continuous process of coding, categorising and reviewing the raw data to reflect on the analysis at various points and make revisions (eg, recoding data). Weekly coding meetings will be held to review coding decisions, discuss discrepancies and check progress. We will use several methods to increase the trustworthiness of the analysis: sample to the point of saturation, keep an audit trail on study decisions and maintain a detailed codebook. Such a comprehensive method will allow us to detail the facilitators and barriers to implementing changes after COVID-19 at the clinic and provider levels, providing insights into how to better prepare for future pandemics.

Patient and public involvement

There will be no direct patient or public involvement in this research. However, we are addressing a critical need to understand the long-term implications of systemic changes emerging from COVID-19—particularly for clinics that primarily serve black and Latinx individuals with OUD. The clinics that will be recruited are generally highly burdened and under-resourced in relation to their mission, particularly among clinics serving the underserved populations. We have designed our data collection process to limit the burden to the director and staff. We believe that our study will inform clinic leadership as well as policymakers with targeted data on treatment services to support the need for fiscal and regulatory interventions to close these gaps.