Introduction Housing instability and homelessness are significant barriers to medical treatment for people living with HIV/AIDS. For these individuals, lack of stable housing and stigma is associated with insufficient access to care, poor adherence to medication and higher cost burdens to the healthcare system. This protocol reports on the efforts to evaluate Sanctum V.1.0, a hospice and transitional care home for adults with HIV/AIDS in Saskatoon, Saskatchewan, Canada. The current project was developed out of a need to identify how Sanctum V.1.0 produces varying programme outcomes to assist in endeavours to replicate the programme in other geographic locations.
Methods and analysis A realist evaluation will be conducted to explore how and why Sanctum V.1.0 is successful or unsuccessful, in which circumstances and for whom. Rather than explore the degree to which a programme is effective, realist evaluations seek to uncover mechanisms that explain processual links between programme inputs and outcomes. The completed first phase of the project involved the development of an initial realist programme theory. Phases 2 and 3 will consist of methods to test, refine and validate the initial theory using various data sources.
Ethics and dissemination Ethics approval was obtained from the institutional review board at the University of Saskatchewan on 2 July 2020. Results will be disseminated according to stakeholders’ desires.
- HIV & AIDS
- public health
- health services administration & management
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Realist evaluation allows for a comprehensive, contextual exploration of how and why a hospice and transitional care home for people living with HIV/AIDS in Saskatchewan, Canada is successful or unsuccessful.
Depicting links between contexts, mechanisms and outcomes in the initial realist programme theory posed a challenge in phase 1, although data collected from phases 2 and 3 may produce more nuanced connections.
Including a patient partner as part of the research team in phase 2 will inform culturally appropriate data collection and dissemination strategies based on programme experience.
The expected sample sizes for qualitative data are small, but align with realist modes of inference, and will be flexible to ensure theoretical saturation and adequate data to refine the initial realist programme theory.
Housing instability and homelessness are significant barriers to medical treatment for people living with HIV/AIDS (PLHIV).1–3 Although housing is a human right4 and a top service priority for PLHIV,5 obtaining stable, safe housing is challenging for these individuals. HIV-related stigma has prevented PLHIV from obtaining and maintaining safe places of residence.1 6 Qualitative accounts illustrate instances of property defacement, neighbourhood gossip and threatening letters tied to one’s HIV status.1 6 Furthermore, PLHIV are often unemployed, have difficulty affording housing costs and, therefore, may be unable to afford safe housing.7–12 PLHIV have also been denied housing due to their HIV status and related identities (eg, nonheteronormative sexual orientations).1 Feeling unsafe in a place of residence (eg, due to abusive relationships or drug activity) has resulted in housing instability for PLHIV.1
Lack of stable housing and HIV-related stigma is associated with poor access to care and adherence to medication.2 3 13–15 While antiretroviral therapy (ART) is known to suppress the virus, thus reducing mortality and morbidity, it is more effective with greater adherence.16 PLHIV who are homeless engage in HIV care less than those who are housed13 and tend to have poorer physical and mental health and higher mortality rates, compared with those with stable housing.12 17–19 Among a sample of people living with AIDS, the 5-year survival rate following diagnosis was significantly higher for those with stable housing (81%) than for those who were homeless (67%).18 Similarly, in a sample of PLHIV, those who were homeless had a significantly higher mortality rate (27 times higher) than those who were housed.19 This highlights the need to provide PLHIV with supportive housing to improve their access and adherence to healthcare and bolster overall health.
Healthcare systems are also impacted by the housing status of PLHIV, as cost burdens tend to be higher when PLHIV are unstably housed.20–22 Unstably housed PLHIV in Vancouver, British Columbia were 1.87 times more likely to use the emergency department and 2.71 times more likely to be classified as nonurgent, compared to those with housing.23 Addressing unstable housing among this population could reduce costs to the healthcare system. Others have found similar results.9 12 23–25
The beneficial effects of housing programmes for PLHIV are well established in the literature. These effects include increased chances of maintaining housing7 26 27; increased adherence to ART,28 29 thereby decreasing viral load and increasing CD4 count8 11 26 29 30; improved overall physical and mental health12 27 and decreased risk of death and improved lifespan.18 31 Positive effects of residential programming also extend to the healthcare system, by decreasing emergency room visits and hospital admission rates.12 22 32 The current paper will report on the efforts to evaluate a housing programme for PLHIV, including a summary of the completed phase and an overview of the next phases. The present project will explore why and how this specific programme functions to produce programme outcomes.
The programme under investigation is situated in Saskatoon, Saskatchewan, Canada. Saskatchewan currently has the highest provincial HIV diagnosis rate in Canada.33 In 2018, the national HIV diagnosis rate was 6.9 per 100 000 people, whereas the rate in Saskatchewan was 14.9 per 100 000 people.33 Furthermore, the diagnosis rate in Saskatoon was one of the highest in Saskatchewan at 17.0 per 100, 000 people.34
Within this setting, it is crucial to discuss HIV/AIDS as part of an intersection between ethnicity, injection drug use and housing status. Indigenous people in Canada are disproportionately affected by HIV/AIDS.35 In 2014, Indigenous persons accounted for approximately 16% of HIV/AIDS cases while comprising approximately 4% of the Canadian population.35–37 Disproportionate diagnosis is especially evident in Saskatchewan. Between 2009 and 2018, the proportion of new HIV diagnoses to Indigenous people ranged from 69% to 81%,34 despite making up 16.3% of the provincial population in 2016.38
Approximately 57% of HIV cases reported in Canada in 2018 included information about exposure. The top three exposure categories included: (a) gay, bisexual and other men who have sex with men (41.4%), (b) heterosexual contact (32.3%) and (c) injection drug use (18.3%).33 Reported HIV cases are almost four times as likely to be linked with injection drug use in Saskatchewan compared with overall cases in Canada.34 Data on injection drug use prevalence are currently unavailable in Saskatchewan, however, data from a 2012 Canadian survey show that 43.9% of the sample from Saskatchewan reported illicit drug use at some point in their life (cocaine, speed, methamphetamine, hallucinogens, extasy, salvia and/or heroine).39 Programming for PLHIV in Saskatchewan needs to be sensitive to these demographic distinctions to best support these individuals.
Sanctum Care Group is a nonprofit organisation in Saskatoon, Saskatchewan to support PLHIV who require assistance. The organisation operates under harm reduction and holistic care philosophies and takes a patient-oriented approach to care. While four programmes are offered by Sanctum Care Group (see table 1), this evaluation focusses on Sanctum V.1.0, which provides supportive care (eg, subacute, rehabilitative), palliative/hospice care (eg, pain control, symptom management) and respite care. Staff also assist those with unstable housing in achieving housing stability following their stay. Most clients at Sanctum V.1.0 are Indigenous, unstably housed and have other mental and physical health conditions (J. Patrick, personal communication, 3 September 2020). These services are comparable to other programmes in Canada, such as the Dr Peter Centre in Vancouver, British Columbia and Bruce House in Ottawa, Ontario.
Aim of the study
The current study is focused on the development, refinement and initial validation of a programme theory for Sanctum V.1.0. Unpublished evaluations of Sanctum V.1.0 by the Saskatoon Health Region in 2016 and 2017 examined the extent to which desired outcomes were achieved but did not examine how the programme itself led to these outcomes. The current project was, therefore, developed out of a need to identify how Sanctum V.1.0 leads to successful or unsuccessful outcomes and to assist in endeavours to replicate the programme in other geographic locations. A realist evaluation is the most suitable approach, as they explore: ‘What works for whom in what circumstances and in what respects, and how?’40 While group-based programming for PLHIV has been researched in Africa using realist approaches,41–43 the programmes were not housing based. Moreover, this study will be the first to report a realist evaluation of a residential programme for PLHIV in Canada. We aim to address the following research question: How and why is Sanctum V.1.0 successful or unsuccessful for which clients and in which circumstances?
Methods and analysis
Three research phases will contribute to creating, testing and refining a programme theory (see figure 1). This is an iterative process, and resulting theories are provisional.40 While phase 1 was completed in 2020, phases 2 and 3 will begin in the near future. Realist programme theories are formulated by constructing context–mechanism–outcome configurations (CMOC), which indicate how mechanisms influence outcomes for certain individuals in certain contexts.44 Contexts are conditions in which a programme is embedded that trigger and/or modify mechanisms.45 In realist evaluation, mechanisms are a combination of programme resources and programme participants’ reasoning.44 46 Mechanisms explain how things work in a causal manner by revealing why and how one event leads to another, opposed to simply demonstrating relationships between variables. Thus, mechanismic explanations move beyond black box explanations that are merely concerned with the degree to which programmes produce outcomes.44 47 48 Mechanisms are usually unobservable, change in response to contextual conditions and are means by which outcomes are produced.46 49 Outcomes may be intended or unintended and short term, intermediate term or long term.45
Researchers have proposed adaptations to CMOCs by adding intervention and actor components, resulting in an intervention–context–actor–mechanism–outcome configuration (ICAMOC).50 Presently, the three central care types offered by Sanctum V.1.0 represent the intervention, and actors refer to specific programme clients. This configuration illustrates how interventions are expected to trigger mechanisms for specific actors, within certain contexts, which is proposed to produce outcomes.50 This approach is advantageous, for it speaks to the ‘what works for whom’44 aspect of realist evaluation inquiry and makes programme resources more explicit.50 Overall, ICAMOCs provide a more nuanced, comprehensive representation of components that contribute to programme success or nonsuccess.
After ICAMOC components are identified, two modes of inference are used to test knowledge of real but fallible mechanisms: abduction and retroduction.51 Abduction is a process through which a set of concrete events or ideas are recontextualised through creative and imaginative approaches.51 Thus, a frequent step in realist evaluations is to come up with a theory (eg, based on hunches from the literature) about generative mechanisms driving programme outcomes. Retroduction is a process to determine what internal features characterise a phenomenon.51 Currently, this entails determining what it is about the programme that is constitutive of successful and/or unsuccessful outcomes. The following sections outline the methods chosen for constructing and testing a programme theory.
Phase 1: initial realist programme theory development
Realist programme theories outline which contexts trigger mechanisms that produce certain outcomes.44 The purpose of establishing an initial realist programme theory (IRPT) is to eventually test and refine the theory. Phase 1 consisted of four steps outlined in figure 1. Deductive content coding was used to identify ICAMOC components from four data sources (discussed below). Outcomes were coded as Sanctum V.1.0 goals and research results that were relevant to Sanctum V.1.0, mechanisms were coded when an explanatory factor (eg, programme resource or participant reasoning) could be tied to an outcome, and contexts were indicated when phrases described conditions that could prompt or alter the mechanism(s). Intervention components were identified as the three main care types offered at Sanctum V.1.0, and the actors represented Sanctum V.1.0 clients. A team member who was familiar with the literature on residential programming for PLHIV approved the list before moving onto the next steps.
First, a literature search was conducted in June 2020 to identify research involving programmes with residential components for PLHIV. Articles were eligible if they were published in English, included study participants who were PLHIV, and the study reported on the impacts of a residential programme. Three searches were completed using the Medline database and the following key terms: HIV, residential or housing facilities/programmes, homeless persons, drug use and programme evaluation. Four out of 43 resulting articles were eligible. More literature was needed to better understand ICAMOC components; thus, we completed an exploratory Google Scholar search and coded articles until we felt there was enough information to construct an IRPT, which is consistent with abduction.51 Four new articles were gathered from this search, resulting in a total of eight articles.
Second, a review of relevant programme documents was completed. Previous unpublished evaluations of Sanctum V.1.0 included resident demographics (context); qualitative and quantitative data indicating why Sanctum V.1.0 is valuable to residents (mechanisms) and outcomes related to medication adherence and health status. Six patient narratives generated by programme staff outlined circumstances surrounding the cases (contexts), client quotes describing what they liked about the programme (mechanisms) and general health outcomes for the individual. A client expectation form included harm-reduction expectations (context) and stated a programme goal (safe environment).
Third, detailed notes from a 1-hour meeting with Sanctum Care Group’s executive director included organisational contexts (eg, healthcare approaches), what makes Sanctum V.1.0 successful (contexts and mechanisms), what might lead to unsuccessful outcomes (eg, clients who do not adhere to organisational policies), what the programme offers clients (intervention) and who attends Sanctum V.1.0 (actors). Finally, a review of Sanctum V.1.0’s website verified previously identified ICAMOC components and revealed programme goals (outcomes).
Programmes are said to exist within social systems.40 Therefore, it is important to consider contexts and mechanisms that reside within varying levels of social reality. A social-ecological lens was applied, in which contexts and mechanisms were organised into micro, meso and macro levels. Individual level factors were deemed microlevel, while mesolevel elements represented interpersonal and organisational factors, and macrolevel elements were community based. This strategy will enable us to discuss the findings in relation to levels of social systems. Outcomes were labelled short term, intermediate term or long term. Table 2 lists CMOC components.
Next, we created a programme logic model (see figure 2), which illustrates how programme components are linked with programme outcomes.52 CMOC narratives were then constructed to connect contexts, mechanisms and outcomes organised in the first two steps. Based on information gathered from the first step, three main social science theories were chosen to guide development of CMOCs, including empowerment theory,53 54 self-efficacy theory55 and the information–motivation–behaviour model.56 57 The CMOCs needed to be condensed to form an IRPT. The first column of intermediate outcomes in the programme logic model (see figure 2) were most representative of Sanctum V.1.0’s main behavioural outcomes; thus, CMOCs were organised accordingly into three streams. The short-term, more cognitive, outcomes were translated into microlevel contexts, while the second column of intermediate outcomes was converted into outcomes in the CMOCs. Long-term outcomes are depicted as overarching programme goals expected to result from all CMOC streams. Intervention and actor components were added to the diagram, so that the IRPT reflected all ICAMOC components (see figure 3).
Phase 2: realist programme theory testing and refinement
Patient and public involvement
A patient partner from Sanctum V.1.0 will be included as part of the evaluation team in phases 2 and 3 to inform data collection strategies, participant recruitment, data interpretation and dissemination of findings. A patient-driven approach is crucial, as the client will be able to inform evaluators about appropriate research methods and language to use considering the vulnerable population. The following section outlines phases that have not yet been completed (see figure 1) and will be informed by the outcomes of phase 1.
Phase 2 methods
A mixed-method design will be employed by collecting and triangulating quantitative and qualitative data to test the programme theory. Although Sanctum V.1.0 mainly serves Indigenous clients, data collection will be open to Sanctum V.1.0 clients with any ethnic identity. Quantitative data will be collected from all Sanctum V.1.0 records, including total and annual summary data on admissions, programme completions, drop outs and staffing (a measure of resident support). Medication adherence will also be recorded, as Sanctum V.1.0 keeps individual charts that track medication use. This information will be analysed using descriptive statistics in SPSS (V.25) to provide relevant contextual information such as demographics and speak to the medication adherence outcome in the IRPT.
Qualitative data will be gathered from multiple stakeholders holding diverse expertise and perspectives related to contexts, mechanisms and/or outcomes.40 44 Using convenience sampling, we expect to conduct 10 in-depth realist interviews with Sanctum Care Group board members and staff and 10 with Sanctum V.1.0 clients, which aligns with realist data collection strategies.44 To explore potentially unsuccessful outcomes, we aim to recruit at least five individuals who have dropped out of the programme. Sample size will be flexible in order to reach theoretical saturation58 59 and ensure that there is enough information to revise the IRPT.
The interviews will be semistructured and the topic will be the realist programme theory.44 60 These interviews can be described as theory-refining,60 as the purpose is to clarify and improve the IRPT. Interviewers will be trained to use the teacher–learner process, in which the interviewer teaches the interviewee the Sanctum V.1.0 IRPT, and, in turn, the interviewee will reveal their ideas about how the programme functions.45 61 Questions suggested by Mukumbang and colleagues at this stage include those about programme resources, constraints and opportunities.61 Additionally, the authors suggest asking programme clients about how they make decisions regarding certain outcomes (eg, adherence to medical treatment).61 When creating the interview guide, we will include our explanation of each stream in the IRPT and follow-up with the participant by asking how they perceive Sanctum V.1.0 functions. We will include prompts to elicit information about barriers and facilitators to achieving programme goals and inquire about clients’ personal journeys regarding each ICAMOC.61 Interviews will be co-led by an evaluator and patient partner.
Using convenience sampling, we will also complete two sharing circles with approximately five clients each on the topic of the IRPT. Sharing circles are a culturally appropriate qualitative group method for participants who are Indigenous.62–64 Sharing circles are an open-ended storytelling method that aims to give voice to those who are marginalised and is associated with healing.62 63 The purpose is to generate knowledge that can be used to address the needs of Indigenous communities64 such as identifying which Sanctum V.1.0 activities and resources are effective for clients and why.
There are important differences between sharing circles and other qualitative group methods, such as focus groups. Practically, focus groups are often 90–120 min,65 whereas sharing circles are not bound by time.63 64 Furthermore, a moderator or facilitator leads focus groups, while a cultural person or elder leads sharing circles.62 64 65 Cultural persons and elders have expertise in cultural protocol and may have meaningful relations with participants who are encouraged, for they promote comfort and trust.62 64 In the present evaluation, an elder who is known to the Sanctum V.1.0 community will be hired to lead the sharing circle. There are also procedural differences. Cultural protocols dictate how sharing circles begin and end (eg, an elder may begin by acknowledging the creator and ancestors with a traditional prayer), which is not common in focus groups.63–66 Within a sharing circle, elders will pose an open-ended question to individuals, who respond one-by-one without interruption from others,63 64 while focus group moderators usually pose questions and anyone can respond in any order.65 66 Interviews and sharing circles will be recorded and transcribed for analysis.
There is no method of analysis that is favoured for qualitative realist data.40 Instead, evaluators must choose a method that will best illustrate IRPT acceptability. Strategies for realist qualitative data will be used to guide analysis.67 68 Transcripts will be coded using NVivo (V.1.0) according to current recommendations.68 Nodes (themes) will be created to represent ICAMOCs from the IRPT. Child nodes (subthemes) will be added to each node to symbolise the data source (eg, literature, programme documents, stakeholder meeting, website). Data from phase 2 will be added to the existing nodes. Memos will be used to document team decisions that were made to refine the IRPT.
Multiple procedures will be followed to ensure that the evaluation is conducted rigorously. Information regarding five indicators of administrative data quality outlined by Henson will be recorded in order to be transparent about the data from Sanctum V.1.0 records: completeness, timeliness, valid representativeness, consistency and integrity.69 Data quality will be commented on when the findings are reported. Once qualitative data are transcribed, the interviewer will scan the transcript for accuracy. To assess trustworthiness and rigour of the coded qualitative data, member checks will be completed by a team member and a patient partner. When analysis is complete, programme stakeholders will have a chance to review the interpreted results in phase 3.
Phase 3: realist programme theory validation
To validate the revised programme theory, meetings with Sanctum Care Group board members, staff and patient partners will take place. The purpose is to ensure that the resulting, provisional theory is an appropriate representation of Sanctum V.1.0. Stakeholders will provide feedback regarding how accurately the theory reflects their perceptions of the programme and the IRPT will be refined if necessary. If required, child nodes will be created to represent refined elements within the respective ICAMOC node.67 Memos will be attached to these child nodes to document the decision-making that occurred within the theory validation meeting.67 68 All results from phase 2 will be reported according to RAMSES II guidelines for realist evaluation.70
Ethics and dissemination
The current project obtained ethical approval from the University of Saskatchewan’s research ethics board on 2 July 2020. Each researcher completed online ethics training based on Canada’s tricouncil policy guidelines for conducting research with humans. Verbal and/or written consent will be obtained prior to all data collection and identifying data will be kept confidential. A partnership agreement between Sanctum Care Group and the evaluation team was signed prior to beginning the project, outlining project objectives, consent procedures, expected time contributions, data ownership and the publication processes. The results of this study will be disseminated according to stakeholders’ desires.
The aim of this evaluation is to establish a refined programme theory describing how and why Sanctum V.1.0 produces successful and/or unsuccessful outcomes for PLHIV in certain contexts. Realist evaluations are highly suitable for this type of inquiry, as they elicit testable, explanatory programme theories. The generative view of causation guides the realist evaluation process by focusing on creating programme theories that exhibit the ways in which underlying mechanisms contribute to programme outcomes.
There are three main advantages to the current project. First, realist evaluations move away from black box programme explanations and toward detailed accounts of why and how a programme operates.41 44 48 71 Second, the evaluation team is a multidisciplinary group with roots in psychology, programme evaluation and community health and epidemiology. Others explain multidisciplinary teams as favourable, for it helped them choose guiding substantive theories.69 Presently, a team with varied expertise will be valuable for uncovering different theories present in participant accounts that would have been missed otherwise. Second, a patient-driven approach will be taken by including a programme client on the evaluation team. Their experience will educate the evaluators on appropriate data collection methods and language use. Furthermore, their assistance synthesising data will help interpret and uncover programme theories.
Realist evaluations are not without challenges.72 First, there are no prescriptive steps for conducting realist evaluations.40 73 74 In the current evaluation, there were uncertainties about how to transform multiple ICAMOCs into a single depiction of a theory, especially as IRPTs have been represented in various ways.43 74–76 Choosing to focus on three central Sanctum V.1.0 outcomes in the programme logic model helped organise the hypothesised links between contexts, mechanisms and outcomes. Without prescribed guidelines, decisions also had to be made regarding data collection and analysis. In line with others’ suggestions, methods were chosen that would aid in revising the IRPT.40
Second, time is a practical challenge when pursuing realist evaluations.72 For example, the iterative nature of realist evaluations may inhibit forward movement in the evaluation.40 In the current study, this may be challenging if crucial ICAMOCs were missed in phase 1. However, multiple data sources were used to inform the IRPT, and feedback was obtained from the multidisciplinary team. Additionally, the refined programme theory will be informed by the patient partner and validated by stakeholders in phases 2 and 3.
This evaluation will contribute to Sanctum Care Group’s desire to extend their services to other communities in need of HIV/AIDS residential programming. With a programme theory, staff and board members will gain knowledge of contexts that are thought to generate mechanisms that foster successful outcomes. The contribution to realist evaluation literature is twofold. First, the study will provide preliminary evidence of a residential programme theory for PLHIV. Although others have completed work on group-based programming for PLHIV in Africa,41–43 the present evaluation will add by focusing on housing needs for PLHIV in Saskatchewan, where most PLHIV are Indigenous. Second, this protocol has implications for using programme logic models to create IRPTs. This strategy is not common in realist evaluations,77 yet it was useful for establishing outcome pathways, and provided the basis for creating the IRPT. As social programmes exist within varying layers of social reality,44 realist programme theories are key to uncovering the essence of programmes and, thus, enabling service providers to make informed decisions that suit the needs of their population. Unique strategies should continue to be shared in the realist research community that may improve the use of realist evaluations for programme staff, practitioners and policy makers.
Our team would like to thank Sanctum Care Group staff and clients for working closely with us to establish a realist evaluation plan for Sanctum 1.0. As well, we are extremely grateful for the time, knowledge, and insight provided by Sanctum Care Group’s executive director, Katelyn Roberts.
Contributors SF-H created the initial realist program theory and wrote the protocol manuscript. HA led the development of the initial grant, supervised the student, co-developed, reviewed and commented on the manuscript. TC assisted with the development of the initial grant proposal and reviewed and commented on the manuscript. KL supervised the student and reviewed and commented on the protocol manuscript. AA organised team meetings, coordinated the patient-oriented research training and reviewed and commented on the manuscript. GG is the principal investigator of the initial grant proposal, supervised the student and reviewed and commented on the protocol manuscript.
Funding The present study is funded by Canadian Institutes of Health Research grant number 425 031.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.