Objectives Our objective was to validate a Social Vulnerabilities Survey that was developed to identify patient barriers in the following domains: (1) salience or priority of health; (2) social support; (3) transportation; and (4) finances.
Design Cross-sectional psychometric study.
Questions for one domain (health salience) were developed de novo while questions for the other domains were derived from national surveys and/or previously validated questionnaires. We tested construct (ie, convergent and discriminative) validity for these new questions through hypothesis testing of correlations between question responses and patient characteristics. Exploratory factor analysis was conducted to determine structural validity of the survey as a whole.
Setting Patients admitted to the inpatient internal medicine service at a tertiary care hospital in Calgary, Canada.
Participants A total of 406 patients were included in the study.
Results The mean age of respondents was 55.5 (SD 18.6) years, with the majority being men (55.4%). In feasibility testing of the first 107 patients, the Social Vulnerabilities Survey was felt to be acceptable, comprehensive and met face validity. Hypothesis testing of the health salience questions revealed that the majority of observed correlations were exactly as predicted. Exploratory factor analysis of the global survey revealed the presence of five factors (eigenvalue >1): social support, health salience, drug insurance, transportation barriers and drug costs. All but four questions loaded to these five factors.
Conclusions The Social Vulnerabilities Survey has face, construct and structural validity. It can be used to measure modifiable social vulnerabilities, such that their effects on health outcomes can be explored and understood.
- statistics & research methods
- internal medicine
- general medicine (see internal medicine)
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
The Social Vulnerabilities Survey is a newly developed questionnaire that meets an important gap, being one of the few tools to identify modifiable social vulnerabilities that may affect the ability of patients to maintain their health.
The domains covered by the survey are those identified by patients as barriers after hospital discharge in prior qualitative studies of patients facing socioeconomic disadvantage.
This study uses multiple methods to comprehensively assess validity of the survey—including face, construct (convergent, discriminant and discriminative) and structural validity.
Validity was assessed only in the inpatient setting at a single large tertiary care hospital, which may limit generalisability.
Hospital discharge signifies a particularly vulnerable time for adverse medical events, with up to 35% of patient being readmitted within 3 months.1 2 Hospital readmissions may be attributable to patient, provider or organisational factors.3 Of these, patient characteristics appear to account for most of the variation in readmission rates across institutions,4 and patient-level interventions are therefore the focus of multidisciplinary efforts to improve post-discharge outcomes.5
Self-management of chronic conditions after hospital discharge requires adequate knowledge, planning and ability on the patient’s part,6 and can therefore be affected by the social determinants of health and more downstream social vulnerabilities (eg, transportation, financial and social support barriers).7–10 In a recent study, patients who reported barriers due to at least two measures of social determinants of health were twice as likely to have preventable readmission than those without these barriers, with the majority of patients reporting the need for more general (non-medical) assistance to stay well after discharge.11 Similarly, in a study of over 13 million patients, there appeared to be a dose-response relationship between health-related social needs and hospital readmissions.12 Recognising the importance of addressing social determinants in improving patient care and health equity, the American College of Physicians recommends improved identification of social determinants of health and their downstream social vulnerabilities.13
Despite their importance, social vulnerabilities are rarely identified or studied, hampering the development of discriminative models to predict hospital readmission and effective interventions to mitigate them.5 14 The main barriers to measuring social vulnerabilities in hospitalised patients are that: (1) they are not routinely collected or available in registry or administrative data, and (2) there is a lack of widely accepted, validated questionnaires. Though Greysen et al created a 22-item survey to measure patient understanding,7 patient engagement with care and barriers to self-care in the post-discharge period, this survey is not specific to patient-level social vulnerabilities (ie, it includes provider and organisation factors), and does not sufficiently detail tangible barriers that can be targeted by interventions. For example, patients are asked whether they had difficulty following a recommended diet, or difficulty taking medications, but there are no other questions in the survey that delve into why patients face such difficulties.
Modifiable social vulnerabilities are the barriers to healthcare access that can be intervened on to improve disease prevention and screening, promote early presentation to care and improve access, uptake and adherence to treatment.15 A validated survey that identifies these social vulnerabilities is essential to identify risk factors for hospital readmissions, in identifying patients at risk for readmission and in developing both patient and population-level interventions that directly address these risk factors. In this study, we describe the development and validation of the Social Vulnerabilities Survey (SVS) in a cohort of medical inpatients in Calgary, Canada.
Development of SVS
The SVS (table 1; online supplemental appendix 1) was developed to explore the role of social vulnerabilities in a patient’s ability to access care and self-manage chronic conditions. It covers four domains of social vulnerabilities, which were selected based on prior qualitative studies of post-discharge barriers in patients with low socioeconomic status.9 10 These domains are: transportation barriers, financial barriers, poor social support and low salience of health due to competing priorities.9 10 Three of these four domains have been previously explored in national surveys or questionnaires validated in international populations.16–18 Questions within these three domains were therefore obtained from these prior sources where available, with items being selected through discussion and consensus of three members of the study team (KLT, M-JS and WAG), and adaptations made based on patient feedback (see Results section):
Four questions relating to having a licence, modes of transportation and frequency of driving (Questions 1, 2, 4 and 5) were obtained directly from the Canadian Community Health Survey—Healthy Aging Questionnaire.17 Two questions relating to travel time to a family doctor’s clinic (Question 6) and travel-related barriers in accessing health services (Question 9) were taken from the Barriers to Care for People with Chronic Conditions (BCPCHC) Survey.16 Two-related questions (Questions 7 and 8) were added to ask about travel time to other health services such as a walk-in or urgent care clinic, and to a laboratory for blood work, respectively. A question about vehicle ownership (Question 3) was added in light of evidence suggesting its associations with health and ability to cope with the demands of illness.19 20
Questions were taken directly from the modified Medical Outcomes Study Social Support Survey,18 assessing the domains of emotional and instrumental social support (Questions 17–24). A single question asking whether patients live alone (Question 25) was added, due to its association with healthcare utilisation, and to provide context to the relative importance of social support based on living arrangements.21 22
Financial barrier questions about drug insurance (Question 26), not taking medications due to cost (Questions 28 and 29), out-of-pocket medication costs (Questions 30 and 31) and barriers to care due to inability to take time off work (Question 32) were adapted from the BCPCHC Survey. Two new questions were added—one asking for the percentage of drug costs paid out-of-pocket (Question 27), to provide context to patient-reported absolute medication costs, and another asking about affordability of childcare as a barrier to healthcare access (Question 33) due to it being a frequently endorsed barriers in the low-income, non-elderly patient population.23
The fourth domain (health salience in the context of competing priorities) has not previously been studied, with no prior questions or questionnaire designed to explore this concept. Seven questions were created for this domain. The content for Questions 10–13 (which asks whether competing priorities results in ability to self-manage health and access care) and Question 16 (which asks participants to identify competing priorities) are based on the previously-mentioned qualitative studies.9 10 Questions about perceived importance of health and ability to keep healthy were added (Questions 13 and 14), given the importance of these health beliefs on patient willingness and ability to prioritise health.24
Information about the patient’s health was obtained through a separate background information survey (online supplemental appendix 2), which was administered along with the SVS. It comprised of 37 questions asking about socio-demographic characteristics, function based on Older Americans Resources and Services questionnaire,25 stress using the Perceived Stress Scale,26 health beliefs,24 self-rated health27 and prior healthcare use.
We assessed acceptability, feasibility, face validity and structural validity of the SVS as a whole.
Because the objective of the SVS is to identify modifiable and diverse social vulnerabilities in medical inpatients, a single ‘SVS score’ would not be clinically meaningful. Furthermore, we did not pursue domain-specific scoring algorithms for a number of reasons: (1) Questions from three of the four domains were derived from existing questionnaires, of which one (social support) already had a scoring algorithm that had been developed and validated18; (2) Questions within the domains consisted of different types of responses (binary, categorical and open-ended) that are not only difficult to synthesise into a single score, but that also make the meaning of a domain-specific score unclear; (3) For prediction of outcomes, there is evidence to suggest that the use of individual facets (or variables) within a domain may be superior to the use of scores because different facets may have different associations with outcomes.28
Patient and public involvement
While patients took part as participants of the study, they were not involved in the design, conduct or reporting of the study.
Study participants were patients admitted to the internal medical service at the Foothills Medical Centre in Calgary, Alberta, between December 2014 and October 2015. Inclusion criteria were that patients must be residents of Alberta and that the discharge destination was home or an independent living facility. Patients discharged to non-independent facilities were excluded, as direct patient care is provided in these settings, making social vulnerabilities and the need for self-management less relevant.
Feasibility and face validity
Feasibility of the SVS was assessed for the first 107 study participants, based on the time to completion and the proportion of incomplete surveys. A research assistant administered and timed the completion of both the SVS and the background information survey via an in-person interview. At the conclusion of these surveys, an additional five questions with free-text responses, were administered:
Was the length of the questionnaire acceptable? Why or why not?
How comprehensive was the questionnaire in identifying social barriers to health?
Which, if any, questions would you recommend removing from the questionnaire?
Are there any questions that you feel are missing and should be added?
Are there any modifications you would recommend to the wording of the questions to improve clarity?
Responses were transcribed concurrently during the in-person interview. Survey data were collected and stored in secure REDCap (Research Electronic Data Capture), a web-based data management application.
Feasibility and face validity
Free-text responses were analysed using thematic content analysis.29 30 Because the goal of this analysis was to explore face validity, rather than to develop or explore theory, a qualitative descriptive approach was undertaken.31 32 On study investigator (KLT) performed open coding, then organised these into themes that captured different aspects of feasibility and face validity of the SVS. Review and interpretation of codes and the development of themes were undertaken through regular meetings between members of the study team (KLT and WAG). Any proposed modifications to the SVS based on patient feedback were discussed among three members of the study team (KLT, M-JS and WAG), and decisions were made by consensus. A record of changes was kept.
Descriptive statistics for socio-demographic and clinical characteristics of the sample population were conducted. For categorical variables, we reported frequencies and proportions. Means and SD were calculated for continuous variables. Because questions were developed entirely de novo for only one (health salience) of the four domains of the SVS, descriptive statistics of response characteristics and hypothesis and known-groups testing (for construct validity—see below) were performed only for this domain.
Construct validity was assessed through hypothesis testing. First, the research team formulated a priori hypotheses about the expected correlations between the health salience questions and patient socio-demographic characteristics, self-rated health, subjective social status33 34 and perceived stress, based on literature. Similar and overlapping constructs were hypothesised to be positively correlated (convergent validity).35 All hypotheses included the direction and strength of correlations: small (0.1≤ r <0.3 or −0.3≤ r <−0.1), moderate (0.3≤ r <0.5 or −0.5≤ r <−0.3) or large (≥0.5 or ≤−0.5).36 Constructs that had no logical overlap were hypothesised to have no correlation, r<0.1 (discriminant validity).35 36 Observed correlations from the data were compared with the hypothesised correlations.
Hypotheses were also formulated about expected differences in responses to health salience questions across known groups, known as discriminative validity.35 Five hypotheses were formulated a priori:
Patients with lower income are more likely to state that money-related concerns make it difficult to focus on health than those with higher income.
Patient not currently working are more likely than those who are working to report that money-related and job-security concerns make it difficult to focus on health.
Patients without permanent housing are more likely to state that their housing situation makes it difficult to focus on health.
Students are more likely to state that school-related concerns make it difficult to focus on health.
Stay-at-home parents are more likely to state that domestic responsibilities make it difficult to focus on health.
Hypotheses were tested by comparing distribution of responses across these known groups, through χ2 testing. P values<0.05 were considered to be statistically significant.
Exploratory factor analysis
Structural validity of the global survey was determined through item factor analysis.35 Exploratory factor analysis based orthogonal factor rotation using the varimax method was first conducted, as the factor structure and the number of dimensions explored in the SVS were unclear (because questions were compiled from different sources, and in some cases, created de novo). Categorical variables with a missing data rate of >20% and nominal variables (where responses are categorical with no implicit or explicit order) were excluded from exploratory factor analysis. Missing responses for ordinal and continuous variables (Questions 6–8, 14, 15, 17–24, 27, 30 and 31) were imputed with the median. Sensitivity analysis was completed, where exploratory factor analysis was re-run using raw data without imputation. The number of factors ultimately retained were based on the following: eigenvalues>1.0, examination of the scree plot, the point at which adding more factors minimally changes the cumulative explained variance, and parallel analysis,.37 A minimum loading of 0.5 was determined to be the threshold at which a variable was retained within a factor. Internal consistency, or the extent to which items within a factor represented the same construct, was evaluated using Cronbach’s alpha for each factor.35
A total of 470 patients were recruited into the study. Of these, 64 were excluded (16 were not internal medicine patients, 19 were not discharged home or to an independent living facility, 2 died in hospital, 14 withdrew consent and 13 were not residents of Alberta). A total of 406 patients were included in the analysis. The mean age was 55.5 (SD 18.6) years (table 2). The majority of the sample was men (55.4%), Caucasian (68.0%), born in Canada (72.4%) and reported English as their first language (85.2%). Approximately 30.5% of the sample were employed, while 9.1% were unemployed and 38.9% were retired.
Feasibility and face validity
The mean time for completion of the SVS and background information survey together was 17 min 25 s (SD 5: 48). Nearly all patients (98.1%) found the length to be acceptable. No patients terminated the survey prematurely, and no removal of questions was suggested. Small wording changes were made to Question 26 for brevity and to Questions 6, 9 and 32 to increase specificity (ie, specifying the mode of transportation when asking about travel time, that ‘travel barriers’ pertained only to transportation barriers, and specifying which health services were being examined when asking about barriers to access, respectively). Participants also recommended splitting a single item into two, in two circumstances. First, for cost-related medication non-adherence, they recommended asking about both skipping medications and not filling a prescriptions (Questions 28 and 29), as these may reflect different levels of financial constraints. Second, participants felt clarity was needed about whose costs were being explored when asking about out-of-pocket drug costs (Questions 30 and 31). The final survey contained 33 questions (table 1; online supplemental appendix 1) in the following domains: transportation (nine questions), health salience (seven questions), social support (nine questions) and finances (eight question).
Response characteristics and construct validity of health salience questions
Distribution of responses for the seven health salience questions are presented in table 3. Approximately 12%–15% of participants described skipping tests, medications or medical appointments due to other life circumstances taking priority; an even higher proportion (30.7%) described difficulty following lifestyle recommendations for this same reason. Despite this, 77.9% of patients indicated that their health was ‘very important’, and 60.2% believed that it would be ‘very easy’ or ‘easy’ to find the time and energy to keep healthy after hospital discharge. When asked about competing priorities that would make it difficult to focus on health, the most commonly reported was finances.
We determined convergent and discriminant validity of the health salience questions through hypothesis testing of correlations. We developed a total of 99 hypotheses (online supplemental appendix 3), 35 of which predicted no correlation between responses to certain health salience questions and background socio-demographic characteristics (discriminant validity), and 64 of which predicted the presence of weak, moderate or strong correlations (convergent validity). These hypotheses were informed by literature suggesting the presence of associations between adherence to lifestyle changes, medications and/or medical appointment-keeping with stress,38 self-rated health,39 subjective social status,40 age,41–44 income45 and employment status.46–48 Of these 64 hypotheses, 39 (61%) observed correlations were as predicted in both strength and direction, with another 16 (25%) in the same direction (but not the same strength) as predicted (tables 4 and 5).
For discriminant validity, one of the 35 hypotheses could not be tested due to the number of missing responses. We demonstrated no correlation, as predicted, between health salience questions and 18 (53%) socio-demographic characteristics (tables 4 and 5). The remaining 16 hypotheses demonstrated primarily small correlations, only two of which met statistical significance.
Discriminative validity was determined through known groups testing. We observed significant differences in proportions as hypothesised:
Patients with low income were more likely to state that money-related concerns made it difficult to focus on health (47.2% vs 22.1%, p<0.01).
Patients not currently working were more likely than those who were working to state that both money-related and job-security concerns made it difficult to focus on health (50.0% vs 26.4%, p<0.01; and 23.9% vs 9.6%, p<0.01).
Patients without permanent housing were more likely to state that their housing situation made it difficult to focus on health (58.8% vs 9.3%, p<0.01).
Students were more likely to state that school-related concerns made it difficult to focus on health (50.0% vs 1.3%, p<0.01).
Stay at home parents were more likely to state that domestic responsibilities made it difficult to focus on health (47.2% vs 22.1%, p<0.01).
Factor analysis of the SVS
Exploratory factor analysis was conducted for 27 of the 33 questions in the SVS. Questions 2, 10, 21 and 33 were excluded from analysis due to a missing data rate >20% (online supplemental appendix 4). Questions 4 and 5 were additionally excluded from analysis due to the nominal nature of response categories (ie, modes of transportation). Exploratory factor analysis demonstrated that five factors had eigenvalues over 1 (see figure 1—scree plot), and that these five factors accounted for 61.4% of the total variance. The five factors were: (1) social support; (2) health salience; (3) drug insurance; (4) transportation barriers; and (5) drug costs (see online supplemental appendix 5), with associated variables and their factor loadings. All questions loaded only to one factor. Four questions (Questions 3, 9, 14 and 15) did not load to any factor. Internal consistency, as measured by Cronbach’s alpha, was reasonable, at 0.94 for factor 1 (social support), 0.78 for factor 2 (health salience), 0.91 for factor 3 (drug insurance), 0.58 for factor 4 (transportation) and 0.74 for factor 5 (drug costs). Within each factor, all variables were correlated with each other (correlation coefficients ≥0.2), but no correlations were >0.9. That is, each factor comprised of correlated but likely not redundant variables.35
Sensitivity analysis was conducted, repeating the exploratory factor analysis on raw data without imputation of variables. Based on parallel analysis,37 five factors were retained. These five factors are the same as the ones noted above (see online supplemental appendix 6 for factor loadings). Questions loaded to the same factors as in the original analysis. The same four questions did not load to any factor, with no additional non-loading items demonstrated.
The SVS is a tool that assesses modifiable social vulnerabilities that may impact the ability of patients to maintain their health. While questions from three domains (transportation, financial and social support barriers) were adapted from prior surveys and instruments, seven questions were created for the domain of health salience in the presence of competing priorities. These questions were found to have high convergent and discriminant validity, with the SVS as a whole demonstrating high structural and factorial validity.
There are few existing validated measures for social vulnerabilities and the social determinants of health. The Social Needs Screening Tool from the American Academy of Family Physicians, and the Accountable Health Communities Screening Tool from the Centers for Medicaid and Medicare Services both ask about housing stability, food insecurity, utilities, transportation and personal safety, with additional questions included about family support/assistance, childcare, employment, education and financial strain.49 50 A similar tool, Protocol for Responding to and Assessing Patient Assets, Risks and Experiences instrument has been implemented in health centres across the USA, and includes questions about personal characteristics, family and home (eg, housing status), money and resources (eg, education, employment, food/utilities/clothing/phone security) and social and emotional needs.51 These tools are broad in scope as they are intended to ‘identify any unmet need likely to have a negative impact on health’.49 Criticisms of this breadth include the resultant difficulty in prioritising unmet needs and, more fundamentally, whether identified needs (that span from inadequate housing/food/supports, to transportation needs, to social integration, to stress) are truly actionable by the healthcare provider or healthcare system.46 Furthermore, there are no published validation studies of these questionnaires. Our study addresses these gaps by validating a new tool that focuses on social vulnerabilities that are prevalent,51 evidence-based9 10 and actionable.
The importance of measuring social vulnerabilities cannot be overstated. In a population, only 10%–20% of preventable mortality can be attributed to medical care; in contrast, social factors are overwhelmingly influential in affecting health behaviours and outcomes.46 The SVS can identify patient and population needs so that these can be addressed in a comprehensive, multilevel and multifaceted way. While approaches to social barriers have traditionally focused on population-level interventions and policy development, individual-level practice changes and clinical innovations also have an important part to play.52 If we take cost-related medication non-adherence as an example, individual-level interventions include increasing physician awareness of medication cost though education and provision of resources, so that a more cost-conscious prescribing approach can be undertaken.53 At the institutional and systemic level, electronic health records can be customised to display an alert showing medication costs at the time of prescribing, along with lower cost alternatives.54 55 Default medication orders in electronic health records can also be shifted to generic, lower cost medications (with the ability to opt out).56 Both approaches have been shown to be effective in increasing the prescribing of lower cost medications.54–56 At the population level, broadening prescription drug coverage, removal of coverage gaps and caps and providing ‘first-dollar’ coverage at no direct cost to patients would all reduce out-of-pocket drug costs to patients.57–59 Ultimately, social vulnerabilities cannot be acted on if there is no accurate way to measure them. The SVS is therefore a validated instrument that has the potential to inform the delivery of more patient-centred, equitable healthcare.
One limitation to our study is that we only conducted validation of the SVS in an inpatient cohort. Given the prevalence of social vulnerabilities, and that the social determinants of health influence health and well-being in not just the inpatient population but rather than general population as a whole, the SVS is likely to be applicable and relevant in any patient population. However, we recognise the limitations of extrapolating our data outside of the inpatient cohort. Second, our survey was developed based on the social vulnerabilities identified in qualitative studies of low-income patients in the USA, without similar studies having been done in our specific patient population of interest (ie, general medical patients in Canada). Therefore, the relevance and representativeness of these social vulnerabilities remains unclear. While it is possible that the SVS does not capture other important social vulnerabilities in our patient population, the domains that are included likely remain relevant, with increasing evidence demonstrating their prevalence and/or their associations with hospital readmissions in heterogeneous, broad, populations.60–63 We also note that in our study, we asked specifically about social vulnerabilities that may be missing from the SVS—no patients felt that additional questions in additional domains were needed. Lastly, we recognise that the generalisability of the SVS may be limited due to the specificity of the questions asked. For example, in densely populated cities, licence and car ownership may not be important determinants of healthcare access.
Despite the recognition that social determinants of health and their downstream social vulnerabilities are important correlates of patient well-being and ability to self-manage conditions, there has thus far not been a questionnaire that delves into these social barriers. The SVS is a reliable and valid instrument that identifies modifiable social barriers in medical inpatients. An understanding of these social vulnerabilities is essential in developing interventions, health and social policy that mitigates these vulnerabilities to improve health outcomes.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Patient consent for publication
Ethics approval was obtained from the Conjoint Health Research Ethics Board at the University of Calgary (REB 14-0696). Each participant included in the study provided written informed consent to participate. Participants gave informed consent to participate in the study before taking part.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Twitter @dr_sajobi, @MariaJ_Santana
Contributors Each of the six authors meets the authorship requirements as established by the International Committee of Medical Journal Editors in the Uniform Requirements for Manuscripts Submitted to Biomedical Journals. KLT, M-JS and WAG conceived of the study and were involved in study design. LT conducted data collection. KLT, TS and OL were involved in statistical analyses. All authors were involved in data interpretation. KLT drafted the manuscript and all authors critically revised the manuscript. All authors have read and approved the manuscript. KLT acts as the guarantor, accepting full responsibility for the work; she had access to the data, and controlled the decision to publish.
Funding This study was funded by the Canadian Institutes of Health Research (grant number 357996). They had no role in the design of the study, data collection, analysis or interpretation, or in the writing of the manuscript.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.