Article Text
Abstract
BACKGROUND Fewer than half of individuals with a mental disorder seek formal care in a given year. Much research has been conducted on the factors that influence service use in this population, but the methods generally used cannot easily identify the complex interactions that are thought to exist. In this paper, we examine predictors of subsequent service use among respondents to a population health survey who met criteria for a past-year mood, anxiety or substance-related disorder.
METHODS To determine service use, we use an administrative database including all physician consultations in the period of interest. To identify predictors, we use classification tree (CART) analysis, a data mining technique with the ability to identify unsuspected interactions. We compare results to those from logistic regression models.
RESULTS We identify 1213 individuals with past-year disorder. In the year after the survey, 24% (n=312) of these had a mental health-related physician consultation. Logistic regression revealed that age, sex and marital status predicted service use. CART analysis yielded a set of rules based on age, sex, marital status and income adequacy, with marital status playing a role among men and by income adequacy important among women. CART analysis proved moderately effective overall, with agreement of 60%, sensitivity of 82% and specificity of 53%.
CONCLUSION Results highlight the potential of data-mining techniques to uncover complex interactions, and offer support to the view that the intersection of multiple statuses influence health and behaviour in ways that are difficult to identify with conventional statistics. The disadvantages of these methods are also discussed.
- Access to Hlth Care
- Mental Health
- Social Inequalities
- Social Science
- Modelling
Statistics from Altmetric.com
Introduction
Although effective treatments for mental health conditions such as depression and anxiety have been available for some time, fewer than half of individuals with a mental disorder seek formal care from a primary care physician or psychiatrist.1–4 Understanding the factors predicting mental healthcare-seeking behaviours is crucial for the formulation of health policy and the design of interventions to address mental health service access inequities.
In what is arguably the most influential model of healthcare utilisation, Andersen5–7 proposed three sets of factors, which together can be used to predict use of services at the individual and population levels: predisposing, enabling and need factors. Predisposing and enabling factors are comprised mostly, though not exclusively, of social (eg, gender, age) and economic (eg, household income) variables, whereas need factors are indicators of objective need (eg, presence of a health condition) and perceived need (eg, self-rated health) for care. In an equitable system, need for care should be the most important determinant of service use. As such, the identification of non-need factors associated with service use serves a critical role in assessing systems of care. When factors such as insurance coverage are found to influence use, it raises questions about possible inequities. Although these factors may also reflect differences in the propensity to seek care, similar concerns may be raised when factors such as gender and socioeconomic status (SES) are found to predict service use. Examination of enabling and predisposing factors that predict mental healthcare service use is especially pertinent, given the long-standing concern over stigma associated with having a disorder8 and with seeking care for it.9
A variety of predisposing and enabling factors have been identified as non-need determinants of use of mental healthcare services. These include age,10 ,11 gender,2 ,12–14 SES,14–16 ethnicity,3 ,17–19 marital status,20 parental status20 ,21 and geographical location.10 As several of these factors are social status positions, concerns around equity appear to be legitimate. It is not clear, however, that the influence of these factors has been thoroughly examined.
Existing research on social factors associated with mental health service use rely on statistical models of competing risks. Need and non-need factors are tested simultaneously, and those effects that are statistically significant become the focus of interpretation. With respect to social determinants, this approach is problematic, because it does not take into account the fact that the social positions occupied by each individual may interact in complex ways. The circumstances of a teenaged single mother, for example, may not be adequately described by the independent effects of marital status, parent status, age and gender.
Intersectionality theory 21 ,22 challenges us to consider social determinants not in terms of single factors (eg, gender or SES), but in terms of multiple, interacting factors. In this framework, social disadvantage arises from a constellation of interrelated and intersecting social roles. This view, sometimes referred to as the ‘double’ or ‘triple’ jeopardy hypothesis, has informed considerable research on health. Applications in the mental healthcare utilisation literature include work showing that single mothers are more likely to seek care for mental health problem, than their married counterparts.20
While intuitive, intersectionality is difficult to explore empirically. In linear models, interactions involving more than two variables tend to require large sample sizes, and also to produce models that are complex, difficult to interpret, and often plagued by multicollinearity or other problems. Moreover, interactions involving non-linear effects are usually difficult to detect. Something as apparently simple as an age-by-sex interaction, for example, can be complicated not only by a non-linear association between age and the outcome, but by the fact that the interaction with sex may itself vary with age in a non-linear way.
One solution to this problem is to eschew linear models entirely. Classification trees (CART) are one alternative popular in machine learning and data mining applications.22–24 The CART approach involves recursively identifying rules that distinguish between groups, usually with certain constraints to avoid overfitting. Although it does not permit the testing of hypotheses in the usual sense, CART has two important advantages: (1) it makes no assumptions about variable distributions or relationships and (2) it is capable of identifying complex and unsuspected interactions. CART results can also be useful from a clinical and health policy perspective, because they yield decision rules that can be used to identify at-risk individuals more easily than, for example, regression coefficients.
In the present study, we use CART to examine the social determinants of mental health service use in a general population sample. Specifically, we are interested in exploring complex interactions between different social determinants and their impact on mental healthcare use. Our focus on social determinants is related to Andersen's concern regarding equity: in a country with a ‘universal’ healthcare system,25 it is critical to evaluate whether factors other than need are influencing use of services.
Methods
Data come from cycle 1.2 of the Canadian Community Health Survey (CCHS 1.2), a population survey conducted by Statistics Canada in 2002/2003.26 The sampling frame of CCHS 1.2 included all Canadians aged 15 or older living in private dwellings, with the exception of full-time members of the armed forces and residents of remote areas or native reserves. The final sample size was 36 984. Statistics Canada linked participants from Ontario (n=13 184) to administrative health data; 10 600 (81%) were linked successfully. From this sample, we selected all participants (n=1213) who met past-year criteria for one or more of the five mood and anxiety disorders, with and without substance dependence. The merged dataset was accessed at the Institute for Clinical Evaluative Sciences (ICES). In order to access these data for these analyses, which included linking healthcare administrative date to survey data previously collected by Statistics Canada, independent review (privacy impact assessment) was conducted to ensure compliance with privacy legislation governing access to personal healthcare information through the ICES at Sunnybrook Hospital. The project was approved through this process.
Measures
Outpatient mental health service use
We used physician billing records in the Ontario Health Insurance Plan (OHIP) database to determine whether each respondent used physician-provided mental health services in the year following the survey. In Ontario, medically necessary care coverage is fully financed by a government-funded health insurance programme. Within this universal healthcare coverage setting, 94% of physicians have a fee-for-service practice that is captured by OHIP billing submission data,27 and physicians who provide services on a salary are mandated to submit ‘shadow billings’ for accountability purposes, resulting in a highly accurate and comprehensive physician activity data source. A visit for mental healthcare was defined as any outpatient encounter with a psychiatrist, or a visit with a primary care provider (ie, family physician) or geriatrician with both a mental health and/or addictions International Classification of Diseases-9 diagnosis and a mental health and/or addictions service billing code. Mental health visits with a primary care provider or geriatrician are defined based on a validated algorithm developed by Steele et al,28 with modifications to include contacts with specialist physicians.
Mental disorders
CCHS 1.2 assessed all respondents for six conditions (major depressive disorder, bipolar disorder, social phobia, panic disorder, agoraphobia and substance abuse or dependence) based on symptom reporting in 12 months prior to the time of the survey using the World Mental Health—Composite International Diagnostic Interview (WMH-CIDI). The WMH-CIDI was developed and validated using the Structured Clinical Interview for DSM-IV (SCID) as a reference standard, and is now widely used in epidemiological surveys.2 ,29 In CCHS 1.2, the WMH-CIDI was administered by trained lay interviewers. Details of the administration and psychometric properties of the measure are provided in Gravel and Béland.26
Social determinants
We selected eight variables capturing different social factors previously shown to be associated with the use of mental healthcare: age (in years), gender (males; females), marital status (married or cohabitating; formerly married (separated, divorced or widowed); never married), parental status (parent living with children; other), income adequacy (low; moderate or high), education (less than secondary; secondary graduate; some post-secondary; post-secondary graduate), rurality (rural; urban) and visible minority status, according to the self-description chosen by the survey participant (‘white’; or any category other than ‘white’). Age was entered as a continuous variable; all others were dummy-coded. Income adequacy is a derived variable, produced by Statistics Canada that combines household income and household size. It is ‘low’ for household incomes, in 2002 Canadian dollars, of <$C15 000 for 1–2 residents; <$C20 000 for 3–4 residents or <$C30 000 for five or more residents.
Analysis
We conducted descriptive, univariate and logistic regression analyses. For the logistic regression models, we modelled any service use and specialist care. We used Stata V.9 for these analyses, and bootstrapped all statistical tests using a set of replication weights supplied by Statistics Canada.
We performed classification tree (CART) analyses using the rpart package and R 2.13. We used the Gini index, a measure of heterogeneity that reflects the difference across groups in the probability of the outcome, to select decision rules. We required a minimum terminal node size of 30 individuals and assigned cost weights of ((1−P)/P) (where P is the prevalence, in this case of help-seeking) to cases and 1 to non-cases. This weighting scheme yields equal sums of weights for cases and non-cases, and therefore assigns equal importance to sensitivity and specificity. This is comparable to the weighting used implicitly in, for example, receiver operating characteristic curve analysis. After fitting, trees were ‘pruned’ by retaining the set of decision rules that minimised the cross-validated error. In the CART analysis, we modelled the ‘any service use’ outcome only, as the sample size for specialist care (N=312) was too small to produce meaningful results.
To take into account the complex design of CCHS 1.2, we applied sampling weights in the CART analysis. We used the master survey weights provided by Statistics Canada, rescaled to have a mean of 1.
Results
The prevalence of any past-year disorder was 9.2% (unweighted n=1213; 95% CI 8.6% to 9.9%). Twenty-four per cent (unweighted n=312) of people with disorder had one or more mental health consultations in the year following the survey. Of those who sought care, 52% had seen a psychiatrist in the year preceding the survey. In the univariate analysis (not shown), service use was significantly associated with female gender, low-income adequacy and greater age.
Table 1 shows the results of two logistic regression models: one for any service use and one for use of specialist services. Initial analyses revealed that the association between age and any service was not linear. For this outcome, we therefore included two non-linear variables for age, obtained with Stata's fracpoly procedure. For any service use, age, gender and marital status were significant in the main effects model. Age terms for this outcome yielded an inverted-U form, with probability of service use lowest at the youngest and oldest ages and peaking at approximately age 55. Women (OR=1.64; 95% CI 1.03 to 2.60) and those who were never married (OR=1.88; 95% CI 1.04 to 3.39) were more likely to have sought any care for mental health than men and those who were married, respectively. Only one variable, gender, significantly predicted specialist care among those who had consulted a physician: this outcome was less common among women (OR=0.36, 95% CI 0.15 to 0.87). This effect, however, largely reflects the fact that women were more likely to have consulted a primary care provider, and thus to be included in this analysis. In the sample as a whole, men and women were equally likely to have consulted a specialist (men, 13.7%, 95% CI 10.5% to 17%; women, 13.1%, 95% CI 9.3% to 16.9%).
Classification tree analysis
The final classification tree for any service use included decision rules based on four of the eight social determinant variables included in this study: age, sex, income adequacy and marital status. Detailed results are shown in figure 1 and table 2. Groups more likely to use services were formerly married men aged 23–46; all participants older than 46 and low-income women aged 23–46. Although CART decision rules do not necessarily reflect meaningful or replicable difference, the final tree implies the possibility that income adequacy plays an important role among women, while marital status is of greater importance among men. In terms of overall fit, the effectiveness of the tree as a classifier was moderate, with overall agreement of 60%, sensitivity of 82% and specificity of 53%.
Discussion
We applied two different methods to explore the social determinants of mental health service use in a population-based sample linked to health administrative data. The results of the conventional regression analysis demonstrated that physician mental health service use has a complex relationship with age, and is positively associated with being woman and never having been married. This is broadly consistent with previous research.2 ,10–14
The CART analysis also supports the finding of a complex relationship between age and service use, but also suggests interactions between gender and other social determinants that were not apparent in the regression analysis. Marital status, for example, appeared to be more important among young and middle-aged men (from approximately age 25–50), while income adequacy played a larger role among women.
To our knowledge, this is the first study that shows complex interactions of gender, age, income and marital status to be associated with use of mental health services in the formal healthcare sector. The comparison of regression and the CART models in our study confirms that the inability of linear models to identify complex interactions is a limitation. When such interactions are likely to be present, data mining techniques merit serious consideration. Their potential ability to identify groups who are underserved, or who have a low propensity to seek care, in particular, holds the promise of improved targeting of outreach efforts. They should therefore be of considerable interest to policymakers and care providers.
Our results are broadly consistent with intersectionality theory,21 ,30 which holds that health outcomes (including service use) are differentially affected by multiple, interacting facets of social advantage and disadvantage. Our results are also consistent with empirical work on the social determinants of healthcare use, showing use of services to be influenced by the joint effects of parental status, marital status and gender. For example, single-parent mothers have been shown to be much more likely to use mental healthcare services than married mothers.20
Importantly, our findings raise some important concerns about the role of non-need factors as important determinants of mental healthcare use in a public system. Splits involving age highlight the low levels of service use among young people. This may reflect, in part, the lower propensity of young people to use treatment services.9 For young and middle-aged men, consultations were more common among those who were separated or divorced. Although CART does not tell us whether this effect differs significantly between the sexes, one interpretation of this result is that the loss of spousal support increases the likelihood of help-seeking in the formal care sector. The role of low income among young and middle-aged women may also bear further examination. Greater contact with social services might facilitate contact with the medical sector. Conversely, higher income women may be able to access mental healthcare from non-physicians (eg, psychologists). Variables such as marital status and income may also function, in part, as indicators of illness type (eg, substance-related) or severity. Nevertheless, results suggest the presence of intriguing differences between men and women in the importance of these social variables as determinants of mental health service use. Bearing in mind the limited sample and the nature of the methods used, we would cautiously suggest that differential use of services by men and women is itself conditioned by other social factors (notably social support and income). While women may have a greater propensity to seek care, results suggest that this is not a simple gender difference, and those interactions with other social and structural factors influence help-seeking for mental health.
There are several limitations that warrant caution when drawing conclusions based on these results. One concerns the difficulty CART methods experience in treating continuous variables: although true age effects are usually incremental, CART is obliged to select specific cut-points. More important is the fact that CART is an exploratory technique unsuited to hypothesis testing. This makes replication of these results in a different dataset imperative. At present, no such comparable data exist in Ontario. However, a new national survey of mental health is currently underway, and if these data can be linked to health administrative data as has been carried out for CCHS 1.2, it will be possible to test these classifications using comparable data.
We have also limited our analysis to social determinants. There are many other factors, related to need (eg, severity and comorbidity), the healthcare system (eg, availability of psychiatrists) and cultural-level or individual-level characteristics (eg, personality and perception of stigma) that influence help-seeking, and these are best considered together. Sample size and other data limitations circumscribed our analysis. Future work, however, may be able to explore how the influence of these other factors interacts with social determinants in shaping help-seeking behaviour.
Further, we included all individuals meeting diagnostic criteria for one or more of the six disorders included in CCHS 1.2. This approach has well-known drawbacks.31 Individuals with other disorders (eg, generalised anxiety disorder) were not included unless they also had one of the six mentioned; and, more importantly, not everyone who meets diagnostic criteria requires formal care at all.
Finally, unmet need in our sample was, at almost 75%, somewhat higher than previous estimates. Research based on self-report has estimated this proportion at 66%.32 One explanation for this discrepancy is that we had no data on non-physician services. Some individuals without ‘service use’ in our data may have been adequately treated by other providers, especially if suffering from mild-to-moderate illness.
Strengths should also be acknowledged. By using survey data to identify individuals in need of care and administrative data to identify service use, we address two important methodological issues: recall and other biases associated with self-report of care, and the problem of relying solely on treatment data to define need. The combination of both data sources allows us to first measure untreated prevalence and then examines service use in this sample.
Conclusion
Using a data mining technique, we identified a set of complex interactions not apparent using more conventional methods. While further research (including replication) is needed, we believe that the approach used in this paper warrants serious consideration as a method for understanding factors associated with help-seeking in general, and mental healthcare in particular.
What is already known on this subject
-
Social factors such as gender, socioeconomic status and age are important determinants of mental healthcare use. However, the associations between these factors are likely very complex and existing research may not have captured this complexity owing to a reliance on standard statistical approaches such as interaction testing using regression analysis. In this paper, we use a technique known as classification and regression tree analysis (CART) to explore complex interactions predicting service use among adults with diagnosable mental disorders.
What this study adds
-
Using CART reveals complex interactions between gender, income and age that were not evident using logistic regression. The results show that service use is dependent on a number of intersecting social status positions. Research on this question should acknowledge and explore this complexity.
References
Footnotes
-
Contributors JC was involved in the design of the study, oversaw the statistical analyses, and was the lead author of the paper. SV was responsible for conducting the statistical analysis, and participated in the writing of the paper. SV participated in writing the paper, and contributed to the interpretation of the results. DLS and TJW were involved in the design of the study, contributed to the interpretation of the results and participated in writing the paper. PK was involved in the design of the study, the development of the analysis plan and participated in writing the paper.
-
Funding This research was supported by the Ministry of Health and Long-term Care (University of Ottawa), Ontario Canada. No number provided. Provided Funding Only.
-
Competing interests None.
-
Ethics approval Institute for Clinical and Evaluative Sciences.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement Unfortunately, privacy legislation in the province of Ontario makes it impossible to release linked health administrative data.