Invited Commentary: Using Area-based Socioeconomic Measures—Think Conceptually, Act Cautiously

Investigators dedicated to understanding social inequalities in health in the United States face major challenges. Key among them are data constraints, limitations in theory development, and disciplinary divides. A central data concern is that many data sets do not include adequate health and socioeconomic information. Growing conceptual interest in contextual influences on health leads some investigators to link census to health data in order to use area-based socioeconomic measures (ABSMs) to estimate contextual effects (1). When data permit, both microlevel socioeconomic variables and ABSMs are included in multilevel models to estimate area impacts on health above and beyond the characteristics of individuals (2). However, investigators have also used ABSMs to substitute for missing microlevel information, raising the difficult question of how well these aggregate variables serve to proxy individual characteristics.

Investigators of this question have often warned of the dangers of making inferences about individual-level relations from aggregate-level variables, including ABSMs (3–12). It is not unusual for researchers to find substantially larger estimated effects when using aggregate variables compared with when using microlevel ones. Researchers from diverse fields have derived the conditions under which overestimates are likely (9, 11, 13). Overestimates are likely when, had both variables been included in the same regression equation, the estimated coefficient on the aggregate variable would have been larger than—and of the same sign as—the estimated coefficient on the missing microlevel variable. In the case where the explanatory variable is an indicator of socioeconomic position, this condition is met when the aggregate variable represents a broader construct than the microlevel variable or when there are contextual effects (8, 11, 12).

Subramanian et al. approach the question of comparability between ABSMs and microlevel socioeconomic variables through an empirical analysis. They offer empirical evidence of contextual effects and argue that ABSMs are not interchangeable with microlevel variables but measure the broader construct of “a shared collective or contextual dimension that all individuals within a context share” (14, p. 829). Thus, their findings suggest that the conditions that other investigators conclude would most often set the stage for overestimating microlevel effects when using ABSMs are met in their analysis. Yet, Subramanian et al. conclude the opposite, stating that their study “demonstrates, at least in the case of disparities in (low) birth weight, that aptly chosen ABSMs provide estimates of health inequalities that are comparable with those based on individual-based socioeconomic measures. The risk, if any, in the absence of individual-based socioeconomic information is a conservative estimate of socioeconomic inequalities in health” (14, p. 833).

In light of contradictory conclusions by other investigators, have Subramanian et al. demonstrated their claim, as they say they do? I believe the evidence they provide falls short of making their case. Their confidence in ABSMs is misplaced.

DO ABSMs RELIABLY YIELD COMPARABLE OR CONSERVATIVE ESTIMATES TO ABSENT MICROLEVEL DATA?

Analyzing Massachusetts data for 1990, Subramanian et al. compare estimates of the effect of socioeconomic position on birth weight using three ABSMs, each measured at the census tract or block group level, with microlevel maternal and paternal education variables. They report that estimates from the two approaches are roughly comparable and, in cases where they differ, that the estimate using the ABSM is lower than that using the microlevel variable. However, also studying health outcomes and performing empirical work in the spirit of that reported by Subramanian et al., Geronimus et al. (11) and Geronimus and Bound (12) concluded that ABSMs have significant limitations as proxies for unavailable microlevel variables and are likely to overestimate effects compared with microlevel measures. Subramanian et al. cite those papers (11, 12) as “an earlier assessment questioning the validity of ABSMs” (14, p. 833) but do not engage in a scientific discussion of how to reconcile the dramatic differences in conclusions.

Complementing the empirical analyses, Geronimus et al. (11) explicated a statistical model illuminating two potential sources of bias inherent in using an ABSM to measure individual characteristics: a downward bias due to measurement error and an upward bias that arises if the aggregate variable has an independent effect on the outcome. Geronimus et al. formalized this logic algebraically by constructing the hypothetical situation where the coefficient on an aggregate and a microlevel measure would be estimates of the same parameter. From this idealized model, they could illuminate the nature of how the real world falls short.

The statistical model of Geronimus et al. (11) led them to predict that most often the aggregation bias would predominate, inflating estimates compared with a microlevel measure. Their empirical work supported this prediction; still, they noted the potential to arrive at underestimates and, in fact, discussed an example where they found this to be true. A critical problem with using ABSMs to impute individual-level effects, they concluded, is that the resulting bias can be substantial and the direction of the bias cannot be interpreted with confidence.

One possibility, then, is that the finding by Subramanian et al. represents an instance where an effect is underestimated. Another possibility is that Subramanian et al. arrived at underestimates because of the way they specified the socioeconomic variables being compared. The most common specification used in the literature when substituting aggregate variables for missing microlevel variables is to define the aggregate variable as the location-specific mean or median of the missing microlevel variable and to assign each individual that value (for income, education, and so on). This approach defines the microlevel and aggregate variables to be as comparable as possible. In the hypothetical world where there is no residual confounding, regressions using the microlevel and aggregate variables would estimate the same parameter (refer to Appendix A).

Following this approach, Geronimus et al. (11) compared the coefficient estimate for a dummy variable representing whether or not an individual had graduated from high school with the coefficient estimate for an aggregate variable that they specified as the percentage of area residents who were high school graduates. Subramanian et al. specify their socioeconomic variables in a different way. At the microlevel, they categorize mother's or father's education as less than high school, high school, some college, college graduates, and missing education data. They specify their aggregate education variable as areas with these adult population fractions with less than a high school education: <15 percent, 15–24.9 percent, 25–39.9 percent, and 40 percent or more.

Setting aside the missing category for parental education, at each level Subramanian et al. allow for six possible contrasts. How does one know which of the six contrasts between the microlevel categories are to be compared with which of the six contrasts between the aggregate categories? Their specification approach provides no intuitive guidance. There are no contrasts where one would expect that, hypothetically, in the absence of residual confounding, the coefficients on the microlevel variable and the ABSM would estimate the same parameter. Instead, their categorization puts circumstances such as being a college graduate on the same plane as being a resident of a census tract with <15 percent high school drop outs.

Subramanian et al. highlight contrasts between best and worst. How meaningful, though, is comparing odds ratios for individuals who are college graduates versus individuals who have less than a high school education, on the microlevel, with odds ratios for census tracts with <15 percent adult population with less than a high school education versus census tracts with 40–100 percent adult population with less than a high school education on the aggregate level? Further, theoretically, “best to worst” is the appropriate contrast for some research questions but not for others where comparisons between high school graduates and either college graduates or those with less than a high school education are pertinent. Won't results depend on which contrasts are compared?

In fact, the ABSM that best approximated the microlevel maternal education estimate of “best to worst” (while being slightly lower) is the fraction less than high school aggregated to the census tract level. The conclusions of Subramanian et al. draw heavily on this comparison. However, using the same ABSM when the comparison is between college and high school graduates yields an estimated birth weight difference that is now 25 percent higher than that yielded by the maternal education variable.

In addition, Subramanian et al. would have arrived at qualitatively different findings had they used a more standard specification. I used the numbers Subramanian et al. reported to calculate rough approximations to the estimates they would likely have obtained had they specified the ABSM fraction high school graduates as a continuous variable and the microlevel variable as a dummy variable indicating high school graduate status. This specification takes the common approach of defining the ABSM as a location-specific mean or median and approximates that used by Geronimus et al. (11), thus facilitating a comparison between the two sets of findings. Using this specification, I estimated that, far from yielding a conservative estimate, the ABSM yielded a coefficient roughly three times the magnitude of the microlevel variable (refer to Appendix B).

At a minimum, these calculations suggest that the size and direction of difference between estimates based on aggregate compared with microlevel socioeconomic measures are highly sensitive to specification, a shaky basis on which to form the conclusion that one empirical comparison alone “demonstrates” the comparability of such estimates or the direction of any bias. Moreover, without clarifying the importance of their specification to their findings, Subramanian et al. imply that investigators who specify their aggregate variables in more standard ways should interpret their estimates as conservative, when, in fact, their estimates will more likely be inflated relative to those using microlevel socioeconomic variables.

“APTLY” CHOSEN ABSMs

Subramanian et al. qualify their conclusion that ABSMs well approximate microlevel variables as pertaining to the use of “aptly chosen” ABSMs. By “apt” they refer to 1) ABSMs that they would classify as “measures of deprivation” rather than “measures of affluence” and 2) ABSMs aggregated to the census tract rather than to the block group. These are post hoc conclusions and open to question.

The view by Subramanian et al. that ABSMs can be classified as “measures of deprivation” or “measures of affluence” is a means of reconciling their finding that results using two of the ABSMs (fraction of the adult population with less than a high school education; fraction of the population living below the poverty level) better approximate results using microlevel variables than when they use their third ABSM, the fraction of the adult population that is college educated. At both the census tract and block group levels and, when comparing best and worst categories, each of the first two ABSMs yielded results that closely approximate those based on microlevel education variables. Increases in birth weight going from the most disadvantaged to the most advantaged socioeconomic levels are 61–79 g for the microlevel variables and 55–72 g for these ABSMs. In contrast, the results using the third ABSM suggest a smaller increase, only about 44 g, associated with going from areas with the lowest fraction of college educated to those with the highest.

The authors interpret away the weaker results for the third ABSM by classifying it as a measure of “affluence,” while classifying the ABSMs yielding estimates consistent with their conclusion as measures of “deprivation.” They then argue that ABSMs reflecting deprivation are better for imputing microlevel effects than are ABSMs reflecting affluence. By implication, the results from using the ABSM referring to the fraction college educated can be disregarded. Subramanian et al. do not even present results for this variable when they report estimated odds ratios of low birth weight by socioeconomic position.

Results for the fraction college educated are weaker than the others when comparing best and worst categories, but not as clearly so in other comparisons, and, in any case, the explanation Subramanian et al. offer is implausible. The correlation coefficient between the fraction with a college education and the fraction with less than a high school education is 0.73, suggesting that these two ABSMs are unlikely to tap very different constructs. Moreover, whether or not the fraction college educated leads to weaker or stronger results depends on which contrast is studied. When the contrast is between mothers with less than a high school education and those who are high school graduates, it is the college ABSM that yields estimates that most closely approximate the results from maternal education. Calling some ABSMs measures of deprivation and others measures of affluence overinterprets the meaning of an ABSM as reflecting “deprivation” or “advantage” based on its name alone (12). Instead, on average, one would expect that advantaged areas would be characterized by a high fraction of the adult population being college educated, a low fraction with less than a high school education, and a low fraction in poverty, while disadvantaged areas would be characterized by a low fraction of the adult population being college educated, a high fraction with less than a high school education, and a high fraction in poverty.

Subramanian et al. also conclude that census tract level ABSMs are preferable to those aggregated to the block group level. This argument is based largely on their interpretation of the second set of empirical analyses that they report, which they refer to as “mutually adjusted” models. These consist of birth weight equations that again contain a few covariates but this time include ABSMs and microlevel education variables in the same models. More often called “multilevel models,” the rationale is to estimate whether contextual variables have an independent association with health outcomes after controlling for individual characteristics.

The literature estimating just such models is large, and much of this published work (1, 2, 15) reports results similar to the general findings of Subramanian et al., who identify their analysis as novel, because it represents “the first time that ABSMs measured at multiple levels of geographies have been considered in a simultaneous multilevel manner, thus allowing an interpretation of the relative importance of different levels” (14, p. 829).

Subramanian et al. conclude that census tract-level ABSMs were “considerably stronger predictors of birth weight than were block group ABSMs” (14, p. 829), because 1) more often than not, coefficient estimates are higher for census tracts than block groups when both are in the same equation (although the differences are rarely large and the authors do not test the statistical significance of the difference), and 2) “the size of the census tract-based socioeconomic differential was similar to that of individual socioeconomic position from models that do not mutually adjust for each other” (14, p. 829). However, the latter rationale is also true of block group ABSMs for the contrast on which Subramanian et al. focus and even more so in other contrasts. Moreover, when comparison is made of coefficients in the “mutually adjusted models” with those in the earlier models where only an ABSM (at the census tract or block group level) or the microlevel education variables are included, the microlevel coefficients evidence stability, while the ABSM coefficients are reduced substantially (on the order of 60–80 percent). The block group-level ABSM coefficients appear to be reduced more than the census tract-level coefficients are, suggesting that the block group-level ABSMs are more highly correlated with the microlevel variables.

The comparisons outlined above suggest the following provisional advice to investigators hoping to use the findings of Subramanian et al. to aid them in making the most apt choice of ABSM level: Block group-level ABSMs are preferable for imputing individual characteristics, while census tract-level ABSMs are preferable for measuring contextual effects. However, the differences between using census tract and block group ABSMs are small. This is consistent with earlier findings comparing ZIP Code with census tract-level ABSMs (12) and should provide some comfort for investigators who have only census tract-level data available to them (or ZIP Code for that matter). Further, the interpretation that the census tract level is the better choice when estimating multilevel models would be strengthened if it were based on prior substantive knowledge pertinent to the specific application, rather than only on the strength of correlations in an academic exercise.

ABSMs: A “ONE SIZE FITS ALL” SOLUTION?

Subramanian et al. promote the use of ABSMs in a host of contexts where their value may be limited. Overall, Subramanian et al. appear to overstate the weaknesses of microlevel variables and understate those of ABSMs. Theoretical acknowledgment that collectively shared or contextual factors can impact health does not imply that ABSMs are always the more appropriate way to measure that impact. Rather than turning to ABSMs as a one-size-fits-all solution, advancing social epidemiologic knowledge in the face of data concerns is better served by having a clear grasp of what theoretical purposes an empirical analysis is intended to address and then seeing how best to use available data to do so.

For example, investigators who posit the existence of structural influences on health that are maintained through dynamic interactions between populations defined by the social construction of race may find microlevel race variables useful to test the working hypotheses that follow. If their hypotheses relate to racial segregation, they may find it appropriate to use theoretically relevant ABSMs as well. However, they would have no reason to prefer to measure race through an ABSM rather than at the individual level, even when they interpret race as a social construct.

Similarly, Subramanian et al. are unable to substantiate their proposition that “area-based assessment may well be the only means to understand the extent and nature of birth weight inequities at younger ages, as the available individual-level socioeconomic data are unlikely to be meaningful (e.g., educational level for teens and young adults aged less than 25 years)” (14, p. 829). The premise, itself, is hasty. Currently, fewer than 30 percent of US women attain college degrees, suggesting that most women complete their educations by their early 20s, especially those who become young mothers. Indeed, according to the 2000 US Census, fewer than 15 percent of mothers aged 20–24 years are in school. Even for those women who continue in or return to school, it is unclear how Subramanian et al. understand that the birth weight of a child born to a young mother will be affected if its mother attains higher levels of education (or income) at a later date.

A priori, other options exist to using an ABSM when a typical microlevel variable is problematic. For the case of young mothers, depending on available data, one could still use microlevel variables. For example, if one is concerned about the fact that a teen mother has not had the opportunity to complete her schooling, one can transform microlevel schooling information into age-for-grade or use information on the educational status or the income of her parents to measure her socioeconomic position—all microlevel variables. In contrast, if one is interested in the extent to which to attribute poor outcomes associated with early childbearing to maternal age, per se, one might exploit natural experiments to account for socioeconomic background characteristics (16–19).

Empirically, Subramanian et al. find that ABSMs produce very imprecise estimates for teen mothers, those mothers who are most likely to have incomplete educations. Still, Subramanian et al. do not reconsider their contention that ABSMs can be “uniquely relevant” for young mothers, concluding instead that their study findings support it. The basis for this conclusion is their finding that, among births to mothers aged 20–24 years, their census tract poverty ABSM detected socioeconomic gradients in birth weight that were lower than, but similar to, those detected by the microlevel parental education variables. However, using microlevel variables as the standard of comparison is surely an odd way to make their case. If the argument in favor of using ABSMs is that microlevel measures “are unlikely to be meaningful” (14, p. 829), then showing that ABSMs perform almost as well as microlevel measures is hardly an endorsement.

Although many of their points and statistical analyses have implications for developing etiologic understandings, Subramanian et al. do not engage the application of ABSMs to etiologic questions in detail, instead citing “surveillance” and “public health monitoring” as the rationales for using ABSMs. However, what contribution is made to public health monitoring by knowing how closely using ABSMs mimics using microlevel variables in health outcome equations? Their empirical exercises are unnecessary for illustrating the descriptive value of using ABSMs to provide snapshots of the health of current residents of specific residential areas. Such snapshots are certainly useful. However, they can also mislead when used to address the question of whether things are “getting better or worse over time” (14, p. 833). For example, if an urban geographic area monitored in one decennial census year is shown to have a decreased mortality rate in the next decennial census, the possibility that this change reflects health, economic, or social welfare improvements of the original population is one interpretation. Imagine, however, if high-risk residents at the beginning of the decade were later displaced by gentrification or became homeless or incarcerated and, thus, were not captured in the next census. In such circumstances, the health of the original population could well have worsened over the decade. By so quickly eschewing the possibility of compositional explanations, Subramanian et al. risk missing potentially important population dynamics, including ones that are distressingly plausible in some urban centers.

Conclusion

In theory, it is reasonable to expect health to be influenced by characteristics of both individuals and the communities in which they reside or with which they are identified. However, distinguishing between the impacts of individual and community characteristics is conceptually and empirically difficult, because of the complexity of the social processes involved, as well as for the obvious reason that individuals are neither randomly assigned to place of residence nor to social identity (20–22). From a measurement perspective, the idea that microlevel variables reflect only individual characteristics is incorrect, as is the idea that aggregate variables reflect only contextual constructs. Rather, the coefficient on each will reflect omitted factors from the other (7–9).

Yet, as long as health data sets offer few other options, social epidemiologists will continue to use ABSMs to proxy for missing microlevel variables or to estimate contextual effects. They should not do so lightly, however. Cautious use of imperfect data with due attention to the limitations is different from promoting the use of imperfect data with enthusiasm. When using ABSMs, investigators should exercise caution in interpreting study results, be transparent about the limitations, and inform assessment of these limitations with thoughtful consideration of theory and substantive knowledge that are relevant to the specific application.

Most importantly, the question of the comparability of ABSMs to microlevel measures is not up for grabs, depending on how well ABSMs perform in the latest empirical example. Sufficient evidence exists to take seriously the dangers of interpreting ABSMs as if they were microlevel variables. Continuing efforts to advance understanding of social inequality in health will be successful to the extent that investigators draw on a deep knowledge of substance, theory, and measurement. Taking too simple an approach to measurement short-circuits discussion about the implications of conceptual and substantive understandings of the social patterning of health for research design, measurement choice, and the interpretation of empirical results in light of measurement limitations. Unwarranted enthusiasm for ABSMs may also distract from the press for better measures, which, in turn, hampers the testing and development of good theory.

APPENDIX A

Now, one could imagine estimating β by running a regression of y_ij on x_ij. However, if one had data only on the location means of x, then one could imagine estimating β by running a regression of y_ij on x_·j. If both x_ij and x_·j are independent of ε, then either procedure will consistently estimate β. In such case, it would be perfectly legitimate to use an aggregate variable as a proxy for a microlevel one. However, the assumption that ε is independent of x is unlikely to hold in practice. In particular, suppose that x represents only a component of socioeconomic position and that, among other things, ε includes other unmeasured components of socioeconomic position. In this case, we would expect that cov(ε,x) > 0, and our estimates of β will tend to exaggerate the causal effect of x alone on the outcome variable.

Under what circumstances will this tend to be more true when we use x_·j rather than x_ij? The use of x_·j will produce the larger magnitude coefficient if, when both are included in the same regression, the coefficients on the two variables are of the same sign (9). In the context where x is an indicator of socioeconomic position, this will be true if either the aggregate variable represents a broader construct than the microlevel variable or the aggregate variable picks up the contextual effects (8, 11, 12).

APPENDIX B

At the aggregate level, Subramanian et al. (14) estimate that moving from census tracts with fewer than 15 percent population with less than a high school education to tracts with 15–24.9 percent with less than a high school education lowers average birth weights by roughly 18 g. Thus, an approximate 10 percentage point change in the proportion with less than a high school education is associated with an 18-g difference in average birth weight. Moving from a census tract with less than 15 percent to one with between 25 percent and 39.9 percent with less than a high school education lowers average birth weight by 50 g, or roughly 2 g per percentage point. Finally, moving from a census tract with less than 15 percent to one with 40 percent or more with less than a high school education lowers average birth weight by 72 g, or roughly 1.8 g per percentage point. These numbers suggest that, had Subramanian et al. (14) used the ABSM fraction in the census tract with less than a high school education and measured it as a continuous variable, they would have estimated a coefficient between 180 and 200.

At the microlevel, given the fact that roughly the same number of mothers fall into each of the categories high school graduate, some college, and college graduate, the contrast between those with and without a high school diploma should be roughly the average of the contrast between those with less than a high school education and each of these three groups, about 64 g. Thus, had Subramanian et al. (14) specified their microlevel and aggregate socioeconomic variables similarly to the way that Geronimus et al. (11) did, they would have found that the ABSM fraction in the census tract with less than a high school education yielded a coefficient roughly three times the magnitude of the microlevel variable.

The author is indebted to John Bound, Gilbert Gee, Margaret Hicken, and Cynthia Colen for comments on a previous draft.

Conflict of interest: none declared.

References