Jump to comment:

• Spurious conclusions from a faulty analysis
  Jose A. Tapia Granados
  Published on: 29 November 2017
• Author Affiliation
  Mahiben Maruthappu
  Published on: 1 February 2016

• Published on: 29 November 2017

  Spurious conclusions from a faulty analysis

  • Jose A. Tapia Granados, Associate Professor Drexel University, Philadelphia, PA, USA

  It is to be commended that the article on the effects of economic crises on population health outcomes in Latin America, by. Callum Williams et al., clearly explains the methods the authors used for the analysis. For that reason, the paper is a very good example of how not to use a specific type of research tool, the panel regression. In a panel regression, as in any time-series investigation of causality, a key issue is to adjust for time trends, so that variables are stationary series (1, 2). If this adjustment is missing, results are biased by trends in the variables. For example, the paper says that “after removing inflation and unemployment as controls from our regression analysis, GDP per capita increases were found to be associated with improvements in all mortality metrics.” This is just an spurious result, as in every country the trend in GDP per capita is a rising one and the trend in mortality is a declining one. If you put the number of Starbucks coffee-shops in the country rather than GDP per capita, it will be also associated with “improvements in all mortality metrics” as Starbucks are also increasing in number.
  Lack of adjustment for time trends in the variables in more than sufficient to make the results of the regression spurious, but the models in this paper have another major flaw: both unemployment and GDP per capita are included at the same time as explanatory variables in the models. Callum Williams and coauthors seem unaware that these two var...

  Show More

  It is to be commended that the article on the effects of economic crises on population health outcomes in Latin America, by. Callum Williams et al., clearly explains the methods the authors used for the analysis. For that reason, the paper is a very good example of how not to use a specific type of research tool, the panel regression. In a panel regression, as in any time-series investigation of causality, a key issue is to adjust for time trends, so that variables are stationary series (1, 2). If this adjustment is missing, results are biased by trends in the variables. For example, the paper says that “after removing inflation and unemployment as controls from our regression analysis, GDP per capita increases were found to be associated with improvements in all mortality metrics.” This is just an spurious result, as in every country the trend in GDP per capita is a rising one and the trend in mortality is a declining one. If you put the number of Starbucks coffee-shops in the country rather than GDP per capita, it will be also associated with “improvements in all mortality metrics” as Starbucks are also increasing in number.
  Lack of adjustment for time trends in the variables in more than sufficient to make the results of the regression spurious, but the models in this paper have another major flaw: both unemployment and GDP per capita are included at the same time as explanatory variables in the models. Callum Williams and coauthors seem unaware that these two variables are both indexing economic conditions, so that they have a very strong correlation. This generates what statisticians call co-linearity, which creates major uncertainties in interpreting the results of any regression model. But since the correlation is negative because GDP per capita rises when unemployment declines, that co-linearity is, say, on steroids. Of course, these reasons make the conclusions of the study fully unsupported. The idea of Callum Williams et at that you can put in a regression annual data of unemployment rates and GDP per capita reveals they have a very fuzzy idea of how the economy works. Of course, as the paper says, this is the first investigation to put unemployment, GDP per capita and inflation as explanatory variables in a regression modelling mortality as dependent variable. This is a quite an accomplishment!
  A number of previous investigations on Latin American countries (3, 4) have shown that in these countries, as in high-income countries (2, 5-12), mortality tends to rise during economic expansions and decline during recessions.
  Since the confusing studies published by Harvey Brenner in the 1970s, which later were considered as mostly flawed (13, 14), investigations on the macroeconomic effects on mortality have been plagued by many faulty studies (11, 15). The publication of this study by Callum Williams et al. confirms that, regretfully, it is to be expected that poor statistical methods will continue plaguing this field and producing spurious results on which nothing can be inferred (16).

  References
  1. Diggle PJ. Time series: A biostatistical introduction. Oxford ; New York: Oxford University Press; 1989.
  2. Ionides E et al. Macroeconomic effects on mortality revealed by panel analysis with nonlinear trends. Annals of Applied Statistics. 2013;7(3):1362-85.
  3. Tapinos GP, Mason A, Bravo J. Demographic responses to economic adjustment in Latin America. Oxford; New York: Clarendon Press; Oxford University Press; 1997.
  4. Gonzalez F, Quast T. Mortality and business cycles by level of development: Evidence from Mexico. Soc Sci Med. 2010 12;71(12):2066-73.
  5. Ruhm CJ. Are recessions good for your health? Q J Econ. 2000;115(2):617-50.
  6. Lindo JM. Aggregation and the estimated effects of economic conditions on health. J Health Econ. 2015 3;40(0):83-96.
  7. Tapia Granados JA. Macroeconomic fluctuations and mortality in postwar Japan. Demography. 2008;45(2):323-43.
  8. Tapia Granados JA. Recessions and mortality in Spain, 1980-1997. European Journal of Population. 2005 Dec;21:393-422.
  9. Tapia Granados JA. The economic crisis and health in Spain and Europe: Is mortality increasing? / La crisis y la salud en España y en Europa: ¿está aumentando la mortalidad? Salud Colectiva (Buenos Aires). 2014;10(1):81-91.
  10. Tapia Granados JA, House JS, Ionides EL, Burgard SA, Schoeni RF. Individual joblessness, contextual unemployment, and mortality risk. American Journal of Epidemiology. 2014;180(3):280-7.
  11. Tapia Granados JA, Ionides EL. Mortality and macroeconomic fluctuations in contemporary Sweden. Eur J Popul. 2011;27(2):157-84.
  12. Lin S. Economic fluctuations and health outcome: A panel analysis of Asian-Pacific countries. Applied Economics. 2009;41:519-30.
  13. Ruhm CJ. Macroeconomic conditions, health, and government policy. In: Schoeni RF, House JS, Kaplan G, Pollack H, editors. Making Americans healthier: Social and economic policy as health policy. New York: Russell Sage; 2008.
  14. Kasl SV, Jones BA. The impact of job loss and retirement on health. In: Berkman LF, Kawachi I, editors. Social epidemiology. Oxford; New York: Oxford University Press; 2000. p. 118-36.
  15. Tapia Granados JA, Ionides EL. Statistical evidence shows that mortality tends to fall during recessions: a rebuttal to Catalano and Bruckner. International Journal of Epidemiology. 2016 September 15;45(5):1683-5.
  16. Tapia Granados JA. Macroeconomic Effects on Mortality: Issues, Controversies, and Directions for Research. In: Scott R, Buchmann M, editors. Emerging Trends in the Social and Behavioral Sciences.2017. New York: John Wiley; 2017. p. 1-16.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top
• Published on: 1 February 2016

  Author Affiliation

  • Mahiben Maruthappu

  Please note that the author's correct affiliation is: Imperial College London.

  Conflict of Interest:

  None declared

  Conflict of Interest:
  None declared.

  • Back to top