Journal of the American Academy of Child & Adolescent Psychiatry
Invited Column: CLINICIANS’ GUIDE TO RESEARCH METHODS AND STATISTICSMeta-Analysis: Formulation and Interpretation
Section snippets
Criteria for Review
While much of the focus of meta-analysis is on statistical procedures, perhaps the most important part of a meta-analysis is the planning of inclusion and exclusion criteria for selecting a study into the meta-analysis. These inclusion and exclusion criteria are often related to internal validity and external validity. Most researchers feel that meta-analyses composed of randomized control trials (RCTs) represent the gold standard for clinical research. An RCT is distinguished by random
Statistical Computations for Individual Studies
There are numerous types of effect size indices. We have made reference to some of these in previous columns. The most common effect size indices used in meta-analyses are d, r, and odds ratio (OR), although risk ratio (RR) and number needed to treat (NNT) also have been used.
Briefly, the effect size d indicates the strength of a relationship between an independent and dependent variable in standard deviation units. In general, d is used when most of the studies to be included in the
Computation of Effect Size for Combined Studies and Related Statistics
When all studies that meet the criteria for inclusion in the meta-analysis have been coded and effect size data entered, a combined effect size can be computed. Frequently there is an effect size computed for each construct. In Connor and colleagues’ meta-analysis, two different mean effect sizes were computed, one for the construct of overt aggression (d = .84) and one for the construct of covert aggression (d = .69). Each of these mean effect sizes was based on a weighted average. Connor and
Follow-up Procedures
When a test for homogeneity of effect size distribution is statistically significant, the researcher can take a number of steps to explain the heterogeneity (Lipsey and Wilson, 2001).
Before undertaking the task of computing a meta-analysis, it is important to consider what generalizations will be made from the resulting effect size estimate. There are two models from which to choose, one with fixed effects and one with random effects. In a fixed effects model, the researcher is attempting to
Conclusions
Meta-analysis is a valuable tool for both the researcher and the clinician. Summarizing the results of many studies as an effect size index provides important strength of relationship information. Caution always should be used concerning the types of studies that went into the meta-analysis, especially design issues.
REFERENCES (6)
- et al.
Psychopharmacology and aggression, I: a meta-analysis of stimulant effects on overt/covert aggression-related behaviors in ADHD
J Am Acad Child Adolesc Psychiatry
(2002) - et al.
The Cochrane Collaboration in the 20th century
Cited by (30)
Executive functions of sedentary elderly may benefit from walking: A systematic review and meta-analysis
2014, American Journal of Geriatric PsychiatryCitation Excerpt :Effect sizes for the individual studies within both groups were pooled into a meta-analytic effect size, reflecting the meta-analytic effect of walking interventions on executive functions for both groups (weighted by the study inverse variance, thereby accounting for sample size and measurement error).26 Q-testing was used to assess heterogeneity of the data for each meta-analytic effect size.27,28 Subsequently, meta-analytic effect sizes were calculated using a fixed approach for homogeneously distributed effect size data,29 whereas a random approach was used for heterogeneously distributed effect size data.30
Effect size estimation: Methods and examples
2012, International Journal of Nursing StudiesLower limb muscle strength (LLMS): Why sedentary life should never start? A review
2012, Archives of Gerontology and GeriatricsCitation Excerpt :For the interpretation of the correlation coefficients and effect sizes, Cohen's (Cohen, 1988) guidelines were used. To test heterogeneity of the effect sizes, a Q test was conducted (Cochran, 1954; Gliner et al., 2003). To study the possibility of publication bias, we used linear regression methods proposed by Egger et al. (1997) to investigate the degree of funnel plot asymmetry (1-sided p) and an additional fail safe N was calculated (Rosenthal, 1995), measuring the necessary number of studies to nullify the overall effect.
Visual perception and visual-motor integration in very preterm and/or very low birth weight children: A meta-analysis
2012, Research in Developmental DisabilitiesCitation Excerpt :Subgroup means and SDs were weighed by their sample sizes, added, and divided by the sum of the total sample size. An overall combined effect size was computed by weighing the study specific effect sizes by the accompanying sample sizes (Gliner, Morgan, & Harmon, 2003). Effect sizes of 0.20, 0.50 and 0.80 were considered small, medium and large, respectively (Cohen, 1988).
Treatment effects for common outcomes of child sexual abuse: A current meta-analysis
2011, Aggression and Violent BehaviorCitation Excerpt :Studies in this meta-analysis differed by age, type of treatment utilized, and other key variables. Thus, effect sizes from each study were expected to differ, which is consistent with a random effects modeling approach to calculating effect sizes (Gliner, Morgan, & Harmon, 2003). In fact, tests of heterogeneity were significant and remained significant when using random effects modeling.
Efficacy of methylphenidate, psychosocial treatments and their combination in school-aged children with ADHD: A meta-analysis
2008, Clinical Psychology Review
The authors thank Nancy Plummer for manuscript preparation. Parts of the column are adapted, with permission from the publisher and the authors, from Gliner JA, Morgan GA (2000), Research Methods in Applied Settings: An Integrated Approach to Design and Analysis. Mahwah, NJ: Erlbaum. Permission to reprint or adapt any part of this column must be obtained from Erlbaum.