Executive Function and the Frontal Lobes: A Meta-Analytic Review

Currently, there is debate among scholars regarding how to operationalize and measure executive functions. These functions generally are referred to as “supervisory” cognitive processes because they involve higher level organization and execution of complex thoughts and behavior. Although conceptualizations vary regarding what mental processes actually constitute the “executive function” construct, there has been a historical linkage of these “higher-level” processes with the frontal lobes. In fact, many investigators have used the term “frontal functions” synonymously with “executive functions” despite evidence that contradicts this synonymous usage. The current review provides a critical analysis of lesion and neuroimaging studies using three popular executive function measures (Wisconsin Card Sorting Test, Phonemic Verbal Fluency, and Stroop Color Word Interference Test) in order to examine the validity of the executive function construct in terms of its relation to activation and damage to the frontal lobes. Empirical lesion data are examined via meta-analysis procedures along with formula derivatives. Results reveal mixed evidence that does not support a one-to-one relationship between executive functions and frontal lobe activity. The paper concludes with a discussion of the implications of construing the validity of these neuropsychological tests in anatomical, rather than cognitive and behavioral, terms.

Appendices

APPENDIX A

Formulas and Procedures for Primary Meta- Analysis

Preliminary Issues

All computations described below were carried out using the following three computer programs – DSTAT (Johnson, 1989), Meta-Analysis Programs (Schwarzer, 1989), and Statistical Package for the Social Sciences (SPSS) – and by hand using formulas cited in Cooper and Hedges (1994), Hedges and Olkin (1985), Orwin (1983), and Rosenthal (1991).

Calculation of Effect Sizes

The unbiased estimator d was chosen as the effect size estimator for the meta-analysis in this paper. Although there are a variety of effect size indicators from which to choose (Glass, 1976; Rosenthal, 1991), the decision to use d was based on theoretical and practical matters. First, d was selected because the data to be examined were represented as differences between groups (usually means and standard deviations). Estimators in the d family are better suited than estimators in the r family for studying the strength and direction of mean differences between groups (Hedges and Olkin, 1985). Second, the unbiased estimator d adjusts for bias, that is, it adjusts each effect size to control for a standard discrepancy between the sample effect size and the population effect size (Hedges and Olkin, 1985). Third, with respect to more practical concerns, the majority of studies reported means and standard deviations, which reduced the need to transform original findings to another effect size index such as r. Lastly, for the present review, negative effect sizes indicate that persons with frontal damage performed worse than the control group (i.e., either healthy controls or persons with non-frontal damage). For instance, a negative effect size on the COWA indicates that the frontal group produced fewer words than the control group. For those samples in which a lower score reflected better performance (e.g., fewer perseverative errors on the WCST), the sign of the effect size was reversed.

When means and standard deviations were available, the following formula was used to calculate d (Rosenthal, 1991):

$$ d = M_1 - M_2 /\sigma _{{\rm pooled}} $$

(A.1)

where M ₁ and M ₂ are means for groups one and two, respectively, and, σ_pooled (the pooled within-group standard deviation) was computed as follows:

$$ \sigma _{{\rm pooled}} = \sqrt {\frac{{N_1 SD_1^2 + N_2 SD_2^2 }}{{N_1 + N_2 - 2}} + \left( {\frac{1}{{N_1 }} + \frac{1}{{N_2 }}} \right)} $$

(A.2)

When means and standard deviations were not available, t-values, F-values, or p-values were used to calculate d using DSTAT (Johnson, 1989) or Meta-Analysis Programs (Schwarzer, 1989).

Nonindependence

Several studies in this analysis employed more than one EF measure (e.g., WCST and verbal fluency). In addition, the same researcher or team of researchers often conducted more than one study examining the relationship between EF measures and the frontal lobes. While including these studies would violate the assumption of independent samples (Hedges and Olkin, 1985), excluding them would result in too few studies to conduct a meta-analysis with adequate statistical power. Thus, four possible approaches to dealing with this problem (Mullen, 1989) were considered: (a) use each effect size as if it came from an independent sample; (b) use the results from the best EF measure; (c) conduct individual meta-analyses for each EF measure; or (d) average the effect sizes of the different EF measures within each study to form one estimate. The last three approaches were discarded for the following three reasons, respectively: (a) there are no criteria available for determining which EF measure is the “best;” (b) there are too few studies to conduct individual meta-analyses for each EF measure; and (c) averaging the effect sizes of the different EF measures within each study would lose information about the uniqueness of each measure and its possible moderating effect on the relationship between EFs and the frontal lobes. Thus, the first approach was chosen (i.e., each effect size was treated as if it came from an independent sample).

Combining Effect Sizes

The individual effect sizes from the 27 studies listed in Table 8 were averaged to form an unweighted grand mean estimate of the relationship between frontal lobe lesions and performance on the WCST, phonemic verbal fluency, and the Stroop test using the following formula:

$$ \overline d = \frac{{\sum\limits_{i = 1}^k {d_i } }}{k} $$

(A.3)

Table 8. Lesion Studies Included in Meta-Analysis

where k is the number of effect sizes combined and d _i is the aggregated effect size from the ith study.

Next, effect sizes were combined after weighting each effect size by its sample size:

$$ d_ + = \frac{{\sum\limits_{i = 1}^k {\frac{{d_i }}{{\frown^\sigma}^2 d_i }} }} {{\sum\limits_{i = 1}^k {\frac{1}{{\frown^\sigma}^2 d_i }}}} $$

(A.4)

where the variance of d is defined as:

$$ \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}}\over \sigma } ^2 _{d_i } = \frac{{n_F + n_C }}{{n_F n_C }} + \frac{{d_i^2 }}{{2\left( {n_F + n_C } \right)}} $$

(A.5)

where n_F is the sample size for the focal group and n_C is the sample size for the control group. Weighting the studies by sample size allowed more emphasis to be placed on studies with larger samples, thereby producing more precise effect size estimates (Hedges and Olkin, 1985 ).

For purposes of interpretation, the strength of obtained effect sizes was evaluated according to criteria outlined in Cohen (1992), that is, a d of .20 indicates a small effect, .50 indicates a moderate effect, and .80 indicates a large effect.

Tests of Homogeneity of Effect Sizes

The Q statistic outlined in formula 6 indicates whether the amount of variance in the 27 studies used to obtain an estimate of the population effect is greater than what would be expected based upon sampling error alone (Hedges and Olkin, 1985).

$$ Q = \sum\limits_{i = 1}^k {\frac{{d_i ^2 }}{{\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}}\over \sigma } ^2 d_i }}} - \frac{{\left( {\sum\limits_{i = 1}^k {\frac{{d_i ^2 }}{{\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}}\over \sigma } ^2 d_i }}} } \right)^2 }}{{\sum\limits_{i = 1}^k {\frac{1}{{\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}}\over \sigma } ^2 d_i }}} }} $$

(A.6)

Estimation of the “Fail-safe N”

There is a risk that the studies sampled for this meta-analysis only comprise a subset of the existing research on EF measures and frontal lobe functioning due to the problem that journals often accept for publication only those studies with “significant” results. Although there were studies included in this meta-analysis that did not find significant differences between groups, the 27 studies in this quantitative review may not adequately represent the entire population of studies on this topic. Thus, a “fail-safe N” statistic was computed as an estimate of the number of additional studies with null results (i.e., effect sizes equal to zero) that would be needed to reduce the weighted combined mean effect size to non-significance (Orwin, 1983). A fail-safe N was calculated using the following formula:

$$ N_{{\rm fail - safe}} = \frac{{N_{{\rm total}} \left( {d_ + - d_{{\rm crit}} } \right)}}{{d_{{\rm crit}} }}$$

(A.7)

where N _total is the number of effect sizes included in the meta-analysis, d _crit is the critical value of d, and d₊ is the weighted mean effect size.

APPENDIX B

Formulas and Procedures for Moderator Analyses

Preliminary Issues

All computations described below were carried out using the following three computer programs – DSTAT (Johnson, 1989), Meta-Analysis Programs (Schwarzer, 1989), and Statistical Package for the Social Sciences (SPSS) – and by hand using formulas cited in Cooper and Hedges (1994).

Examination of Moderator Variables

Analyses of potential moderator variables were conducted by means of fixed effects strategies based on whether the variables of interest were categorical or continuous (Hedges, 1994). For categorical moderators (i.e., type of test and comparison group), estimates of both between (Q _b) and within (Q _w) group variances were derived and tested along the χ ² distribution to determine whether they were statistically significant moderators. A significant Q _b indicates that there is significant variability between the groups that comprise the categorical moderator variable that is greater than what would be expected simply by chance. On the other hand, a significant Q _w indicates that there is still significant variability within each effect size that is not being explained by the categorical moderator. Thus, a categorical moderator variable explains all of the heterogeneity present in the grand mean effect size only when the variance is significant between groups (and is not significant within groups). Similar to the analysis of variance (ANOVA), the Q _b statistic provides an omnibus test for between-group differences. Follow-up contrasts should be conducted when Q _b is significant and there are three or more levels of the moderator variable. [Note that only “type of test” had three levels (i.e., WCST, verbal fluency, and Stroop). A significant Q _w was obtained along with a significant Q _b for this moderator variable and, thus, a contrast analysis was conducted accordingly.]

The following computational formulas, which weight effect sizes by sample size, were used for the two categorical moderator analyses (Hedges, 1994):

$$ TW = \sum\limits_{i = 1}^j {(n_i - 3} ) $$

(B.8)

$$ TWD = \sum\limits_{i = 1}^j {(n_i - 3} )d_i $$

(B.9)

$$ TWDS = \sum\limits_{i = 1}^j {(n_i - 3)d_i ^2 } $$

(B.10)

$$ Q_{{\rm Total}} = {\rm TWDS} - \frac{{({\rm TWD})^2 }}{{TW}}$$

(B.11)

$$ Q_{wi} = {\rm TWDS}_i - \frac{{{\rm TWD}_i ^2 }}{{{\rm TW}_i }} $$

(B.12)

$$Q_b = Q_T - Q_w$$

(B.13)

[For formulas 8–10, j = total number of cases in each sub-group.]

Weighted least squares regression procedures (WLS) were used to test the continuous moderator, age (Hedges, 1994). Effect sizes were weighted by their sample size and then regressed onto the relevant predictor variable (i.e., age). A Z-test of the unstandardized regression coefficient (b) was used to determine the statistical significance (i.e., whether b differed significantly from zero) of the moderator using formulas 14–16; 95% confidence intervals were constructed according to formula 16:

$$Z_j = \frac{{b_j }}{{S_j }}$$

(B.14)

$$S_j = \frac{{SE_j }}{{\sqrt {MS_{{\rm error}} } }}$$

(B.15)

$$b_j \pm 1.96(S_j )$$

(B.16)

j = 1, …, k; k = total # of predictors in equation.