Table 1

Simulation estimates of performance characteristics including mean and Monte Carlo Standard Error in parantheses of different models using each DGP.

ModelDGP 1: Embedded Image = 0DGP 2: Embedded Image = −3DGP 3: Embedded Image = −3DGP 4: Embedded Image = −3
Estimate
 MLM1.1(0.0024)−1.36 (0.0023)−0.26(0.0024)−0.23 (0.0025)
 ML Tobit−0.0056(0.0038)−3.03 (0.0037)−3.04(0.0037)−3.01 (0.0037)
 ML Tobit Embedded Image = 5−0.13(0.0044)−3.01 (0.0042)−3.13(0.0046)−2.57 (0.0038)
 ML Tobit Embedded Image = 10−0.093(0.0044)−3.09 (0.0042)−3.14(0.0045)−3.05 (0.0041)
 ML Tobit Embedded Image = 15−0.057(0.0044)−3.09 (0.0042)−3.12(0.0045)−3.11 (0.0043)
 ML Tobit Embedded Image = 20−0.04(0.0044)−3.08 (0.0042)−3.11(0.0045)−3.12 (0.0044)
 ML Tobit Embedded Image = 25−0.031(0.0044)−3.07 (0.0042)−3.1(0.0045)−3.11 (0.0044)
 ML Tobit Embedded Image = 30−0.026(0.0044)−3.07 (0.0042)−3.09(0.0045)−3.1 (0.0045)
 OLS SACS1.1(0.0024)−1.36 (0.0023)−0.26(0.0024)−0.23 (0.0025)
 OLS ANCOVA−0.31(0.0022)−1.36 (0.002)−1.69(0.0021)−2.43 (0.0019)
 OLS Post−1.36(0.0023)−1.36 (0.0022)−2.72(0.0023)−2.69 (0.0018)
 Tobit SACS−0.72(0.0044)−3.04 (0.0041)−3.78(0.0044)−4.83 (0.0048)
 Tobit ANCOVA−0.5(0.0046)−3.09 (0.0042)−3.61(0.0045)−5.5 (0.0042)
 Tobit Post−3.07(0.0049)−3.06 (0.0047)−6.13(0.005)−6.14 (0.004)
Coverage
 MLM0(0)0 (0)0(0)0 (0)
 ML Tobit94.8(0.7)95.3 (0.67)93.9(0.76)95.4 (0.66)
 ML Tobit Embedded Image = 567.5(1.48)86.7 (1.07)71.6(1.43)4.6 (0.66)
 ML Tobit Embedded Image = 1083.5(1.17)85 (1.13)77.4(1.32)92.2 (0.85)
 ML Tobit Embedded Image = 1591.1(0.9)87.7 (1.04)85.7(1.11)86.8 (1.07)
 ML Tobit Embedded Image = 2092.7(0.82)90 (0.95)88.2(1.02)87.1 (1.06)
 ML Tobit Embedded Image = 2593.4(0.79)91.6 (0.88)89.5(0.97)88.2 (1.02)
 ML Tobit Embedded Image = 3093.6(0.77)91.9 (0.86)91.1(0.9)88.9 (0.99)
 OLS SACS0(0)0 (0)0(0)0 (0)
 OLS ANCOVA0.8(0.28)0 (0)0(0)0 (0)
 OLS post0(0)0 (0)2.4(0.48)0 (0)
 Tobit SACS0.1(0.1)94.4 (0.73)0(0)0 (0)
 Tobit ANCOVA7.2(0.82)89.7 (0.96)1.2(0.34)0 (0)
 Tobit post0(0)93.3 (0.79)0(0)0 (0)
Model SE
 MLM0.074(2E-05)0.074 (2E-05)0.077(2E-05)0.078 (2E-05)
 ML Tobit0.12(3E-05)0.12 (3E-05)0.12(3E-05)0.12 (3E-05)
 ML Tobit Embedded Image = 50.1(3E-05)0.1 (3E-05)0.11(3E-05)0.11 (4E-05)
 ML Tobit Embedded Image = 100.12(3E-05)0.12 (3E-05)0.13(4E-05)0.12 (3E-05)
 ML Tobit Embedded Image = 150.13(4E-05)0.13 (4E-05)0.14(4E-05)0.13 (3E-05)
 ML Tobit Embedded Image = 200.13(5E-05)0.13 (5E-05)0.14(5E-05)0.14 (4E-05)
 ML Tobit Embedded Image = 250.14(5E-05)0.13 (5E-05)0.14(6E-05)0.14 (5E-05)
 ML Tobit Embedded Image = 300.14(5E-05)0.13 (5E-05)0.15(6E-05)0.14 (5E-05)
 OLS SACS0.074(2E-05)0.074 (2E-05)0.077(2E-05)0.078 (2E-05)
 OLS ANCOVA0.07(2E-05)0.065 (2E-05)0.072(2E-05)0.059 (2E-05)
 OLS post0.07(2E-05)0.07 (2E-05)0.072(2E-05)0.055 (2E-05)
 Tobit SACS0.13(5E-05)0.13 (5E-05)0.14(6E-05)0.15 (6E-05)
 Tobit ANCOVA0.14(6E-05)0.14 (6E-05)0.15(6E-05)0.13 (6E-05)
 Tobit post0.15(7E-05)0.15 (6E-05)0.16(7E-05)0.13 (6E-05)
  • ANCOVA, analysis of covariance; DGP, data generating process; MLM, multilevel model; OLS, ordinarily least squares; SACS, Simple Analysis of Change Scores.