A Monte Carlo Comparison Study of the Power of the Analysis of Covariance, Simple Difference, and Residual Change Scores in Testing Two-Wave Data

Educational and Psychological Measurement - Tập 73 Số 1 - Trang 47-62 - 2013
Yasemin Kisbu‐Sakarya1, David P. MacKinnon1, Leona S. Aiken1
1Arizona State University, Tempe, USA

Tóm tắt

This study compares the analysis of covariance (ANCOVA), difference score, and residual change score methods in testing the group effect for pretest–posttest data in terms of statistical power and Type I error rates using a Monte Carlo simulation. Previous research has mathematically shown the effect of stability of individual scores from pretest to posttest, reliability, and nonrandomization (i.e., pretest imbalance) on the performance of the ANCOVA, difference score, and residual change score methods. However, related power issues have not been adequately addressed. The authors examined the impact of stability of measurement over time, reliability of covariate and criterion, nonrandomization, sample size, and treatment effect size on statistical power of the three methods. Across conditions, ANCOVA and residual change score methods had similar power rates. When reliability was less than perfect, ANCOVA had more power than the difference score method when there was an increase from pretest to posttest and a positive baseline imbalance (i.e., treatment group had higher pretest scores than the control group), or when there was a decrease from pretest to posttest and a negative baseline imbalance, and vice versa. In case of perfect reliability, the statistical power of ANCOVA did not differ from the difference score method. For the difference score method, when reliability was low, there was no effect of stability on power, whereas when reliability was high or perfect, power increased as stability increased for medium and large effect sizes. Difference scores may be preferred over ANCOVA under certain circumstances.

Từ khóa


Tài liệu tham khảo

10.1201/9781420035926

Cohen J., 1988, Statistical power for the behavioral sciences

Cohen J., 2003, Applied multiple regression/correlation analysis for the behavioral sciences

10.1177/00131640021970970

Fitzmaurice G. M., 2004, Applied longitudinal analysis

10.1016/j.jclinepi.2004.12.002

Holland P. W., 1983, Principals of modern psychological measurement, 3

10.1177/0013164495055001004

10.1016/S0167-8760(98)00048-8

10.1177/001316445801800301

10.3102/10769986010003197

10.1177/0193841X0102500101

10.1111/j.1745-3984.2010.00129.x

Rogosa D., 1988, Methodological issues in aging research, 171

Rogosa D., 1995, The analysis of change, 4

10.1037/0033-2909.92.3.726

Thomas D. R., Educational and Psychological Measurement

10.1111/j.1745-3984.1982.tb00124.x

Zumbo B. D., 1999, Advances in social science methodology, 269

Thông tin
Thông tin xuất bản

Educational and Psychological Measurement

Tập 73 Số 1

47-62

Thông tin tác giả