An empirical study of the effect of heterogeneous within-groups variance upon certainF-tests of significance in analysis of variance

In the application of the analysis of variance to data obtained in educational methods experiments which involve several classes of several schools, one assumption is that of homogeneity in the variances of pupil scores from school to school. It is shown that such variances on representative educational achievement tests are heterogeneous. The effects of this heterogeneity upon theF-tests of significance commonly employed in methods experiments are investigated by comparing the actual distribution ofF values for a large number of “experiments” involving marked heterogeneity with a theoretical distribution based on the assumption of homogeneity. Although the findings, which vary somewhat with the type of variance ratio, are not entirely conclusive, they apparently demonstrate that departure from homogeneity does not invalidate the use of the customaryF-tests for evaluating results of the typical methods experiment.