Effect size, confidence interval and statistical significance: a practical guide for biologists

Biological Reviews - Tập 82 Số 4 - Trang 591-605 - 2007
Shinichi Nakagawa1, Innes C. Cuthill2
1Department of Animal and Plant Sciences, University of Sheffield, Sheffield S10 2TN, UK (E-mail: [email protected])
2School of Biological Sciences, University of Bristol, Bristol BS8 1UG, UK (E‐mail: [email protected])

Tóm tắt

AbstractNull hypothesis significance testing (NHST) is the dominant statistical approach in biology, although it has many, frequently unappreciated, problems. Most importantly, NHST does not provide us with two crucial pieces of information: (1) the magnitude of an effect of interest, and (2) the precision of the estimate of the magnitude of that effect. All biologists should be ultimately interested in biological importance, which may be assessed using the magnitude of an effect, but not its statistical significance. Therefore, we advocate presentation of measures of the magnitude of effects (i.e. effect size statistics) and their confidence intervals (CIs) in all biological journals. Combined use of an effect size and its CIs enables one to assess the relationships within data more effectively than the use ofpvalues, regardless of statistical significance. In addition, routine presentation of effect sizes will encourage researchers to view their results in the context of previous research and facilitate the incorporation of results into future meta‐analysis, which has been increasingly used as the standard method of quantitative review in biology. In this article, we extensively discuss two dimensionless (and thus standardised) classes of effect size statistics:dstatistics (standardised mean difference) andrstatistics (correlation coefficient), because these can be calculated from almost all study designs and also because their calculations are essential for meta‐analysis. However, our focus on these standardised effect size statistics does not mean unstandardised effect size statistics (e.g. mean difference and regression coefficient) are less important. We provide potential solutions for four main technical problems researchers may encounter when calculating effect size and CIs: (1) when covariates exist, (2) when bias in estimating effect size is possible, (3) when data have non‐normal error structure and/or variances, and (4) when data are non‐independent. Although interpretations of effect sizes are often difficult, we provide some pointers to help researchers. This paper serves both as a beginner’s instruction manual and a stimulus for changing statistical practice for the better in the biological sciences.

Từ khóa


Tài liệu tham khảo

10.4135/9781412984560

10.7326/0003-4819-134-8-200104170-00012

American Psychological Association, 2001, Publication Manual of the American Psychological Association

10.1016/S0169-5347(00)89073-4

10.1016/j.pt.2003.11.008

10.1111/j.2044-8317.1988.tb00901.x

10.1177/014920630302900106

Burnham K. P., 2002, Model Selection and Multimodel Inference: a Practical Information‐Theoretic Approach

10.1111/j.1461-0248.2004.00702.x

10.1016/j.tree.2006.03.016

Clark J. S., 2001, Design and Analysis of Ecological Experiments, 327, 10.1093/oso/9780195131871.003.0017

Cohen J, 1988, Statistical Power Analysis for the Behavioral Sciences

10.1037/0003-066X.45.12.1304

10.1037/0003-066X.49.12.997

10.4135/9781412984010

Crawley M. J, 2002, Statistical Computing: an Introduction to Data Analysis Using S‐Plus

10.1177/0013164401614002

10.1017/CBO9780511802843

Dixon P. M, 2001, Design and Analysis of Ecological Experiments, 267, 10.1093/oso/9780195131871.003.0014

Dobson A. J, 2002, An Introduction to Generalized Linear Models

10.1037/1082-989X.1.2.170

10.1002/9780470693926

10.1111/j.1461-0248.2004.00603.x

10.1111/j.1095-8312.2005.00540.x

Faraway J. J, 2005, Linear Models with R

Faraway J. J, 2006, Extending the Linear Model with R

10.1086/209469

10.1111/j.1523-1739.2006.00525.x

10.1016/j.socec.2004.09.035

Fisher R. A, 1935, The Design of Experiments

Fletcher T. D, 2007, The psychometric Package: Appliced Psychometric Theory

Fleiss J. L, 1994, The Handbook of Research Synthesis, 245

Fox J, 2002, An R and S‐Plus Companion to Applied Regression

Gabbay D. M., 2002, Handbook of the logic of argument and inference

Gage N. L, 1978, The Scientific Basis of the Art of Teaching

10.1093/beheco/ark005

Gelman A., 2007, Data Analysis Using Regression and Multilevel/Hierarchical Models

Glass G. V, 1976, Review of Research in Education Vol. 5, 351

Grafen A., 2002, Modern Statistics for the Life Sciences

10.1037/1082-989X.6.2.135

Grissom R. J., 2005, Effect Sizes for Research: a Broad Practical Approach

Gurevitch J., 1993, Design and Analysis of Ecological Experiments, 347

10.2193/0022-541X(2005)069[0457:ITIWSC]2.0.CO;2

10.1017/CBO9780511542053

Harlow L. L., 1997, What If There Were No Significance Tests?

Hedges L., 1985, Statistical Methods for Meta‐Analysis

10.3102/10769986006002107

10.1515/9781400847310

Hopkins W. G. (2004).New View of Statistics.http://www.sportsci.org/resource/stats/.

10.1177/0013164402062002002

Hunt M, 1997, How Science Takes Stock: the Story of Meta‐Analysis

Hunter J. E., 2004, Methods of Meta-Analysis: Correcting Error and Bias in Research Finding, 10.4135/9781412985031

10.1111/1467-9876.00197

10.1093/beheco/14.3.438

10.1016/j.tree.2003.10.013

10.2466/PMS.98.1.3-18

10.1007/978-1-4613-1839-2_9

10.2307/2111095

10.1177/0013164496056005002

10.1037/10693-000

Lipsey M. W., 1993, The efficacy of psychological educational, and behavioral treatment: conformation from meta‐analysis, American Psychologist, 48, 1181, 10.1037/0003-066X.48.12.1181

Lipsey M. W., 2001, Practical Meta‐Analysis

10.2307/2111505

Maindonald J., 2003, Data Analysis and Graphics Using R: an Example‐Based Approach

Manly B. R. J, 2007, Randomization, Bootstrap and Monte Carlo Methods in Biology

10.1017/CBO9780511802454

10.1007/978-1-4899-3242-6

10.1007/978-1-4757-3449-2

10.1111/j.1540-6261.1973.tb01423.x

10.1093/beheco/arh107

10.1007/s10211-004-0095-z

10.1037/1082-989X.5.2.241

10.1016/S1471-4922(03)00149-1

10.1007/978-1-4419-0318-1

10.1017/CBO9780511806384

Rice J. A, 1995, Mathematical Statistics and Data Analysis

Rosenberg M. S., 2000, MetaWin: Statistical Software for Meta‐Analysis

Rosenthal R, 1994, The Handbook of Research Synthesis, 231

Rosenthal R., 2000, Contrasts and Effect Sizes in Behavioral Research: A Correlational Approach

10.1037/1082-989X.1.2.115

Shadish W. R., 1994, Handbook of Research Synthesis, 261

Shadish W. R., 1999, ES

10.1177/00131640121971392

Snijders T., 1999, Multilevel Analysis: an Introduction to Basic and Advanced Multilevel Modeling

10.1016/j.tree.2006.12.003

10.1111/j.1365-2664.2005.01002.x

10.1080/00220970109599499

10.1002/j.1556-6678.2002.tb00167.x

10.3102/0013189X031003025

10.1007/978-0-387-21706-2

10.1037/0003-066X.54.8.594

10.1111/j.1365-2656.2006.01141.x

Woodworth G. G, 2004, Biostatistics: a Bayesian Introduction

10.2307/20167258

Zar J, 1999, Biostatistical Analysis

Thông tin
Thông tin xuất bản

Biological Reviews

Tập 82 Số 4

591-605

Thông tin tác giả