On making causal claims: A review and recommendations

Aiken, 2008, Doctoral training in statistics, measurement, and methodology in psychology — Replication and extension of Aiken, West, Sechrest, and Reno's (1990) survey of PhD programs in North America, American Psychologist, 63, 32, 10.1037/0003-066X.63.1.32 Angrist, 1999, Empirical strategies in labor economics, 3, 1277 Angrist, 2001, Instrumental variables and the search for identification: From supply and demand to natural experiments, Journal of Economic Perspectives, 15, 69, 10.1257/jep.15.4.69 Angrist, 2008 Antonakis, 2009, “Emotional intelligence”: What does it measure and does it matter for leadership?, VII, 163 Antonakis, J. (in press). Predictors of leadership: The usual suspects and the suspect traits. In A. Bryman, D. Collinson, K. Grint, B. Jackson & M. Uhl-Bien (Eds.), Sage handbook of leadership. Thousand Oaks: Sage Publications. Antonakis, 2009, Does leadership need emotional intelligence?, The Leadership Quarterly, 20, 247, 10.1016/j.leaqua.2009.01.006 Antonakis, 2002, Leader distance: A review and a proposed theory, The Leadership Quarterly, 13, 673, 10.1016/S1048-9843(02)00155-8 Antonakis, 2003, Context and leadership: An examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire, The Leadership Quarterly, 14, 261, 10.1016/S1048-9843(03)00030-4 Antonakis, 2004, Leadership: Past, present, future, 3 Antonakis, 2010, Emotional intelligence: On definitions, neuroscience, and marshmallows, Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 165, 10.1111/j.1754-9434.2010.01219.x Antonakis, J., & Dietz, J. (in press-a). Looking for Validity or Testing It? The Perils of Stepwise Regression, Extreme-Scores Analysis, and Heteroscedasticity. http://dx.doi.org/10.1016/j.paid.2010.09.014. Antonakis, J., & Dietz, J. (in press-b). More on Testing for Validity Instead of Looking for It. Personality and Individual Differences. http://dx.doi.org/10.1016/j.paid.2010.10.008. Antonakis, J., House, R. J., Rowold, J., & Borgmann, L. (submitted for publication). A fuller full-range leadership theory: Instrumental, transformational, and transactional leadership. Baltagi, 2002 Barling, 1996, Effects of tranfsormational leadership training on attitudinal and financial outcomes: A field experiment, Journal of Applied Psychology, 81, 827, 10.1037/0021-9010.81.6.827 Baron, 1986, The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations, Journal of Personality and Social Psychology, 51, 1173, 10.1037/0022-3514.51.6.1173 Bascle, 2008, Controlling for endogeneity with instrumental variables in strategic management research, Strategic Organization, 6, 285, 10.1177/1476127008094339 Basmann, 1960, On finite sample distributions of generalized classical linear identifiability test statistics, Journal of the American Statistical Association, 55, 650, 10.2307/2281588 Baum, 2007, Enhanced routines for instrumental variables/generalized method of moments estimation and testing, The Stata Journal, 7, 465, 10.1177/1536867X0800700402 Bentler, 1990, Comparative fit indexes in structural models, Psychological Bulletin, 107, 238, 10.1037/0033-2909.107.2.238 Bertrand, 2004, How much should we trust differences-in-differences estimates?, Quarterly Journal of Economics, 119, 249, 10.1162/003355304772839588 Bollen, 1989 Bollen, 1990, Overall fit in covariance structure models — 2 types of sample-size effects, Psychological Bulletin, 107, 256, 10.1037/0033-2909.107.2.256 Bollen, 1996, An alternative two stage least squares (2SLS) estimator for latent variable equations, Psychometrika, 61, 109, 10.1007/BF02296961 Bollen, 2007, Latent variable models under misspecification — Two-stage least squares (2SLS) and maximum likelihood (ML) estimators, Sociological Methods & Research, 36, 48, 10.1177/0049124107301947 Breusch, 1980, The Lagrange multiplier test and its applications to model specification in econometrics, Review of Economic Studies, 47, 239, 10.2307/2297111 Browne, 1993, Alternative ways of assessing model fit, 136 Cameron, A. C., Gelbach, J. B., & Miller, D. L. (in press). Robust Inference with Multi-way Clustering. Journal of Business and Economic Statistics. Cameron, 2005 Campbell, 1963, Experimental and quasi-experimental designs for research on teaching, 171 Campbell, 1966 Chen, 2008, An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models, Sociological Methods & Research, 36, 462, 10.1177/0049124108314720 Cohen, 1960, A coefficient agreement for nominal scales, Educational and Psychological Measurement, 20, 37, 10.1177/001316446002000104 Cole, 2007, The insidious effects of failing to include design-driven correlated residuals in latent variable covariance structure analysis, Psychological Methods, 12, 381, 10.1037/1082-989X.12.4.381 Cong, 2001, Treatment effects model, Stata Technical Bulletin, 10, 25 Cook, 2008, “Waiting for life to arrive”: A history of the regression-discontinuity design in psychology, statistics and economics, Journal of Econometrics, 142, 636, 10.1016/j.jeconom.2007.05.002 Cook, 1979 Cook, 2008, Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons, Journal of Policy Analysis and Management, 27, 724, 10.1002/pam.20375 D'Agostino, 1998, Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group, Statistics in Medicine, 17, 2265, 10.1002/(SICI)1097-0258(19981015)17:19<2265::AID-SIM918>3.0.CO;2-B de Cremer, 2004, Leader self-sacrifice and leadership effectiveness: The moderating role of leader self-confidence, Organizational Behavior and Human Decision Processes, 95, 140, 10.1016/j.obhdp.2004.04.002 Diamond, 2010 Draper, 1998 Dvir, 2002, Impact of transformational leadership on follower development and performance: A field experiment, Academy Management Journal, 45, 735, 10.2307/3069307 Eagly, 2004, Women and men as leaders, 279 Eagly, 2003, Transformational, transactional, and laissez-faire leadership styles: A meta-analysis comparing women and men, Psychological Bulletin, 129, 569, 10.1037/0033-2909.129.4.569 Fan, 2005, Sensitivity of fit indexes to misspecified structural or measurement model components: Rationale of two-index strategy revisited, Structural Equation Modeling: A Multidisciplinary Journal, 12, 343, 10.1207/s15328007sem1203_1 Fiori, M., & Antonakis, J. (in press). The ability model of emotional intelligence: Searching for valid measures. Personality and Individual Differences. http://dx.doi.org/10.1016/j.paid.2010.10.010. Fisher, 1922, On the interpretation of χ2 from contingency tables, and the calculation of P, Journal of the Royal Statistical Society, 85, 87, 10.2307/2340521 Foster, 1996, An illustration of the use of instrumental variables: Do neighborhood conditions affect a young person's change of finishing high school?, Psychological Methods, 1, 249, 10.1037/1082-989X.1.3.249 Frese, 2003, Action training for charismatic leadership: Two evaluations of studies of a commercial training module on inspirational communication of a vision, Personnel Psychology, 56, 671, 10.1111/j.1744-6570.2003.tb00754.x Gennetian, 2008, From statistical associations to causation: What developmentalists can learn from instrumental variables techniques coupled with experimental data, Developmental Psychology, 44, 381, 10.1037/0012-1649.44.2.381 Grant, 2009, The neglected science and art of quasi-experimentation: Why-to, when-to, and how-to advice for organizational researchers, Organizational Research Methods, 12, 653, 10.1177/1094428108320737 Greene, 2008 Hahn, 2001, Identification and estimation of treatment effects with a regression-discontinuity design, Econometrica, 69, 201, 10.1111/1468-0262.00183 Hair, 2006 Halaby, 2004, Panel models in sociological research: Theory into practice, Annual Review of Sociology, 30, 507, 10.1146/annurev.soc.30.012703.110629 Hamilton, 2003, Correcting for endogeneity in strategic management research, Strategic Organization, 1, 51, 10.1177/1476127003001001218 Hansen, 1982, Large sample properties of generalized method of moments estimators, Econometrica, 50, 1029, 10.2307/1912775 Hausman, 1978, Specification tests in econometrics, Econometrica, 46, 1251, 10.2307/1913827 Hausman, 1981, Panel data and unobservable individual effects, Econometrica, 49, 1377, 10.2307/1911406 Hayduk, 1996 Hayduk, 2007, Testing! testing! one, two, three — Testing the theory in structural equation models!, Personality and Individual Differences, 42, 841, 10.1016/j.paid.2006.10.001 Heckman, 1979, Sample selection bias as a specification error, Econometrica, 47, 153, 10.2307/1912352 Henseler, 2010, On the convergence of the partial least squares path modeling algorithm, Computational Statistics, 25, 107, 10.1007/s00180-009-0164-x Hofmann, 1997, An overview of the logic and rationale of hierarchical linear models, Journal of Management, 23, 723, 10.1177/014920639702300602 Howell, 1986, A laboratory study of charismatic leadership, Organizational Behavior and Human Decision Processes, 43, 243, 10.1016/0749-5978(89)90052-6 Huber, 1967, The behavior of maximum likelihood estimates under nonstandard conditions, Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1, 221 Hwang, 2010, A Comparative Study on Parameter Recovery of Three Approaches to Structural Equation Modeling, Journal of Marketing Research, 37, 699, 10.1509/jmkr.47.4.699 James, 1982 Jones, 2005, Do leaders matter? National leadership and growth since World War II, The Quarterly Journal of Economics, 835 Kennedy, 2003 Kenny, 1979 Keppel, 2004 Kline, 2010 Kmenta, 1986 Koh, 1995, The effects of transformational leadership on teacher attitudes and student performance in Singapore, Journal of Organizational Behavior, 16, 319, 10.1002/job.4030160404 Lalive, 2008, How do extended benefits affect unemployment duration? A regression discontinuity approach, Journal of Econometrics, 142, 785, 10.1016/j.jeconom.2007.05.013 Landis, 1977, The measurement of observer agreement for categorical data, Biometrics, 33, 159, 10.2307/2529310 Larcker, 2010, On the use of instrumental variables in accounting research, Journal of Accounting and Economics, 49, 186, 10.1016/j.jacceco.2009.11.004 Lee, 2009, Regression discontinuity designs in economics, 10.1596/1813-9450-5097 Levitt, 1997, Using electoral cycles in police hiring to estimate the effects of police on crime, American Economic Review, 87, 270 Levitt, 2002, Using electoral cycles in police hiring to estimate the effects of police on crime: Reply, American Economic Review, 92, 1244, 10.1257/00028280260344777 Liden, 2009, Considering context in psychological leadership research, Human Relations, 62, 1587, 10.1177/0018726709346374 Loehlin, 1992 Long, 2006 Maddala, 1977 Maddala, 1983 Marcoulides, 2006, PLS: A silver bullet?, MIS Quarterly, 30, III, 10.2307/25148727 Marsh, 2004, In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings, Structural Equation Modeling: A Multidisciplinary Journal, 11, 320, 10.1207/s15328007sem1103_2 Maxwell, 1996 Maxwell, 1991, A comparison of methods for increasing power in randomized between-subjects designs, Psychological Bulletin, 110, 328, 10.1037/0033-2909.110.2.328 McDonald, 1996, Path analysis with composite variables. [Article], Multivariate Behavioral Research, 31, 239, 10.1207/s15327906mbr3102_5 McIntosh, 2007, Rethinking fit assessment in structural equation modelling: A commentary and elaboration on Barrett (2007), Personality and Individual Differences, 42, 859, 10.1016/j.paid.2006.09.020 Meyer, 1995, Natural and quasi-experiments in economics, Journal of Business & Economics Statistics, 13, 151, 10.2307/1392369 Mooney, 1997 Morgan, 2007 Mount, 2001, Multisource feedback ratings: What do they really measure?, 155 Mulaik, 1995, Objectivity and reasoning in science and structural equation modeling, 118 Mundlak, 1978, Pooling of time-series and cross-section data, Econometrica, 46, 69, 10.2307/1913646 Muthén, 1989, Latent variable modeling in heterogenous populations, Psychometrika, 54, 557, 10.1007/BF02296397 Muthén, 2007 Nickerson, 1998, Confirmation bias: A ubiquitous phenomenon in many guises, Review of General Psychology, 2, 175, 10.1037/1089-2680.2.2.175 Onyskiw, 2001, Processes underlying childern's adjustment in families characterized by physical aggression, Family Relations, 50, 376, 10.1111/j.1741-3729.2001.00376.x Pearl, 2000 Podsakoff, 2003, Common method biases in behavioral research: A critical review of the literature and recommended remedies, Journal of Applied Psychology, 89, 879, 10.1037/0021-9010.88.5.879 Podsakoff, 1986, Self-reports in organizational research: Problems and prospects, Journal of Management, 12, 531, 10.1177/014920638601200408 Rabe-Hesketh, 2008 Ramsey, 1969, Tests for specification errors in classical linear least-squares regression analysis, Journal of the Royal Statistical Society, Series B, 31, 350 Richardson, 2009, A tale of three perspectives examining post hoc statistical techniques for detection and correction of common method variance, Organizational Research Methods, 12, 762, 10.1177/1094428109332834 Roodman Rosenbaum, 1983, The central role of the propensity score in observational studies for causal effects, Biometrika, 70, 41, 10.1093/biomet/70.1.41 Rosenbaum, 1984, Reducing bias in observational studies using subclassification on the propensity score, Journal of the American Statistical Association, 79, 516, 10.2307/2288398 Rosenbaum, 1985, Constructing a control-group using multivariate matched sampling methods that incorporate the propensity score, American Statistician, 39, 33, 10.2307/2683903 Rubin, 1974, Estimating causal effects of treatments in randomized and nonrandomized studies, Journal of Educational Psychology, 66, 688, 10.1037/h0037350 Rubin, 2008, For objective causal inference, design trumps analysis, Annals of Applied Statistics, 2, 808, 10.1214/08-AOAS187 Rubin, 1996, Matching using estimated propensity scores: Relating theory to practice, Biometrics, 52, 249, 10.2307/2533160 Sargan, 1958, The estimation of economic relationships using instrumental variables, Econometrica, 26, 393, 10.2307/1907619 Schaffer Schriesheim, 2001, The folly of theorizing “A” but testing “B”: A selective level-of-analysis review of the field and a detailed Leader–Member Exchange illustration, The Leadership Quarterly, 12, 515, 10.1016/S1048-9843(01)00095-9 Scullen, 2000, Understanding the latent structure of job performance ratings, Journal of Applied Psychology, 85, 956, 10.1037/0021-9010.85.6.956 Shadish, 1999, Comment-design rules: More steps toward a complete theory of quasi-experimentation, Statistical Science, 14, 294 Shadish, 2009, The renaissance of field experimentation in evaluating interventions, Annual Review of Psychology, 60, 607, 10.1146/annurev.psych.60.110707.163544 Shadish, 2002 Shaver, 1998, Accounting for endogeneity when assessing strategy performance: Does entry mode choice affect FDI survival?, Management Science, 44, 571, 10.1287/mnsc.44.4.571 Shipley, 2000 Shrout, 2002, Mediation in experimental and nonexperimental studies: New procedures and recommendations, Psychological Methods, 7, 422, 10.1037/1082-989X.7.4.422 Sobel, 1982, Asymptotic intervals for indirect effects in structural equations models, 290 Spector, 2006, Method variance in organizational research — Truth or urban legend?, Organizational Research Methods, 9, 221, 10.1177/1094428105284955 StataCorp. (2009). Stata Statistical Software: Release 11. College Station, TX: StataCorp LP. Steiger, 2001, Driving fast in reverse — The relationship between software development, theory, and education in structural equation modeling, Journal of the American Statistical Association, 96, 331, 10.1198/016214501750332893 Stock, 2002, A survey of weak instruments and weak identification in generalized method of moments, Journal of Business & Economic Statistics, 20, 518, 10.1198/073500102288618658 Temme, D., Kreis, H., & Hildebrandt, L. (2006). PLS Path Modeling—A Software Review. SFB 649 Discussion Paper 2006-084, Institute of Marketing, Humboldt-Universität zu Berlin, Germany. Thistlethwaite, 1960, Regression-discontinuity analysis: An alternative to the ex post facto experiment, Journal of Educational Psychology, 51, 309, 10.1037/h0044319 Tobin, 1958, Estimation of relationships for limited dependent variables, Econometrica, 26, 24, 10.2307/1907382 Tucker, 1973, A reliability coefficient for maximum likelihood factor analysis, Psychometrika, 38, 1, 10.1007/BF02291170 White, 1980, A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity, Econometrica, 48, 817, 10.2307/1912934 Widaman, 1985, Hierarchically nested covariance structure models for multitrait-multimethod data, Applied Psychological Measurement, 9, 1, 10.1177/014662168500900101 Winship, 1999, The estimation of causal effects from observational data, Annual Review of Sociology, 25, 659, 10.1146/annurev.soc.25.1.659 Wooldridge, 2002 Wooldridge, 2009 Wooldrige, 2002 Yammarino, 2005, Leadership and levels of analysis: A state-of-the-science review, The Leadership Quarterly, 16, 879, 10.1016/j.leaqua.2005.09.002 Yin, 1994 Zellner, 1962, 3-Stage least-squares — Simultaneous estimation of simultaneous-equations, Econometrica, 30, 54, 10.2307/1911287