Small Effects: The Indispensable Foundation for a Cumulative Psychological Science

Perspectives on Psychological Science - Tập 17 Số 1 - Trang 205-215 - 2022
Friedrich M. Götz1,2, Samuel D. Gosling3,4, Peter J. Rentfrow2
1Department of Psychology, University of British Columbia
2Department of Psychology, University of Cambridge
3Department of Psychology, University of Texas at Austin
4Melbourne School of Psychological Sciences, University of Melbourne

Tóm tắt

We draw on genetics research to argue that complex psychological phenomena are most likely determined by a multitude of causes and that any individual cause is likely to have only a small effect. Building on this, we highlight the dangers of a publication culture that continues to demand large effects. First, it rewards inflated effects that are unlikely to be real and encourages practices likely to yield such effects. Second, it overlooks the small effects that are most likely to be real, hindering attempts to identify and understand the actual determinants of complex psychological phenomena. We then explain the theoretical and practical relevance of small effects, which can have substantial consequences, especially when considered at scale and over time. Finally, we suggest ways in which scholars can harness these insights to advance research and practices in psychology (i.e., leveraging the power of big data, machine learning, and crowdsourcing science; promoting rigorous preregistration, including prespecifying the smallest effect size of interest; contextualizing effects; changing cultural norms to reward accurate and meaningful effects rather than exaggerated and unreliable effects). Only once small effects are accepted as the norm, rather than the exception, can a reliable and reproducible cumulative psychological science be built.

Từ khóa


Tài liệu tham khảo

Abelson R. P. (1985). A variance explanation paradox: When a little is a lot. Psychological Bulletin, 97, 129–133. https://doi.org/10.1037/0033-2909.97.1.129

Adjerid I., Kelley K. (2018). Big data in psychology: A framework for research advancement. American Psychologist, 73, 899–917. https://doi.org/10.1037/amp0000190

Ahadi S., Diener E. (1989). Multiple determinants and effect size. Journal of Personality and Social Psychology, 56, 398–406. https://doi.org/10.1037/0022-3514.56.3.398

Altshuler D., Daly M. J., Lander E. S. (2008). Genetic mapping in human disease. Science, 322, 881–888. https://doi.org/10.1126/science.1156409

Anderson C. A., Boucher G., Lees C. W., Franke A., D’Amato M., Taylor K. D., Lagacé C. (2011). Meta-analysis identifies 29 additional ulcerative colitis risk loci, increasing the number of confirmed associations to 47. Nature Genetics, 43, 246–252. https://doi.org/10.1038/ng.764

Anvari F., Lakens D. (2019). Using anchor-based methods to determine the smallest effect size of interest. PsyArXiv. https://doi.org/10.31234/osf.io/syp5a

Blanton H., Jaccard J. (2006). Arbitrary metrics in psychology. American Psychologist, 61, 27–41. https://doi.org/10.1037/0003-066x.61.1.27

Bleidorn W., Hill P. L., Back M. D., Denissen J. J. A., Hennecke M., Hopwood C. J., Jokela M., Kandler C., Lucas R. E., Luhmann M., Orth U., Wagner J., Wrzus C., Zimmermann J., Roberts B. (2019). The policy relevance of personality traits. American Psychologist, 74(9), 1056–1067. https://doi.org/10.1037/amp0000503

Bleidorn W., Hopwood C. J. (2019). Using machine learning to advance personality assessment and theory. Personality and Social Psychology Review, 23, 190–203. https://doi.org/10.1177/1088868318772990

Bleidorn W., Hopwood C. J., Lucas R. E. (2016). Life events and personality trait change. Journal of Personality, 86, 83–96. https://doi.org/10.1111/jopy.12286

Bleidorn W., Kandler C., Hülsheger U. R., Riemann R., Angleitner A., Spinath F. M. (2010). Nature and nurture of the interplay between personality traits and major life goals. Journal of Personality and Social Psychology, 99, 366–379. https://doi.org/10.1037/a0019982

Bond R. M., Fariss C. J., Jones J. J., Kramer A. D. I., Marlow C., Settle J. E., Fowler J. H. (2012). A 61-million-person experiment in social influence and political mobilization. Nature, 489, 295–298. https://doi.org/10.1038/nature11421

Bosco F. A., Aguinis H., Singh K., Field J. G., Pierce C. A. (2015). Correlational effect size benchmarks. Journal of Applied Psychology, 100, 431–449. https://doi.org/10.1037/a0038047

Bouchard T. J. (2004). Genetic influence on human psychological traits. Current Directions in Psychological Science, 13, 148–151. https://doi.org/10.1111/j.0963-7214.2004.00295.x

Boyle E. A., Li Y. I., Pritchard J. K. (2017). An expanded view of complex traits: From polygenic to omnigenic. Cell, 169, 1177–1186. https://doi.org/10.1016/j.cell.2017.05.038

Buss D. M. (2009). How can evolutionary psychology successfully explain personality and individual differences? Perspectives on Psychological Science, 4, 359–366. https://doi.org/10.1111/j.1745-6924.2009.01138.x

Button K. S., Ioannidis J. P. A., Mokrysz C., Nosek B. A., Flint J., Robinson E. S. J., Munafò M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365–376. https://doi.org/10.1038/nrn3475

Button K. S., Kounali D., Thomas L., Wiles N. J., Peters T. J., Welton N. J., Ades A. E., Lewis G. (2015). Minimal clinically important difference on the Beck Depression Inventory–II according to the patient’s perspective. Psychological Medicine, 45, 3269–3279. https://doi.org/10.1017/S0033291715001270

Bycroft C., Freeman C., Petkova D., Band G., Elliott L. T., Sharp K., Motyer A., Vukcevic D., Delaneau O., O’Connell J., Cortes A., Welsh S., Young A., Effingham M., McVean G., Leslie S., Allen N., Donnelly P., Marchini J. (2018). The UK Biobank resource with deep phenotyping and genomic data. Nature, 562, 203–209. https://doi.org/10.1038/s41586-018-0579-z

Camerer C. F., Dreber A., Forsell E., Ho T.H., Huber J., Johannesson M., Kirchler M., Almenberg J., Altmejd A., Chan T., Heikensten E., Holzmeister F., Imai T., Isaksson S., Nave G., Pfeiffer T., Razen M., Wu H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351, 1433–1436. https://doi.org/10.1126/science.aaf0918

Camerer C. F., Dreber A., Holzmeister F., Ho T.H., Huber J., Johannesson M., Kirchler M., Nave G., Nosek B. A., Pfeiffer T., Altmejd A., Buttrick N., Chan T., Chen Y., Forsell E., Gampa A., Heikensten E., Hummer L., Imai T. Wu H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2, 637–644. https://doi.org/10.1038/s41562-018-0399-z

Chabris C. F., Lee J. J., Cesarini D., Benjamin D. J., Laibson D. I. (2015). The fourth law of behavior genetics. Current Directions in Psychological Science, 24, 304–312. https://doi.org/10.1177/0963721415580430

Chambers C. (2019). What’s next for registered reports? Nature, 573, 187–189. https://doi.org/10.1038/d41586-019-02674-6

Chartier C. R., Riegelman A., McCarthy R. J. (2018). StudySwap: A platform for interlab replication, collaboration, and resource exchange. Advances in Methods and Practices in Psychological Science, 1, 574–579. https://doi.org/10.1177/2515245918808767

Chen E. E., Wojcik S. P. (2016). A practical guide to big data research in psychology. Psychological Methods, 21, 458–474. https://doi.org/10.1037/met0000111

Church T. A. (2010). Current perspectives in the study of personality across cultures. Perspectives on Psychological Science, 5, 441–449. https://doi.org/10.1177/1745691610375559

Cohen J., 1988, Statistical power analysis for the behavioral sciences, 2

Cook P. J., Dodge K., Farkas G., Fryer R. G., Guryan J., Ludwig J., Mayer S. (2015). Not too late: Improving academic outcomes for disadvantaged youth (Working paper WP-15-01). Institute for Policy Research Northwestern University.

De Boeck P., Jeon M. (2018). Perceived crisis and reforms: Issues, explanations, and remedies. Psychological Bulletin, 144, 757–777. https://doi.org/10.1037/bul0000154

Donnelly P. (2008). Progress and challenges in genome-wide association studies in humans. Nature, 456, 728–731. https://doi.org/10.1038/nature07631

Duncan G. J., Magnuson K. (2007). Penny wise and effect size foolish. Child Development Perspectives, 1, 46–51. https://doi.org/10.1111/j.1750-8606.2007.00009.x

Eisenberg N., Duckworth A. L., Spinrad T. L., Valiente C. (2014). Conscientiousness: Origins in childhood? Developmental Psychology, 50, 1331–1349. https://doi.org/10.1037/a0030977

Fanelli D., Costas R., Ioannidis J. P. A. (2017). Meta-assessment of bias in science. Proceedings of the National Academy of Sciences, USA, 114, 3714–3719. https://doi.org/10.1073/pnas.1618569114

Fischer R., Lee A., Verzijden M. N. (2018). Dopamine genes are linked to extraversion and neuroticism personality traits, but only in demanding climates. Scientific Reports, 8, Article 1733. https://doi.org/10.1038/s41598-017-18784-y

Frisvold D. E. (2015). Nutrition and cognitive achievement: An evaluation of the School Breakfast Program. Journal of Public Economics, 124, 91–104. https://doi.org/10.1016/j.jpubeco.2014.12.003

Funder D. C., Levine J. M., Mackie D. M., Morf C. C., Sansone C., Vazire S., West S. G. (2014). Improving the dependability of research in personality and social psychology. Personality and Social Psychology Review, 18, 3–12. https://doi.org/10.1177/1088868313507536

Funder D. C., Ozer D. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2, 156–168. https://doi.org/10.1177/2515245919847202

Furnham A., Cheng H. (2018). Early predictors of trait extraversion in adulthood: Findings from a nationally representative sample. Personality and Individual Differences, 135, 242–247. https://doi.org/10.1016/j.paid.2018.07.026

Garretsen H., Stoker J. I., Soudis D., Martin R. L., Rentfrow P. J. (2018). Brexit and the relevance of regional personality traits: More psychological openness could have swung the regional vote. Cambridge Journal of Regions, Economy and Society, 11, 165–175. https://doi.org/10.1093/cjres/rsx031

Gelman A., Carlin J. (2014). Beyond power calculations: Assessing type s (sign) and type m (magnitude) errors. Perspectives on Psychological Science, 9, 641–651. https://doi.org/10.1177/1745691614551642

Gignac G. E., Szodorai E. T. (2016). Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102, 74–78. https://doi.org/10.1016/j.paid.2016.06.069

Gladstone J. J., Matz S. C., Lemaire A. (2019). Can psychological traits be inferred from spending? Evidence from transaction data. Psychological Science, 30, 1087–1096. https://doi.org/10.1177/0956797619849435

Götz F. M., Gvirtz A., Galinsky A. D., Jachimowicz J. M. (2021). How personality and policy predict pandemic behavior: Understanding sheltering-in-place in 55 countries at the onset of COVID-19. American Psychologist, 76(1), 39–49. https://doi.org/10.1037/amp0000740

Götz F. M., Stieger S., Gosling S. D., Potter J., Rentfrow P. J. (2020). Physical topography is associated with human personality. Nature Human Behaviour, 4, 1135–1144. https://doi.org/10.1038/s41562-020-0930-x

Götz F. M., Yoshino S., Oshio A. (2020). The association between walkability and personality: Evidence from a large socioecological study in Japan. Journal of Environmental Psychology, 69, Article 101438. https://doi.org/10.1016/j.jenvp.2020.101438

Grand J. A., Rogelberg S. G., Banks G. C., Landis R. S., Tonidandel S. (2018). From outcome to process focus: Fostering a more robust psychological science through registered reports and results-blind reviewing. Perspectives on Psychological Science, 13, 448–456. https://doi.org/10.1177/1745691618767883

Greenwald A. G., Banaji M. R., Nosek B. A. (2015). Statistically small effects of the Implicit Association Test can have societally large effects. Journal of Personality and Social Psychology, 108, 553–561. https://doi.org/10.1037/pspa0000016

Harari G. M., Lane N. D., Wang R., Crosier B. S., Campbell A. T., Gosling S. D. (2016). Using smartphones to collect behavioral data in psychological science. Perspectives on Psychological Science, 11, 838–854. https://doi.org/10.1177/1745691616650285

Hardwicke T. E., Ioannidis J. P. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2, 793–796. https://doi.org/10.1038/s41562-018-0444-y

Harris D. N. (2009). Toward policy-relevant benchmarks for interpreting effect sizes: Combining effects with costs. Educational Evaluation and Policy Analysis, 31, 3–29. https://doi.org/10.3102/0162373708327524

Hill C. J., Bloom H. S., Black A. R., Lipsey M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2, 172–177. https://doi.org/10.1111/j.1750-8606.2008.00061.x

Hindorff L. A., Sethupathy P., Junkins H. A., Ramos E. M., Mehta J. P., Collins F. S., Manolio T. A. (2009). Potential etiologic and functional implications of genome-wide association loci for human diseases and traits. Proceedings of the National Academy of Sciences, USA, 106, 9362–9367. https://doi.org/10.1073/pnas.0903103106

Hoffman L. W. (1991). The influence of the family environment on personality: Accounting for sibling differences. Psychological Bulletin, 110, 187–203. https://doi.org/10.1037/0033-2909.110.2.187

Hufer A., Kornadt A. E., Kandler C., Riemann R. (2020). Genetic and environmental variation in political orientation in adolescence and early adulthood: A nuclear twin family analysis. Journal of Personality and Social Psychology, 118, 762–776. https://doi.org/10.1037/pspp0000258

Hutteman R., Nestler S., Wagner J., Egloff B., Back M. D. (2015). Wherever I may roam: Processes of self-esteem development from adolescence to emerging adulthood in the context of international student exchange. Journal of Personality and Social Psychology, 108, 767–783. https://doi.org/10.1037/pspp0000015

Jokela M. (2020). Selective residential mobility and social influence in the emergence of neighborhood personality differences: Longitudinal data from Australia. Journal of Research in Personality, 86, Article 103953. https://doi.org/10.1016/j.jrp.2020.103953

Jokela M., Bleidorn W., Lamb M. E., Gosling S. D., Rentfrow P. J. (2015). Geographically varying associations between personality and life satisfaction in the London metropolitan area. Proceedings of the National Academy of Sciences, USA, 112, 725–730. https://doi.org/10.1073/pnas.1415800112

Kazdin A. E. (1999). The meanings and measurement of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332–339. https://doi.org/10.1037/0022-006x.67.3.332

Kerr N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196–217. https://doi.org/10.1207/s15327957pspr0203_4

Kitayama S., Conway L. G., Pietromonaco P. R., Park H., Plaut V. C. (2010). Ethos of independence across regions in the United States: The production-adoption model of cultural change. American Psychologist, 65, 559–574. https://doi.org/10.1037/a0020277

Klein R. A., Vianello M., Hasselman F., Adams B. G., Adams R. B.Jr., Alper S., Aveyard M., Axt J. R., Babalola M. T., Bahník S., Batra R., Berkics M., Bernstein M. J., Berry D. R., Bialobrzeska O., Binan E. D., Bocian K., Brandt M. J., Busching R. Nosek B. A. (2018). Many Labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1, 443–490. https://doi.org/10.1177/2515245918810225

Kosinski M., Matz S. C., Gosling S. D., Popov V., Stillwell D. (2015). Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist, 70, 543–556. https://doi.org/10.1037/a0039210

Kraft M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49, 241–253. https://doi.org/10.3102/0013189x20912798

Kramer A. D. I., Guillory J. E., Hancock J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, USA, 111, 8788–8790. https://doi.org/10.1073/pnas.1320040111

Krapohl E., Rimfeld K., Shakeshaft N. G., Trzaskowski M., McMillan A., Pingault J.B., Asbury K., Harlaar N., Kovas Y., Dale P. S., Plomin R. (2014). The high heritability of educational achievement reflects many genetically influenced traits, not just intelligence. Proceedings of the National Academy of Sciences, USA, 111, 15273–15278. https://doi.org/10.1073/pnas.1408777111

Krauss S., Orth U., Robins R. W. (2020). Family environment and self-esteem development: A longitudinal study from age 10 to 16. Journal of Personality and Social Psychology, 119, 457–478. https://doi.org/10.1037/pspp0000263

Lakens D. (2019). The value of preregistration for psychological science: A conceptual analysis. Japanese Psychology Review, 62, 272–280. https://doi.org/10.31234/osf.io/jbh4w

Lee J. J., Wedow R., Okbay A., Kong E., Maghzian O., Zacher M., Nguyen-Viet T. A., Bowers P., Sidorenko J., Karlsson Linnér R., Fontana M. A., Kundu T., Lee C., Li H., Li R., Royer R., Timshel P. N., Walters R. K., Willoughby E. A. Cesarini D. (2018). Gene discovery and polygenic prediction from a genome-wide association study of educational attainment in 1.1 million individuals. Nature Genetics, 50, 1112–1121. https://doi.org/10.1038/s41588-018-0147-3

Levin H. M., Belfield C. (2015). Guiding the development and use of cost-effectiveness analysis in education. Journal of Research on Educational Effectiveness, 8, 400–418. https://doi.org/10.1080/19345747.2014.915604

Lindsay D. S. (2020). Seven steps toward transparency and replicability in psychological science. Canadian Psychology/Psychologie canadienne, 61, 310–317. https://doi.org/10.1037/cap0000222

Liu M., Jiang Y., Wedow R., Li Y., Brazel D. M., Chen F., Datta G., Davila-Velderrain J., McGuire D., Tian C., Zhan X., Choquet H., Docherty A. R., Faul J. D., Foerster J. R., Fritsche L. G., Gabrielsen M. E., Gordon S. D., Haessler J. Tian C. (2019). Association studies of up to 1.2 million individuals yield new insights into the genetic etiology of tobacco and alcohol use. Nature Genetics, 51, 237–244. https://doi.org/10.1038/s41588-018-0307-5

Lu J. G., Quoidbach J., Gino F., Chakroff A., Maddux W. W., Galinsky A. D. (2017). The dark side of going abroad: How broad foreign experiences increase immoral behavior. Journal of Personality and Social Psychology, 112, 1–16. https://doi.org/10.1037/pspa0000068

Ludwig J., Duncan G. J., Gennetian L. A., Katz L. F., Kessler R. C., Kling J. R., Sanbonmatsu L. (2012). Neighborhood effects on the long-term well-being of low-income adults. Science, 337, 1505–1510. https://doi.org/10.1126/science.1224648

Luhmann M., Hofmann W., Eid M., Lucas R. E. (2012). Subjective well-being and adaptation to life events: A meta-analysis. Journal of Personality and Social Psychology, 102, 592–615. https://doi.org/10.1037/a0025948

Mackay T. F. C., Stone E. A., Ayroles J. F. (2009). The genetics of quantitative traits: Challenges and prospects. Nature Reviews Genetics, 10, 565–577. https://doi.org/10.1038/nrg2612

Matz S. C., Gladstone J. J., Stillwell D. (2016). Money buys happiness when spending fits our personality. Psychological Science, 27, 715–725. https://doi.org/10.1177/0956797616635200

Matz S. C., Gladstone J. J., Stillwell D. (2017). In a world of big data, small effects can still matter: A reply to Boyce, Daly, Hounkpatin, and Wood (2017). Psychological Science, 28, 547–550. https://doi.org/10.1177/0956797617697445

Matz S. C., Kosinski M., Nave G., Stillwell D. J. (2017). Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences, USA, 114, 12714–12719. https://doi.org/10.1073/pnas.1710966114

McCrae R. R., Costa P. T., Ostendorf F., Angleitner A., Hřebíčková M., Avia M. D., Sanz J., Sánchez-Bernardos M. L., Kusdil M. E., Woodfield R., Saunders P. R., Smith P. B. (2000). Nature over nurture: Temperament, personality, and life span development. Journal of Personality and Social Psychology, 78, 173–186. https://doi.org/10.1037/0022-3514.78.1.173

Meyer G. J., Finn S. E., Eyde L. D., Kay G. G., Moreland K. L., Dies R. R., Eisman E. J., Kubiszyn T. W., Reed G. M. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128–165. https://doi.org/10.1037/0003-066x.56.2.128

Miller D. I. (2019). When do growth mindset interventions work? Trends in Cognitive Sciences, 23, 910–912. https://doi.org/10.1016/j.tics.2019.08.005

Moshontz H., Campbell L., Ebersole C. R., IJzerman H., Urry H. L., Forscher P. S., Grahe J. E., McCarth R. J., Musser E. D., Atfolk J., Castille C. M., Evans R. R., Fiedler S., Flake J. K., Forero D. A., Janssen S. M. J., Keene J. R., Protzko J., Aczel B., Chartier C. R. (2018). The psychological science accelerator: Advancing psychology through a distributed collaborative network. Advances in Methods and Practices in Psychological Science, 1, 501–515. https://doi.org/10.1177/2515245918797607

Mõttus R., Kandler C., Bleidorn W., Riemann R., McCrae R. R. (2017). Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 112, 474–490. https://doi.org/10.1037/pspp0000100

Munafò M. R., Nosek B. A., Bishop D. V. M., Button K. S., Chambers C. D., Percie du, Sert N., Simonsohn U., Wagenmakers E.J., Ware J. J., Ioannidis J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, Article 0021. https://doi.org/10.1038/s41562-016-0021

Murray S. L., Lamarche V., Seery M. D., Jung H. Y., Griffin D. W., Brinkman C. (2021). The social-safety system: Fortifying relationships in the face of the unforeseeable. Journal of Personality and Social Psychology, 120(1), 99–130. https://doi.org/10.1037/pspi0000245

Nelson L. D., Simmons J., Simonsohn U. (2018). Psychology’s renaissance. Annual Review of Psychology, 69, 511–534. https://doi.org/10.1146/annurev-psych-122216-011836

Noftle E. E., Robins R. W. (2007). Personality predictors of academic outcomes: Big five correlates of GPA and SAT scores. Journal of Personality and Social Psychology, 93, 116–130. https://doi.org/10.1037/0022-3514.93.1.116

Nosek B. A., Ebersole C. R., DeHaven A. C., Mellor D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, USA, 115, 2600–2606. https://doi.org/10.1073/pnas.1708274114

Nosek B. A., Lakens D. (2014). Registered reports: A method to increase the credibility of published results. Social Psychology, 45, 137–141. https://doi.org/10.1027/1864-9335/a000192

Nosek B. A., Spies J. R., Motyl M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7, 615–631. https://doi.org/10.1177/1745691612459058

Obschonka M., Stuetzer M., Rentfrow P. J., Shaw-Taylor L., Satchell M., Silbereisen R. K., Potter J., Gosling S. D. (2018). In the shadow of coal: How large-scale industries contributed to present-day regional differences in personality and well-being. Journal of Personality and Social Psychology, 115, 903–927. https://doi.org/10.1037/pspp0000175

Ofosu E. K., Chambers M. K., Chen J. M., Hehman E. (2019). Same-sex marriage legalization associated with reduced implicit and explicit antigay bias. Proceedings of the National Academy of Sciences, USA, 116, 8846–8851. https://doi.org/10.1073/pnas.1806000116

Okbay A., Baselmans B. M., De Neve J. E., Turley P., Nivard M. G., Fontana M. A., Meddens S. F. W., Linnér R. K., Rietveld C. A., Derringer J., Gratten J., Lee J. J., Liu J. Z., de Vlaming R., Ahulawalia T. S., Buchwald J., Cavadino A., Frazier-Wood A. C., Furlotte N. A. Gratten J. (2016). Genetic variants associated with subjective well-being, depressive symptoms, and neuroticism identified through genome-wide analyses. Nature Genetics, 48, 624–633. https://doi.org/10.1038/ng.3552

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, Article aac4716. https://doi.org/10.1126/science.aac4716

Orth U. (2018). The family environment in early childhood has a long-term effect on self-esteem: A longitudinal study from birth to age 27 years. Journal of Personality and Social Psychology, 114, 637–655. https://doi.org/10.1037/pspp0000143

Ozer D. J., Benet-Martínez V. (2006). Personality and the prediction of consequential outcomes. Annual Review of Psychology, 57, 401–421. https://doi.org/10.1146/annurev.psych.57.102904.190127

Paluck E. L. (2009). Reducing intergroup prejudice and conflict using the media: A field experiment in Rwanda. Journal of Personality and Social Psychology, 96, 574–587. https://doi.org/10.1037/a0011989

Plomin R. (1999). Genetics and general cognitive ability. Nature, 402, C25–C29. https://doi.org/10.1038/35011520

Plomin R., Owen M., McGuffin P. (1994). The genetic basis of complex human behaviors. Science, 264, 1733–1739. https://doi.org/10.1126/science.8209254

Polderman T. J. C., Benyamin B., de Leeuw C. A., Sullivan P. F., van Bochoven A., Visscher P. M., Posthuma D. (2015). Meta-analysis of the heritability of human traits based on fifty years of twin studies. Nature Genetics, 47, 702–709. https://doi.org/10.1038/ng.3285

Prentice D. A., Miller D. T. (1992). When small effects are impressive. Psychological Bulletin, 112, 160–164. https://doi.org/10.1037/0033-2909.112.1.160

Revelle W. (1995). Personality processes. Annual Review of Psychology, 46, 295–328. https://doi.org/10.1146/annurev.ps.46.020195.001455

Richard F. D., Bond C. F., Stokes-Zoota J. J. (2003). One hundred years of social psychology quantitatively described. Review of General Psychology, 7, 331–363. https://doi.org/10.1037/1089-2680.7.4.331

Roberts B. W., Kuncel N. R., Shiner R., Caspi A., Goldberg L. R. (2007). The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological Science, 2, 313–345. https://doi.org/10.1111/j.1745-6916.2007.00047.x

Robertson A. A., Grimes P. W., Rogers K. E. (2001). A short-run cost-benefit analysis of community-based interventions for juvenile offenders. Crime & Delinquency, 47, 265–284. https://doi.org/10.1177/0011128701047002006

Rosenthal R. (1990). How are we doing in soft psychology? American Psychologist, 45, 775–777. https://doi.org/10.1037/0003-066x.45.6.775

Rosenthal R., DiMatteo M. R. (2001). Meta-analysis: Recent developments in quantitative methods for literature reviews. Annual Review of Psychology, 52, 59–82. https://doi.org/10.1146/annurev.psych.52.1.59

Rosnow R. L., Rosenthal R. (2003). Effect sizes for experimenting psychologists. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 57, 221–237. https://doi.org/10.1037/h0087427

Rothbart M. K., Ahadi S. A., Evans D. E. (2000). Temperament and personality: Origins and outcomes. Journal of Personality and Social Psychology, 78, 122–135. https://doi.org/10.1037/0022-3514.78.1.122

Sawaoka T., Monin B. (2018). The paradox of viral outrage. Psychological Science, 29, 1665–1678. https://doi.org/10.1177/0956797618780658

Schäfer T., Schwarz M. A. (2019). The meaningfulness of effect sizes in psychological research: Differences between sub-disciplines and the impact of potential biases. Frontiers in Psychology, 10, Article 813. https://doi.org/10.3389/fpsyg.2019.00813

Schooler J. (2011). Unpublished results hide the decline effect. Nature, 470, Article 437. https://doi.org/10.1038/470437a

Shrout P. E., Rodgers J. L. (2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69, 487–510. https://doi.org/10.1146/annurev-psych-122216-011845

Sisk V. F., Burgoyne A. P., Sun J., Butler J. L., Macnamara B. N. (2018). To what extent and under which circumstances are growth mind-sets important to academic achievement? Two meta-analyses. Psychological Science, 29, 549–571. https://doi.org/10.1177/0956797617739704

Smith-Woolley E., Selzam S., Plomin R. (2019). Polygenic score for educational attainment captures DNA variants shared between personality traits and educational achievement. Journal of Personality and Social Psychology, 117, 1145–1163. https://doi.org/10.1037/pspp0000241

Son H.Y., Hwangbo Y., Yoo S.K., Im S.W., Yang S. D., Kwak S.J., Park M. S., Kwak S. H., Cho S. W., Ryu J. S., Kim J., Jung Y.S., Kim T. H., Kim S.j., Lee K. E., Park D. J., Cho N. H., Sung J., Seo J.S. Kim J.I. (2017). Genome-wide association and expression quantitative trait loci studies identify multiple susceptibility loci for thyroid cancer. Nature Communications, 8, Article 15966. https://doi.org/10.1038/ncomms15966

Soto C. J. (2019). How replicable are links between personality traits and consequential life outcomes? The life outcomes of personality replication project. Psychological Science, 30, 711–727. https://doi.org/10.1177/0956797619831612

Specht J., Egloff B., Schmukle S. C. (2011). Stability and change of personality across the life course: The impact of age and major life events on mean-level and rank-order stability of the Big Five. Journal of Personality and Social Psychology, 101, 862–882. https://doi.org/10.1037/a002495

Spellman B. A. (2015). A short (personal) future history of revolution 2.0. Perspectives on Psychological Science, 10, 886–899. https://doi.org/10.1177/1745691615609918

Steering Committee of the Physician’s Health Study Research Group. (1988). Preliminary report: Findings from the aspirin component of the ongoing physicians’ health study. New England Journal of Medicine, 318, 262–264. https://doi.org/10.1056/nejm198801283180431

Sun B. B., Maranville J. C., Peters J. E., Stacey D., Staley J. R., Blackshaw J., Burgess S., Jiang T., Paige E., Surendran P., Oliver-Williams C., Kamat M. A., Prins B. P., Wilcox S. K., Zimmerman E. S., Chi A., Bansal N., Spain S. L., Wood A. M. Butterworth A. S. (2018). Genomic atlas of the human plasma proteome. Nature, 558, 73–79. https://doi.org/10.1038/s41586-018-0175-2

Sutin A. R., Luchetti M., Stephan Y., Robins R. W., Terracciano A. (2017). Parental educational attainment and adult offspring personality: An intergenerational life span approach to the origin of adult personality traits. Journal of Personality and Social Psychology, 113, 144–166. https://doi.org/10.1037/pspp0000137

Szucs D., Ioannidis J. P. A. (2017). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLOS Biology, 15, Article e2000797. https://doi.org/10.1371/journal.pbio.2000797

Tajfel H. (1970). Experiments in intergroup discrimination. Scientific American, 223, 96–102. https://doi.org/10.1038/scientificamerican1170-96

Talhelm T., Zhang X., Oishi S., Shimin C., Duan D., Lan X., Kitayama S. (2014). Large-scale psychological differences within China explained by rice versus wheat agriculture. Science, 344, 603–608. https://doi.org/10.1126/science.1246850

Tankard M. E., Paluck E. L. (2017). The effect of a supreme court decision regarding gay marriage on social norms and personal attitudes. Psychological Science, 28, 1334–1344. https://doi.org/10.1177/0956797617709594

Turkheimer E., Pettersson E., Horn E. E. (2014). A phenotypic null hypothesis for the genetics of personality. Annual Review of Psychology, 65, 515–540. https://doi.org/10.1146/annurev-psych-113011-143752

Uhlmann E. L., Ebersole C., Chartier C., Errington T., Kidwell M., Lai C. K., McCarthy R., Riegelman A., Silberzahn R., Nosek B. A. (2019). Scientific utopia III: Crowdsourcing science. Perspectives on Psychological Science, 14, 711–733. https://doi.org/10.1177/1745691619850561

Uskul A. K., Kitayama S., Nisbett R. E. (2008). Ecocultural basis of cognition: Farmers and fishermen are more holistic than herders. Proceedings of the National Academy of Sciences, USA, 105, 8552–8556. https://doi.org/10.1073/pnas.0803874105

Van de Vliert E., Van Lange P. A. M. (2019). Latitudinal psychology: An ecological perspective on creativity, aggression, happiness, and beyond. Perspectives on Psychological Science, 14, 860–884. https://doi.org/10.1177/1745691619858067

Visscher P. M., Wray N. R., Zhang Q., Sklar P., McCarthy M. I., Brown M. A., Yang J. (2017). 10 years of GWAS discovery: Biology, function, and translation. The American Journal of Human Genetics, 101, 5–22. https://doi.org/10.1016/j.ajhg.2017.06.005

Wagenmakers E.J., Wetzels R., Borsboom D., van der Maas H. L. J., Kievit R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632–638. https://doi.org/10.1177/1745691612463078

Weedon M. N., Lango H., Lindgren C. M., Wallace C., Evans D. M., Mangino M., Freathy R. M., Perry J. R. B., Stevens S., Hall A. S., Samani N. J., Shields B., Prokopenko I., Farrall M., Dominiczak A., Johnson T., Bergmann S., Beckmann J. S., Vollenweider P. Frayling T. M. (2008). Genome-wide association analysis identifies 20 loci that influence adult height. Nature Genetics, 40, 575–583. https://doi.org/10.1038/ng.121

Wei W., Lu J. G., Galinsky A. D., Wu H., Gosling S. D., Rentfrow P. J., Yuan W., Zhang Q., Guo Y., Zhang M., Gui W., Guo X.Y., Potter J., Wang J., Li B., Li X., Han Y.M., Lv M., Guo X.Q. Wang L. (2017). Regional ambient temperature is associated with human personality. Nature Human Behaviour, 1, 890–895. https://doi.org/10.1038/s41562-017-0240-0

Weston S. J., Gladstone J. J., Graham E. K., Mroczek D. K., Condon D. M. (2018). Who are the scrooges? Personality predictors of holiday spending. Social Psychological and Personality Science, 10, 775–782. https://doi.org/10.1177/1948550618792883

Wicherts J. M., Veldkamp C. L. S., Augusteijn H. E. M., Bakker M., van Aert R. C. M., van Assen M. A. L. M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, Article 1832. https://doi.org/10.3389/fpsyg.2016.01832

Yarkoni T., Westfall J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12, 1100–1122. https://doi.org/10.1177/1745691617693393

Yeager D. S., Hanselman P., Walton G. M., Murray J. S., Crosnoe R., Muller C., Tipton E., Schneider B., Hulleman C. S., Hinojosa C. P., Paunesku D., Romero C., Flint K., Roberts A., Trott J., Iachan R., Buontempo J., Yang S. M., Carvalho C. M. Dweck C. S. (2019). A national experiment reveals where a growth mindset improves achievement. Nature, 573, 364–369. https://doi.org/10.1038/s41586-019-1466-y