Evaluation of Pressing Issues in Ecological Momentary Assessment
Tóm tắt
The use of repeated, momentary, real-world assessment methods known as the Experience Sampling Method and Ecological Momentary Assessment (EMA) has been broadly embraced over the last few decades. These methods have extended our assessment reach beyond lengthy retrospective self-reports as they can capture everyday experiences in their immediate context, including affect, behavior, symptoms, and cognitions. In this review we evaluate nine conceptual, methodological, and psychometric issues about EMA with the goal of stimulating conversation and guiding future research on these matters: the extent to which participants are actually reporting momentary experiences, respondents’ interpretation of momentary questions, the use of comparison standards in responding, efforts to increase the EMA reporting period beyond the moment to longer periods within a day, training of EMA study participants, concerns about selection bias of respondents, the impact of missing EMA assessments, the reliability of momentary data, and for which purposes EMA might be considered a gold standard for assessment. Resolution of these issues should have far-reaching implications for advancing the field.
Từ khóa
Tài liệu tham khảo
2004, Reasons for missing interviews in the daily electronic assessment of pain, mood, and stress, Pain, 109, 389, 10.1016/j.pain.2004.02.014
1998, Fibromyalgia and women's pursuit of personal goals: a daily process analysis, Health Psychol, 17, 40, 10.1037/0278-6133.17.1.40
1998, Combining experiences over time: the effects of duration, intensity changes, and on-line measurements on retrospective pain evaluations, J. Behav. Decis. Mak., 11, 19, 10.1002/(SICI)1099-0771(199803)11:1<19::AID-BDM277>3.0.CO;2-B
2021, Routinely randomize potential sources of measurement reactivity to estimate and adjust for biases in subjective reports, Psychol. Methods, 26, 175, 10.1037/met0000294
2019, Recruitment via social media: advantages and potential biases, Digit. Health, 5, 2055207619867223, 10.1177/2055207619867223
2001, How can I deal with missing data in my study?, Aust. N.Z. J. Public Health, 25, 464, 10.1111/j.1467-842X.2001.tb00294.x
2018, Conversion of standard retrospective patient-reported outcomes to momentary versions: cognitive interviewing reveals varying degrees of momentary compatibility, Qual. Life Res., 27, 1065, 10.1007/s11136-017-1762-7
2003, Diary methods: capturing life as it is lived, Annu. Rev. Psychol., 54, 579, 10.1146/annurev.psych.54.101601.145030
2013, Intensive Longitudinal Methods: An Introduction to Diary and Experience Sampling Research
2008, The accuracy of pain and fatigue items across different reporting periods, Pain, 139, 146, 10.1016/j.pain.2008.03.024
2006, Recalled pain ratings: a complex and poorly defined task, J. Pain, 7, 142, 10.1016/j.jpain.2005.09.012
1941, Systematic and Representative Design of Psychological Experiments
2018, The reliability of end of day and ecological momentary assessments of pain and pain interference in individuals with spinal cord injury, Qual. Life Res., 27, 3003, 10.1007/s11136-018-1952-y
2020, Personality as a predictor of unit nonresponse in an Internet panel, Sociol. Methods Res., 49, 672, 10.1177/0049124117747305
2014, Validation of a brief yesterday measure of hedonic well-being and daily activities: comparison with the day reconstruction method, Soc. Indic. Res., 115, 907, 10.1007/s11205-013-0240-z
2008, Milnacipran for the treatment of fibromyalgia in adults: a 15-week, multicenter, randomized, double-blind, placebo-controlled, multiple-dose clinical trial, Clin. Ther., 30, 1988, 10.1016/j.clinthera.2008.11.009
2001, A comparison of inclusive and restrictive strategies in modern missing data procedures, Psychol. Methods, 6, 330, 10.1037/1082-989X.6.4.330
2012, Trends in ambulatory self-report: the role of momentary experience in psychosomatic medicine, Psychosom. Med., 74, 327, 10.1097/PSY.0b013e3182546f18
2006, A procedure for evaluating sensitivity to within-person change: Can mood measures in diary studies detect change reliably?, Personal. Soc. Psychol. Bull., 32, 917, 10.1177/0146167206287721
2003, Happiness in everyday life: the uses of experience sampling, J. Happiness Stud., 4, 185, 10.1023/A:1024409732742
2019, Latent trait shared-parameter mixed models for missing ecological momentary assessment data, Stat. Med., 38, 660, 10.1002/sim.7989
2020, Content validity and methodological considerations in ecological momentary assessment studies on physical activity and sedentary behaviour: a systematic review, Int. J. Behav. Nutr. Phys. Act., 17, 35, 10.1186/s12966-020-00932-9
1987, Investigating mental disorders in their natural settings: introduction to the special issue, J. Nerv. Ment. Dis., 175, 509, 10.1097/00005053-198709000-00001
2018, Reliabilities of intraindividual variability indicators with autocorrelated longitudinal data: implications for longitudinal study designs, Multivar. Behav. Res., 53, 502, 10.1080/00273171.2018.1457939
2018, Response patterns and intra-dyadic factors related to compliance with ecological momentary assessment among mothers and children, Transl. Behav. Med., 8, 233, 10.1093/tbm/ibx002
2009, Ecological momentary assessment of mood disorders and mood dysregulation, Psychol. Assess., 21, 463, 10.1037/a0017075
2013, Trait anxiety and trait anger measured by ecological momentary assessment and their correspondence with traditional trait questionnaires, J. Res. Personal., 47, 843, 10.1016/j.jrp.2013.08.005
1999, Intraindividual variability in affect: reliability, validity, and personality correlates, J. Personal. Soc. Psychol., 76, 662, 10.1037/0022-3514.76.4.662
2021a, Affective structure, measurement invariance, and reliability across different experience sampling protocols, J. Res. Personal, 92, 104094, 10.1016/j.jrp.2021.104094
2022, The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population, Assessment, 29, 136, 10.1177/1073191120957102
2021b, Reported affect changes as a function of response delay: findings from a pooled dataset of nine experience sampling studies, Front. Psychol., 12, 580684, 10.3389/fpsyg.2021.580684
2011, Missing not at random models for latent growth curve analyses, Psychol. Methods, 16, 1, 10.1037/a0022640
2020, A model-based imputation procedure for multilevel regression models with random coefficients, interaction effects, and nonlinear terms, Psychol. Methods, 25, 88, 10.1037/met0000228
2012, A Monte Carlo simulation study of the reliability of intraindividual variability, Psychol. Aging, 27, 560, 10.1037/a0026669
2018, Lack of group-to-individual generalizability is a threat to human subjects research, PNAS, 115, E6106
2012, Using experience sampling methodology in organizational behavior, J. Organ. Behav., 33, 865, 10.1002/job.1803
2014, Reliability estimation in a multilevel confirmatory factor analysis framework, Psychol. Methods, 19, 72, 10.1037/a0032138
2014, The effectiveness of financial incentives for health behaviour change: systematic review and meta-analysis, PLOS ONE, 9, e90347, 10.1371/journal.pone.0090347
2001, Recall biases and cognitive errors in retrospective self-reports: a call for momentary assessments, Handbook of Health Psychology, 405
2018, Multiple imputation of missing data for multilevel models: simulations and recommendations, Organ. Res. Methods, 21, 111, 10.1177/1094428117703686
2017, No time like the present: discovering the hidden dynamics in intensive longitudinal data, Curr. Dir. Psychol. Sci., 26, 10, 10.1177/0963721416666518
2017, Using mobile-technology-based ecological momentary assessment (EMA) methods with youth: a systematic review and recommendations, J. Pediatr. Psychol., 42, 1087, 10.1093/jpepsy/jsx078
2019, A comparison of signal- and event-contingent ambulatory assessment of interpersonal behavior and affect in social situations, Psychol. Assess., 31, 952, 10.1037/pas0000718
2016, Test reliability at the individual level, Struct. Equ. Model., 23, 532, 10.1080/10705511.2016.1148605
1997, Randomly sampling thinking in the natural environment, J. Consult. Clin. Psychol., 65, 941, 10.1037/0022-006X.65.6.941
2006, The Descriptive Experience Sampling method, Phenomenol. Cogn. Sci., 5, 271
2017, Response: Commentary: Can inner experience be apprehended in high fidelity? Examining brain activation and experience from multiple perspectives, Front. Psychol., 8, 628, 10.3389/fpsyg.2017.00628
2015, Investigating pristine inner experience: implications for experience sampling and questionnaires, Conscious. Cogn., 31, 148, 10.1016/j.concog.2014.11.002
2022, Identification of careless responding in ecological momentary assessment research: from posthoc analyses to real-time data monitoring, Psychol. Methods, 27, 958
2008, Validity of 24-h recall ratings of pain severity: biasing effects of “Peak” and “End” pain, Pain, 137, 422, 10.1016/j.pain.2007.10.006
2019, Compliance with ecological momentary assessment protocols in substance users: a meta-analysis, Addiction, 114, 609, 10.1111/add.14503
2018, Frames of reference in self-reports of health, well-being, fatigue, and pain: a qualitative examination, Appl. Res. Qual. Life, 13, 585, 10.1007/s11482-017-9546-3
2011, Thinking, Fast and Slow
2004, A survey method for characterizing daily life experience: the Day Reconstruction Method, Science, 306, 1776, 10.1126/science.1103572
1997, Back to Bentham? Explorations of experienced utility, Q. J. Econ., 112, 375, 10.1162/003355397555235
2019, Willingness to participate in passive mobile data collection, Public Opin. Q., 83, 210
2006, Reliability of recalled self-report on headache intensity: investigation using ecological momentary assessment technique, Cephalalgia, 26, 1335, 10.1111/j.1468-2982.2006.01221.x
2021, Making the black box transparent: a template and tutorial for registration of studies using experience-sampling methods, Adv. Methods Pract. Psychol. Sci., 4
2011, Assessing research participants’ perceptions of their clinical research experiences, Clin. Transl. Sci., 4, 403, 10.1111/j.1752-8062.2011.00349.x
1991, Response strategies for coping with the cognitive demands of attitude measures in surveys, Appl. Cogn. Psychol., 5, 213, 10.1002/acp.2350050305
2021, Composite reliability of multilevel data: It's about observed scores and construct meanings, Psychol. Methods, 26, 90, 10.1037/met0000287
2017, The consequences of ignoring item parameter drift in longitudinal item response models, Appl. Meas. Educ., 30, 129, 10.1080/08957347.2017.1283317
2020, Micro-randomized trials for promoting engagement in mobile health data collection: adolescent/young adult oral chemotherapy adherence as an example, Curr. Opin. Syst. Biol., 21, 1, 10.1016/j.coisb.2020.07.002
2016, A systematic review of methods and procedures used in ecological momentary assessments of diet and physical activity research in youth: an adapted STROBE checklist for reporting EMA studies (CREMAS), J. Med. Internet Res., 18, e151, 10.2196/jmir.4954
2018, A shared parameter location scale mixed effect model for EMA data subject to informative missing, Health Serv. Outcomes Res. Methodol., 18, 227, 10.1007/s10742-018-0184-5
2017, Testing measurement invariance in longitudinal data with ordered-categorical measures, Psychol. Methods, 22, 486, 10.1037/met0000075
2018, Ecological momentary assessment methodology in chronic pain research: a systematic review, J. Pain, 19, 699, 10.1016/j.jpain.2018.01.006
2017, Explaining system missing: missing data and experience sampling method, Soc. Psychol. Personal. Sci., 8, 434, 10.1177/1948550617708015
2011, Handbook of Research Methods for Studying Daily Life
1982, Contingency table models for assessing item bias, J. Educ. Stat., 7, 105, 10.3102/10769986007002105
2011, Factors associated with missing data in an experience sampling investigation of substance use determinants, Drug Alcohol Depend, 114, 153
2009, Memories of yesterday's emotions: Does the valence of experience affect the memory-experience gap?, Emotion, 9, 885, 10.1037/a0017823
2009, Compliance with momentary pain measurement using electronic diaries: a systematic review, Eur. J. Pain, 13, 354, 10.1016/j.ejpain.2008.05.010
2022, Validation of a brief measure of aggression for ecological momentary assessment research: the Aggression-ES-A, Assessment, 29, 296, 10.1177/1073191120976851
1997, Latent variable modeling of longitudinal and multilevel data, Sociol. Methodol., 27, 453, 10.1111/1467-9531.271034
2009, Experience sampling research in psychopathology: opening the black box of daily life, Psychol. Med., 39, 1533, 10.1017/S0033291708004947
2020, How was your day? Convergence of aggregated momentary and retrospective end-of-day affect ratings across the adult life span, J. Personal. Soc. Psychol., 119, 185, 10.1037/pspp0000248
2011, Multilevel Modeling for Social and Personality Psychology
1986, The category effect with rating scales: number of categories, number of stimuli, and method of presentation, J. Exp. Psychol. Hum. Percept. Perform., 12, 496, 10.1037/0096-1523.12.4.496
2019, What predicts within-person variance in applied psychology constructs? An empirical examination, J. Appl. Psychol., 104, 727, 10.1037/apl0000374
2016, Measurement invariance conventions and reporting: the state of the art and future directions for psychological research, Dev. Rev., 41, 71, 10.1016/j.dr.2016.06.004
2009, Time-structured and net intraindividual variability: tools for examining the development of dynamic characteristics and processes, Psychol. Aging, 24, 778, 10.1037/a0017915
1996, Patients' memories of pain medical treatments: real-time and retrospective evaluations of two minimally invasive procedures, Pain, 66, 3, 10.1016/0304-3959(96)02994-6
2000, Event-sampling and other methods for studying everyday experience, Handbook of Research Methods in Social and Personality Psychology, 190
2020, Momentary predictors of compliance in studies using the experience sampling method, Psychiatry Res, 286, 112896, 10.1016/j.psychres.2020.112896
2007, Systematic review identifies number of strategies important for retaining study participants, J. Clin. Epidemiol., 60, 757
2002, Belief and feeling: evidence for an accessibility model of emotional self-report, Psychol. Bull., 128, 934, 10.1037/0033-2909.128.6.934
1989, Relation of implicit theories to the construction of personal histories, Psychol. Rev., 96, 341, 10.1037/0033-295X.96.2.341
2002, Missing data: our view of the state of the art, Psychol. Methods, 7, 147, 10.1037/1082-989X.7.2.147
2018, Careless responding in Internet-based quality of life assessments, Qual. Life Res., 27, 1077, 10.1007/s11136-017-1767-2
2016, Ambulatory and diary methods can facilitate the measurement of patient-reported outcomes, Qual. Life Res., 25, 497, 10.1007/s11136-015-1054-z
2011, Peak and end effects in patients' daily recall of pain and fatigue: a within-subjects analysis, J. Pain, 12, 228, 10.1016/j.jpain.2010.07.001
1981, Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context
2019, Measurement error and person-specific reliability in multilevel autoregressive modeling, Psychol. Methods, 24, 70, 10.1037/met0000188
2013, Compliance with an EMA monitoring protocol and its relationship with participant and smoking characteristics, Nicotine Tob. Res., 16, S88
2007, Analysis of real-time momentary data: a practical guide, The Science of Real-Time Data Capture, 76, 10.1093/oso/9780195178715.003.0005
1999, Self-reports: how the questions shape the answers, Am. Psychol., 54, 93, 10.1037/0003-066X.54.2.93
2007, Retrospective and concurrent self-reports: the rationale for real-time data capture, The Science of Real-Time Data Capture, 11, 10.1093/oso/9780195178715.003.0002
2001, Asking questions about behavior: cognition, communication, and questionnaire construction, Am. J. Eval., 22, 127, 10.1177/109821400102200202
2009, Experience sampling: promises and pitfalls, strengths and weaknesses, Social Indicators Research Series, Vol. 39: Assessing Well-Being: The Collected Works of Ed Diener, 157
2015, The Effects of Stress on Cognitive Aging, Physiology and Emotion (ESCAPE) project, BMC Psychiatry, 15, 146, 10.1186/s12888-015-0497-7
2017, Age differences in negative emotional responses to daily stressors depend on time since event, Dev. Psychol., 53, 177, 10.1037/dev0000257
2020, A coordinated analysis of variance in affect in daily life, Assessment, 27, 1683, 10.1177/1073191118799460
2014, Conceptualizing analyses of ecological momentary assessment data, Nicotine Tob. Res., 16, S76
2008, Ecological momentary assessment, Annu. Rev. Clin. Psychol., 4, 1, 10.1146/annurev.clinpsy.3.022806.091415
2013, Missed beeps and missing data: dispositional and situational predictors of nonresponse in experience sampling research, Soc. Sci. Comput. Rev., 31, 471, 10.1177/0894439313479902
2014, Planned missing-data designs in experience-sampling research: Monte Carlo simulations of efficient designs for assessing within-person constructs, Behav. Res. Methods, 46, 41, 10.3758/s13428-013-0353-y
2008, Measurement-burst designs for social health research, Soc. Personal. Psychol. Compass, 2, 245, 10.1111/j.1751-9004.2007.00043.x
2019, Moving beyond self-report data collection in the natural environment: a review of the past and future directions for ambulatory assessment in eating disorders, Int. J. Eat. Disord., 52, 1157, 10.1002/eat.23124
2016, Pain intensity rating training: results from an exploratory study of the ACTTION PROTECCT system, Pain, 157, 1056, 10.1097/j.pain.0000000000000502
2021, Influence of ecological momentary assessment study design features on reported willingness to participate and perceptions of potential research studies: an experimental study, BMJ Open, 11, e049154, 10.1136/bmjopen-2021-049154
2017, A slice of life: ecologically valid methods for research on social relationships and health across the lifespan, Soc. Personal. Psychol. Compass, 11, e12356, 10.1111/spc3.12356
1998, Stressors and mood measured on a momentary basis are associated with salivary cortisol secretion, Psychoneuroendocrinology, 23, 353, 10.1016/S0306-4530(98)00008-0
2003, Ecological Momentary Assessment research in behavioral medicine, J. Happiness Stud., 4, 35, 10.1023/A:1023657221954
2005, Power and sample size in multilevel linear models, Encyclopedia of Statistics in Behavioral Science, 3, 1570
2014, Factors predicting compliance to ecological momentary assessment among adolescent smokers, Nicotine Tob. Res., 16, 351, 10.1093/ntr/ntt154
2004, Understanding recall of weekly pain from a momentary assessment perspective: absolute agreement, between- and within-person consistency, and judged change in weekly pain, Pain, 107, 61, 10.1016/j.pain.2003.09.020
2007, Real-time data collection for pain: appraisal and current status, Pain Med, 8, S85, 10.1111/j.1526-4637.2007.00372.x
2021, High resolution, field approaches for assessing pain: Ecological Momentary Assessment, Pain, 162, 4, 10.1097/j.pain.0000000000002049
2001, Individual differences in the diurnal cycle of salivary free cortisol: a replication of flattened cycles for some individuals, Psychoneuroendocrinology, 26, 295, 10.1016/S0306-4530(00)00057-3
1994, Ecological Momentary Assessment (EMA) in behavioral medicine, Ann. Behav. Med., 16, 199, 10.1093/abm/16.3.199
2002, Capturing momentary, self-report data: a proposal for reporting guidelines, Ann. Behav. Med., 24, 236, 10.1207/S15324796ABM2403_09
2007, The Science of Real-Time Data Capture: Self-Reports in Health Research
1984, Cognitive sciences and survey methods, Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines, 73
2000, The Psychology of Survey Response
2020, Ambulatory assessment in psychopathology research: a review of recommended reporting guidelines and current practices, J. Abnorm. Psychol., 129, 56, 10.1037/abn0000473
2005, What is perfect health to an 85-year-old? Evidence for scale recalibration in subjective health ratings, Med. Care, 43, 1054, 10.1097/01.mlr.0000178193.38413.70
2020, Overcoming compliance bias in self-report studies: a cross-study analysis, Int. J. Hum. Comput. Stud., 134, 1, 10.1016/j.ijhcs.2019.10.003
2001, The occurrence of recall bias in pediatric headache: a comparison of questionnaire and diary data, Headache, 41, 11, 10.1046/j.1526-4610.2001.111006011.x
2021, Ecological momentary assessment of emotional awareness: preliminary evaluation of psychometric properties, Curr. Psychol., 40, 1402, 10.1007/s12144-018-0074-6
2002, Conversational norms and the interpretation of vague quantifiers, Appl. Cogn. Psychol., 16, 301, 10.1002/acp.787
2013, Pain ratings reflect cognitive context: a range frequency model of pain perception, Pain, 154, 743, 10.1016/j.pain.2013.01.016
2021, The effect of training on participant adherence with a reporting time frame for momentary subjective experiences in Ecological Momentary Assessment: cognitive interview study, JMIR Form. Res., 5, e28007, 10.2196/28007
2017, Compliance with mobile ecological momentary assessment protocols in children and adolescents: a systematic review and meta-analysis, J. Med. Internet Res., 19, e132, 10.2196/jmir.6641
2020, Indicators of affect dynamics: structure, reliability, and personality correlates, Eur. J. Personal., 34, 1060, 10.1002/per.2277
2005, Cognitive Interviewing: A Tool for Improving Questionnaire Design