Predicting panel attrition in longitudinal HRQoL surveys during the COVID-19 pandemic in the US

Health and Quality of Life Outcomes - Tập 20 Số 1 - 2022
Tianzhou Yu1, Jiafan Chen1, Nong Gu2, Joel W. Hay3, Chen Gong4
1Department of Pharmaceutical and Health Economics, School of Pharmacy, University of Southern California, Los Angeles, CA, USA
2School of Nursing and Health Professions, University of San Francisco, San Francisco, CA, USA
3Schaeffer Center for Health Policy and Economics, School of Pharmacy, University of Southern California, Los Angeles, CA, USA
4Fetal and Neonatal Institute, Division of Neonatology, Department of Pediatrics, Children's Hospital Los Angeles, Los Angeles, CA, USA

Tóm tắt

Abstract Background Online longitudinal surveys may be subject to potential biases due to sample attrition. This study was designed to identify potential predictors of attrition using a longitudinal panel survey collected during the COVID-19 pandemic. Methods Three waves of data were collected using Amazon Mechanical Turk (MTurk), an online crowd-sourced platform. For each wave, the study sample was collected by referencing a US national representative sample distribution of age, gender, and race, based on US census data. Variables included respondents’ demographics, medical history, socioeconomic status, COVID-19 experience, changes of health behavior, productivity, and health-related quality of life (HRQoL). Results were compared to pre-pandemic US norms. Measures that predicted attrition at different times of the pandemic were identified via logistic regression with stepwise selection. Results 1467 of 2734 wave 1 respondents participated in wave 2 and, 964 of 2454 wave 2 respondents participated in wave 3. Younger age group, Hispanic origin (p ≤ 0.001) and higher self-rated survey difficulty (p ≤ 0.002) consistently predicted attrition in the following wave. COVID-19 experience, employment, productivity, and limited physical activities were commonly observed variables correlated with attrition with specific measures varying by time periods. From wave 1, mental health conditions, average daily hours worked (p = 0.004), and COVID-19 impact on work productivity (p < 0.001) were associated with a higher attrition rate at wave 2, additional to the aforementioned factors. From wave 2, support of social distancing (p = 0.032), being Republican (p < 0.001), and having just enough money to make ends meet (p = 0.003) were associated with predicted attrition at wave 3. Conclusions Attrition in this longitudinal panel survey was not random. Besides commonly identified demographic factors that contribute to panel attrition, COVID-19 presented novel opportunities to address sample biases by correlating attrition with additional behavioral and HRQoL factors in a constantly evolving environment. While age, ethnicity, and survey difficulty consistently predicted attrition, other factors, such as COVID-19 experience, changes of employment, productivity, physical health, mental health, and financial situation impacted panel attrition during the pandemic at various degrees.

Từ khóa


Tài liệu tham khảo

Hay JW, Gong CL, Jiao X, et al. A US Population Health Survey on the Impact of COVID-19 Using the EQ-5D-5L. J Gen Intern Med. 2021;36(5):1292–301. https://doi.org/10.1007/s11606-021-06674-z.

Ali SH, Foreman J, Capasso A, Jones AM, Tozan Y, DiClemente RJ. Social media as a recruitment platform for a nationwide online survey of COVID-19 knowledge, beliefs, and practices in the United States: methodology and feasibility analysis. BMC Med Res Methodol. 2020;20(1):116. https://doi.org/10.1186/s12874-020-01011-0.

Dixon BE, Mukherjee S, Wiensch A, Gray ML, Ferres JML, Grannis SJ. Capturing COVID-19-like symptoms at scale using banner ads on an online news platform: pilot survey study. J Med Internet Res. 2021;23(5):e24742. https://doi.org/10.2196/24742.

De Man J, Campbell L, Tabana H, Wouters E. The pandemic of online research in times of COVID-19. BMJ Open. 2021;11(2):e043866. https://doi.org/10.1136/bmjopen-2020-043866.

Eysenbach G. Improving the quality of web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3):e34. https://doi.org/10.2196/jmir.6.3.e34.

Eysenbach G, Wyatt J. Using the internet for surveys and health research. J Med Internet Res. 2002;4(2):e13. https://doi.org/10.2196/jmir.4.2.e13.

Amazon Mechanical Turk. https://www.mturk.com/worker/help#what_is. Accessed 9 Aug 2021.

US Census Bureau. Household Income: HINC-01. The United States Census Bureau. https://www.census.gov/data/tables/time-series/demo/income-poverty/cps-hinc/hinc-01.html. Accessed 20 Aug 2021.

US Census Bureau. QuickFacts: United States. https://www.census.gov/quickfacts/fact/table/US/PST045219. Accessed 20 Aug 2021.

US Census Bureau. Age and Sex Composition in the United States: 2019. The United States Census Bureau. https://www.census.gov/data/tables/2019/demo/age-and-sex/2019-age-sex-composition.html. Accessed 20 Aug 2021.

US Census Bureau. Educational Attainment in the United States: 2019. The United States Census Bureau. https://www.census.gov/data/tables/2019/demo/educational-attainment/cps-detailed-tables.html. Accessed 20 Aug 2021.

US Census Bureau. U.S. and World Population Clock. https://www.census.gov/popclock/. Accessed 20 Aug 2021.

US Census Bureau. Marital Status, Table ID: S1201. https://data.census.gov/cedsci/table?q=marriage&hidePreview=false&tid=ACSST1Y2018.S1201&vintage=2018. Accessed 20 Aug 2021.

Smith G. Step away from stepwise. J Big Data. 2018;5(1):32. https://doi.org/10.1186/s40537-018-0143-6.

Czeisler MÉ, Wiley JF, Czeisler CA, Rajaratnam SMW, Howard ME. Uncovering survivorship bias in longitudinal mental health surveys during the COVID-19 pandemic. Epidemiol Psychiatr Sci. 2021. https://doi.org/10.1017/S204579602100038X.

Cheng A, Zamarro G, Orriens B. Personality as a predictor of unit nonresponse in an internet panel. Sociol Methods Res. 2020;49(3):672–98. https://doi.org/10.1177/0049124117747305.

Satherley N, Milojev P, Greaves LM, et al. Demographic and psychological predictors of panel attrition: evidence from the New Zealand Attitudes and Values Study. PLoS ONE. 2015;10(3):e0121950. https://doi.org/10.1371/journal.pone.0121950.

Chin A, Couper MP, Beckett D. Attrition in a longitudinal online randomized controlled trial with prospective homeowners. Field Methods. 2021;33(1):3–24. https://doi.org/10.1177/1525822X20948308.

Biddle N, Sollis K. Determinants of participation in a longitudinal survey during the COVID-19 pandemic: the case of a low-infection country. ANU Centre for Social Research & Methods. Published June 4, 2021. https://csrm.cass.anu.edu.au/research/publications/determinants-participation-longitudinal-survey-during-covid-19-pandemic-case. Accessed 18 Aug 2021.

Frankel LL, Hillygus DS. Looking beyond demographics: panel attrition in the ANES and GSS. Polit Anal. 2014;22(3):336–53. https://doi.org/10.1093/pan/mpt020.

Barber J, Kusunoki Y, Gatny H, Schulz P. Participation in an intensive longitudinal study with weekly web surveys over 2.5 years. J Med Internet Res. 2016;18(6):e5422. https://doi.org/10.2196/jmir.5422.

Daniels RC, Ingle K, Brophy T. Determinants of attrition in NIDS-CRAM waves 1 & 2. 2020. https://localhost:8080/handle/11090/994. Accessed 18 Aug 2021.

CDC. COVID Data Tracker. Centers for Disease Control and Prevention. Published March 28, 2020. https://covid.cdc.gov/covid-data-tracker. Accessed 11 Nov 2021.

Sollis K, Biddle N, Edwards B, Herz D. COVID-19 survey participation and wellbeing: a survey experiment. J Empir Res Hum Res Ethics. 2021;16(3):179–87. https://doi.org/10.1177/15562646211019659.

Ramsey I, de Rooij BH, Mols F, et al. Cancer survivors who fully participate in the PROFILES registry have better health-related quality of life than those who drop out. J Cancer Surviv. 2019;13(6):829–39. https://doi.org/10.1007/s11764-019-00793-7.

Levay KE, Freese J, Druckman JN. The demographic and political composition of Mechanical Turk samples. SAGE Open. 2016;6(1):2158244016636433. https://doi.org/10.1177/2158244016636433.

Paolacci G, Chandler J, Ipeirotis PG. Running experiments on Amazon Mechanical Turk. Social Science Research Network; 2010. https://papers.ssrn.com/abstract=1626226. Accessed 16 Oct 2021.

Paolacci G, Chandler J. Inside the Turk: understanding Mechanical Turk as a participant pool. Curr Dir Psychol Sci. 2014;23(3):184–8. https://doi.org/10.1177/0963721414531598.

Huff C, Tingley D. “Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents. Res Polit. 2015;2(3):2053168015604648. https://doi.org/10.1177/2053168015604648.

Mortensen K, Alcalá MG, French MT, Hu T. Self-reported health status differs for Amazon’s Mechanical Turk respondents compared with nationally representative surveys. Med Care. 2018;56(3):211–5. https://doi.org/10.1097/MLR.0000000000000871.

Shapiro DN, Chandler J, Mueller PA. Using Mechanical Turk to study clinical populations. Clinical Psychological Science. 2013;1(2):213–20. https://doi.org/10.1177/2167702612469015.

Berinsky AJ, Huber GA, Lenz GS. Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Polit Anal. 2012;20(3):351–68. https://doi.org/10.1093/pan/mpr057.

Ross J, Irani L, Silberman MS, Zaldivar A, Tomlinson B. Who are the crowdworkers? Shifting demographics in Mechanical Turk. In CHI ’10 extended abstracts on human factors in computing systems. CHI EA ’10. Association for Computing Machinery; 2010. p. 2863–72. doi:https://doi.org/10.1145/1753846.1753873

Sprouse J. A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory. Behav Res. 2011;43(1):155–67. https://doi.org/10.3758/s13428-010-0039-7.

Walters K, Christakis DA, Wright DR. Are Mechanical Turk worker samples representative of health status and health behaviors in the U.S.? PLoS ONE. 2018;13(6):e0198835. https://doi.org/10.1371/journal.pone.0198835.

Thông tin
Thông tin xuất bản

Health and Quality of Life Outcomes

Tập 20 Số 1

Thông tin tác giả