Self-reported data in environmental health studies: mail vs. web-based surveys
Tóm tắt
Internet has been broadly employed as a facilitator for epidemiological surveys, as a way to provide a more economical and practical alternative to traditional survey modes. A current trend in survey research is to combine Web-based surveys with other survey modes by offering the participant the possibility of choosing his/her preferred response method (i.e. mixed-mode approach). However, studies have also demonstrated that the use of different survey modes may produce different responses to the same questions, posing potential challenges on the use of mixed-mode approaches. In this paper, we have implemented a statistical comparison between mixed-mode survey responses collected via mail (i.e. paper) and Web methods obtained from a cross-sectional study in non-urban areas of Denmark. Responses provided by mail and Web participants were compared in terms of: 1) the impact of reminder letters in increasing response rates; 2) differences in socio-demographic characteristics between response groups; 3) changes on the likelihood of reporting health symptoms and negative attitudes towards environmental stressors. Comparisons were mainly performed by two sample t-test, Pearson’s Chi-squared test and multinomial logistic regression models. Among 3104 contacted households, 1066 residents decided to participate on the study. Out of those, 971 selected to respond via mail, whereas 275 preferred the Web method. The majority of socio-demographic characteristics between these two groups of respondents were shown to be statistically different. The use of mailed surveys increased the likelihood of reporting health symptoms and negative attitudes towards environmental stressors, even after controlling for demographic characteristics. Furthermore, the use of reminder letters had a higher positive impact in increasing responses of Web surveys when compared to mail surveys. Our main findings suggest that the use of mail and Web surveys may produce different responses to the same questions posed to participants, but, at the same time, may reach different groups of respondents, given that the overall characteristics of both groups considerably differ. Therefore, the tradeoff between using mixed-mode survey as a way to increase response rate and obtaining undesirable measurement changes may be attentively considered in future survey studies.
Tài liệu tham khảo
Hayslett MM, Wildemuth BM. Pixels or pencils? The relative effectiveness of web-based versus paper surveys. Libr Inf Sci Res. 2004;26:73–93.
Daikeler J, Bošnjak M, Manfreda KL. Web versus other survey modes: an updated and extended meta-analysis comparing response rates. J Surv Stat Methodol. 2019;0:1–27.
Braunsberger K, Wybenga H, Gates R. A comparison of reliability between telephone and web-based surveys. J Bus Res. 2007;60:758–64.
Marta-Pedroso C, Freitas H, Domingos T. Testing for the survey mode effect on contingent valuation data quality: a case study of web based versus in-person interviews. Ecol Econ. 2007;62:388–98.
Beck KH, Yan AF, Wang MQ. A comparison of web-based and telephone surveys for assessing traffic safety concerns, beliefs, and behaviors. J Saf Res. 2009;40:377–81. https://doi.org/10.1016/j.jsr.2009.07.007.
Szolnoki G, Hoffmann D. Online, face-to-face and telephone surveys - comparing different sampling methods in wine consumer research. Wine Econ Policy. 2013;2:57–66. https://doi.org/10.1016/j.wep.2013.10.001.
Fleming CM, Bowden M. Web-based surveys as an alternative to traditional mail methods. J Environ Manag. 2009;90:284–92. https://doi.org/10.1016/j.jenvman.2007.09.011.
Smyth JD, Dillman DA, Christian LM, O’Neill AC. Using the internet to survey small towns and communities: limitations and possibilities in the early 21st century. Am Behav Sci. 2010;53:1423–48.
Carini RM, Hayek JC, Kuh GD, Kennedy JM, Ouimet JA. College student responses to web and paper surveys: does mode matter? Res High Educ. 2003;44:1–19.
Kim Y, Dykema J, Stevenson J, Black P, Moberg DP. Straightlining: overview of measurement, comparison of indicators, and effects in mail–web mixed-mode surveys. Soc Sci Comput Rev. 2019;37:214–33.
Sinclair M, O’Toole J, Malawaraarachchi M, Leder K. Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches. BMC Med Res Methodol. 2012;12:132. https://doi.org/10.1186/1471-2288-12-132.
Keramitsoglou KM, Manfreda KL, Anastasiou C, Skjak KK, Tsagarakis KP. Mode comparison study on willingness to buy and willingness to pay for organic foods: paper-and-pencil versus computerized questionnaire. Electron Commer Res. 2018;18:587–603.
Fowler FJ, Cosenza C, Cripps LA, Edgman-Levitan S, Cleary PD. The effect of administration mode on CAHPS survey response rates and results: a comparison of mail and web-based approaches. Health Serv Res. 2019;54:714–21.
de Leeuw ED, Hox JJ. Self-administered questionnaires: mail surveys and other applications. In: International Handbook of Survey Methodology. 2012.
de Leeuw E, Berzelak N. Survey mode or survey modes? In: The SAGE Handbook of Survey Methodology. 2016. p. 1–18.
Dykema J, Stevenson J, Klein L, Kim Y, Day B. Effects of E-mailed versus mailed invitations and incentives on response rates, data quality, and costs in a web survey of university faculty. Soc Sci Comput Rev. 2013;31:359–70.
Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Bangkok). 2005;27:281–91.
Dillman DA, Phelps G, Tortora R, Swift K, Kohrell J, Berck J, et al. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the internet. Soc Sci Res. 2009;38:1–18. https://doi.org/10.1016/j.ssresearch.2008.03.007.
Dillman DA, Christian LM. Survey mode as a source of instability in responses across surveys. Field Methods. 2005;17:30–52.
Tourangeau R, Rips LJ, Rasinki K. The psychology of survey response. Cambridge: Cambridge University Press; 2000.
Fan W, Yan Z. Factors affecting response rates of the web survey: a systematic review. Comput Hum Behav. 2010;26:132–9. https://doi.org/10.1016/j.chb.2009.10.015.
Koitsalu M, Eklund M, Adolfsson J, Grönberg H, Brandberg Y. Effects of pre-notification, invitation length, questionnaire length and reminder on participation rate: a quasi-randomised controlled trial. BMC Med Res Methodol. 2018;18:1–5.
Pit SW, Vo T, Pyakurel S. The effectiveness of recruitment strategies on general practitioner’s survey response rates – a systematic review. BMC Med Res Methodol. 2014;14:1–14.
Kaplowitz MD, Hadlock TD, Levine R. A comparison of web and mail survey response rates. Public Opin Q. 2004;68:94–101. https://doi.org/10.1093/poq/nfh006.
Cook C, Heath F, Thompson R. A meta-analysis of response rates in web-or internet-based surveys. Educ Psychol Meas. 2000;60:821–36. https://doi.org/10.1177/00131640021970934.
Nakash RA, Hutton JL, Jørstad-Stein EC, Gates S, Lamb SE. Maximising response to postal questionnaires – a systematic review of randomised trials in health research. BMC Med Res Methodol. 2006;6:1–9. https://doi.org/10.1186/1471-2288-6-5.
Heckman JJ. Selection bias and self-selection. In: Durlauf SN, Blume LE, editors. Microeconometrics. London: Palgrave Macmillan UK; 2010. p. 242–66. https://doi.org/10.1057/9780230280816_29.
Brauer C, Mikkelsen S, Skov P. Reliability and validity of a new questionnaire for investigation of symptoms related to “the sick building syndrome” and perceived indoor air quality. 2000.
Villeneuve PJ, Ali A, Challacombe L, Hebert S. Intensive hog farming operations and self-reported health among nearby rural residents in Ottawa, Canada. BMC Public Health. 2009;9:1–10.
The American Associations for Public Opinion Research. Standard definitions: final dispositions of case codes and outcome rates for surveys. 8th ed; 2015.
Eurostat - Statistics Explained. Frequency of internet use. 2017. http://ec.europa.eu/eurostat/statistics-explained/index.php/Main_Page. Accessed 21 May 2017.
Eurostat. Digital economy and society statistics - households and individuals: statistics explained. 2019.
Rowe G, Frewer LJ. Public participation methods: a framework for evaluation. Sci Technol Hum Values. 2000;25:3–29. https://doi.org/10.1177/016224390002500101.
Kittleson MJ. An assessment of the response rate via the postal service and e-mail. Health Values. 1995;18:27–9.
Shih TH, Fan X. Comparing response rates in e-mail and paper surveys: a meta-analysis. Educ Res Rev. 2009;4:26–40.
Sax LJ, Gilmartin SK, Bryant AN. Assessing response rate and nonreponse bias in web and paper surveys. Res High Educ. 2003;44:409–32. https://doi.org/10.1023/A:1024232915870.
Czaja SJ, Charness N, Fisk AD, Hertzog C, Nair SN, Rogers WA, et al. Factors predicting the use of technology: findings from the center for research and education on aging and technology enhancement (create). Psychol Aging. 2006;21:333–52. https://doi.org/10.1037/0882-7974.21.2.333.
Schleife K. Computer use and the employment status of older workers - an analysis based on individual data. Labour. 2006;20:325–48.
Graefe A, Mowen A, Covelli E, Trauntvein N. Recreation participation and conservation attitudes: differences between mail and online respondents in a mixed-mode survey. Hum Dimens Wildl. 2011;16:183–99.
Millar MM, Dillman DA, Neill ACO. Are mode preferences real? 2009.
Groves RM, Cialdini RB, Couper MP. Understanding the decision to participate in a survey. Public Opin Q. 1992;56:475–95.
Halfacree KH. Talking about rurality: social representations of the rural as expressed by residents of six English parishes. J Rural Stud. 1995;11:1–20.
Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al. Increasing response rates to postal questionnaires. BMJ. 2002;324:1–9. https://doi.org/10.1136/bmj.325.7361.444.
Rothman KJ, Mikkelsen EM, Sørensen HT, Wise LA, Hatch EE. Randomized trial of questionnaire length. Epidemiology. 2009;20:154.
Dillman DA, Smyth JD. Design effects in the transition to web-based surveys. Am J Prev Med. 2007;32(5 SUPPL):S90–6.
Green KE. Sociodemographic factors and mail survey response. Psychol Mark. 1996;13:171–84.
Census Bureau US. 2011–2015 American Community Survey, 5 years estimates; 2015.
Thirthalli J, Reddy KS, Kare M, Das S, Gangadhar BN. Rural–urban differences in accessing mental health treatment in patients with psychosis. Int J Soc Psychiatry. 2017;63:694–8.
Lee GR, Lassey ML. Rural-urban differences among the elderly: economic, social, and subjective factors. J Soc Issues. 1980;36:62–74.
Carpiniello B, Carta MG, Rudas N. Depression among elderly people: a psychosocial study of urban and rural populations. Acta Psychiatr Scand. 1989;80:445–50.
Blanes-Vidal V, Suh H, Nadimi ES, Løfstrøm P, Ellermann T, Andersen HV, et al. Residential exposure to outdoor air pollution from livestock operations and perceived annoyance among citizens. Environ Int. 2012;40:44–50. https://doi.org/10.1016/j.envint.2011.11.010.
Blanes-Vidal V, Nadimi ES, Ellermann T, Andersen HV, Løfstrøm P. Perceived annoyance from environmental odors and association with atmospheric ammonia levels in non-urban residential communities: a cross-sectional study. Environ Health. 2012;11:1–10.
Blanes-Vidal V, Bælum J, Schwartz J, Løfstrøm P, Christensen LP. Respiratory and sensory irritation symptoms among residents exposed to low-to-moderate air pollution from biodegradable wastes. J Expo Sci Environ Epidemiol. 2014;24:388–97. https://doi.org/10.1038/jes.2014.20.