DigitalExposome: quantifying impact of urban environment on wellbeing using sensor fusion and deep learning

Computational Urban Science
Thomas J. Johnson1, Eiman Kanjo1, Kieran Woodward1
1Department of Computer Science, Nottingham Trent University, Nottingham, UK

Tóm tắt

AbstractThe increasing level of air pollutants (e.g. particulates, noise and gases) within the atmosphere are impacting mental wellbeing. In this paper, we define the term ‘DigitalExposome’ as a conceptual framework that takes us closer towards understanding the relationship between environment, personal characteristics, behaviour and wellbeing using multimodal mobile sensing technology. Specifically, we simultaneously collected (for the first time) multi-sensor data including urban environmental factors (e.g. air pollution including: Particulate Matter (PM1), (PM2.5), (PM10), Oxidised, Reduced, Ammonia (NH3) and Noise, People Count in the vicinity), body reaction (physiological reactions including: EDA, HR, HRV, Body Temperature, BVP and movement) and individuals’ perceived responses (e.g. self-reported valence) in urban settings. Our users followed a pre-specified urban path and collected the data using a comprehensive sensing edge device. The data is instantly fused, time-stamped and geo-tagged at the point of collection. A range of multivariate statistical analysis techniques have been applied including Principle Component Analysis, Regression and Spatial Visualisations to unravel the relationship between the variables. Results showed that Electrodermal Activity (EDA) and Heart Rate Variability (HRV) are noticeably impacted by the level of Particulate Matter in the environment. Furthermore, we adopted Convolutional Neural Network (CNN) to classify self-reported wellbeing from the multimodal dataset which achieved an f1-score of 0.76.

Từ khóa


Tài liệu tham khảo

Air, C., & Plan S. (2021) Clean Air Strategy Plan. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/770715/cleanair-strategy-2019.pdf

Alajmi, N., Kanjo, E., Chamberlain, A., & Mawass, N. E. (2013) Shopmobia: An emotion-based shop rating system. https://doi.org/10.1109/ACII.2013.138

Al-barrak, L., Kanjo, E., & Younis, E. M. G. (2017). Neuroplace: Categorizing urban places according to mental states. PLOS ONE, 12, 0183890. https://doi.org/10.1371/JOURNAL.PONE.0183890

Aspinall, P., Mavros, P., Coyne, R., & Roe, J. (2015). The urban brain: Analysing outdoor physical activity with mobile eeg. British Journal of Sports Medicine, 49, 272–276. https://doi.org/10.1136/bjsports-2012-091877

Bai, Shaojie et al. (2018) “An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling.” ArXiv abs/1803.01271. n. pag

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25, 49–59. https://doi.org/10.1016/0005-7916(94)90063-9

Cummins, R.A., & Ps, F.A.S. (2013) Personal Wellbeing Index-Adult (PWI-A) (English) 5 Th Edition The International Wellbeing Group MANUAL 2013 Personal Wellbeing Index-Adult

DeBord, D. G., Carreón, T., Lentz, T. J., Middendorf, P. J., Hoover, M. D., & Schulte, P. A. (2016). Use of the “exposome’’ in the practice of epidemiology: A primer on -omic technologies. American Journal of Epidemiology, 184, 302–314. https://doi.org/10.1093/aje/kwv325

Dobrin, A. (2015) A Review of Properties and Variations of Voronoi Diagrams. http://www.whitman.edu/mathematics/SeniorProjectArchive/2005/dobrinat.pdf. Accessed 18 Oct 22

Donaire-Gonzalez, D., Valentín, A., van Nunen, E., Curto, A., Rodriguez, A., Fernandez-Nieto, M., Naccarati, A., Tarallo, S., et al. (2019). Expoapp: An integrated system to assess multiple personal environmental exposures. Environment International, 126, 494–503. https://doi.org/10.1016/j.envint.2019.02.054

Guite, H. F., Clark, C., & Ackrill, G. (2006). The impact of the physical and urban environment on mental well-being. Public Health, 120, 1117–1126. https://doi.org/10.1016/j.puhe.2006.10.005

Ji, H. I. (2022) Ambient Air Quality and Health. http://www.who.int/mediacentre/factsheets/fs313/en/. Accessed 19 Dec 22

Johnson, T., & Kanjo, E. (2021) Sensor fusion and the city: Visualisation and aggregation of environmental & wellbeing data. IEEE. https://doi.org/10.1109/ISC253183.2021.9562852, https://ieeexplore.ieee.org/document/9562852. Accessed 03 Mar 2023 

Johnson, T., Kanjo, E., & Woodward, K. (2020) Sensor data and the city: Urban visualisation and aggregation of well-being data. arXiv

Jollife, I. T., & Cadima, J. (2016). Principal Component Analysis: A Review and Recent Developments. https://doi.org/10.1098/rsta.2015.0202

Kanjo, E. (2010). Noisespy: A real-time mobile phone platform for urban noise monitoring and mapping. Mobile Networks and Applications, 15, 562–574. https://doi.org/10.1007/S11036-009-0217-Y

Kanjo, E., Younis, E. M. G., & Sherkat, N. (2018). Towards unravelling the relationship between on-body, environmental and emotion data using sensor information fusion approach. Information Fusion, 40, 18–31. https://doi.org/10.1016/j.inffus.2017.05.005

Kinnunen, H., Rantanen, A., Kentt, T., & Koskimäki, H. (2020). Feasible assessment of recovery and cardiovascular health: Accuracy of nocturnal hr and hrv assessed via ring ppg in comparison to medical grade ecg. Physiological Measurement, 41, 04–01. https://doi.org/10.1088/1361-6579/ab840a

Laville, S. (2020) Air Pollution a Cause in Girl’s Death, Coroner Rules in Landmark Case | London | The Guardian

Liang, Y., Zheng, X., & Zeng, D. D. (2019). A survey on big data-driven digital phenotyping of mental health. Information Fusion, 52, 290–307. https://doi.org/10.1016/j.inffus.2019.04.001

Lisetti, C. L., & Nasoz, F. (2014) Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. https://doi.org/10.1155/S1110865704406192

Loh, M., Sarigiannis, D., Gotti, A., Karakitsios, S., Pronk, A., Kuijpers, E., Annesi-Maesano, I., Baiz, N., et al. (2017). How sensors might help define the external exposome. International Journal of Environmental Research and Public Health, 14, 343. https://doi.org/10.3390/ijerph14040434

Maitre, L., Bont, J. D., Casas, M., Robinson, O., Aasvang, G. M., Agier, L., Andrušaitytė, S., Ballester, F., et al. (2018) Human Early Life Exposome (HELIX) Study: A European Population-based Exposome Cohort. https://doi.org/10.1136/bmjopen-2017-021311

Mashima, D., Kobourov, S., & Hu, Y. (2012). Visualizing dynamic data with maps. IEEE Transactions on Visualization and Computer Graphics, 18, 1424–1437. https://doi.org/10.1109/TVCG.2011.288

Münzel, T., Gori, T., Babisch, W., & Basner, M. (2014). Cardiovascular Effects of Environmental Noise. Exposure.https://doi.org/10.1093/eurheartj/ehu030

Needham, J. (1959). Science and civilisation in china: Mathematics and the sciences of the heavens and the earth. Cambridge University Press, 3, 147.

Nieuwenhuijsen, M. J., Donaire-Gonzalez, D., Foraster, M., Martinez, D., & Cisneros, A. (2014). Using personal sensors to assess the exposome and acute health effects. International Journal of Environmental Research and Public Health, 11, 7805–7819. https://doi.org/10.3390/ijerph110807805

Pokojski, W., & Pokojska, P. (2018). Voronoi diagrams - inventor, method, applications. Polish Cartographical Review, 50, 141–150. https://doi.org/10.2478/pcr-2018-0009

Rosenthal, J. A. (2012) Statistics and data interpretation for social work. New York: Springer Publishing Company

Rumchev, K., Soares, M., Zhao, Y., Reid, C., & Huxley, R. (2018). The association between indoor air quality and adult blood pressure levels in a high-income setting. International Journal of Environmental Research and Public Health, 15,. https://doi.org/10.3390/IJERPH15092026

Scott, D. (2015) Q-Q Plots. http://onlinestatbook.com/2/advanced_graphs/q-q_plots.html. Accessed 15 Mar 2023

Sharma, N., & Gedeon, T. (2012). Objective measures, sensors and computational techniques for stress recognition and classification: A survey. Computer Methods and Programs in Biomedicine, 108, 1287–1301. https://doi.org/10.1016/j.cmpb.2012.07.003

Siroux, V., Agier, L., & Slama, R. (2016). The exposome concept: A challenge and a potential driver for environmental health research. European Respiratory Review, 25, 124–129. https://doi.org/10.1183/16000617.0034-2016

Stamatelopoulou, A., Chapizanis, D., Karakitsios, S., Kontoroupis, P., Asimakopoulos, D. N., Maggos, T., & Sarigiannis, D. (2018). Assessing and enhancing the utility of low-cost activity and location sensors for exposure studies. Environmental Monitoring and Assessment, 190. https://doi.org/10.1007/s10661-018-6537-2

Ueberham, M., & Schlink, U. (2018). Wearable sensors for multifactorial personal exposure measurements - a ranking study. Environment International, 121, 130–138. https://doi.org/10.1016/j.envint.2018.08.057

Vrijheid, M. (2014). Ethe exposome: A new paradigm to study the impact of environment on health. Thorax, 69, 876–878. https://doi.org/10.1136/thoraxjnl-2013-204949

Wild, C. P. (2005). Complementing the genome with an “exposome’’: The outstanding challenge of environmental exposure measurement in molecular epidemiology. Cancer Epidemiology Biomarkers and Prevention, 14, 1847–1850. https://doi.org/10.1158/1055-9965.EPI-05-0456

Wild, C. P. (2012). The exposome: From concept to utility. International Journal of Epidemiology, 41, 24–32. https://doi.org/10.1093/ije/dyr236

Woodward, K., Kanjo, E., Tsanas, A., 2020. Combining Deep Transfer Learning with Signal-image Encoding for Multi-Modal Mental Wellbeing Classification. https://doi.org/10.48550/arxiv.2012.03711

Woodward, K., Kanjo, E., Brown, D., McGinnity, T. M., Inkster, B., Macintyre, D. J., & Tsanas, A. (2019) Beyond mobile apps: A survey of technologies for mental well-being. arXiv,

Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018). Convolutional neural networks: an overview and application in radiology. Insights into Imaging, 9, 611–629. https://doi.org/10.1007/S13244-018-0639-9/FIGURES/15

Thông tin
Thông tin xuất bản

Computational Urban Science

Thông tin tác giả