Robust Transformer-based model for spatiotemporal PM $$_{2.5}$$ prediction in California
Springer Science and Business Media LLC - Trang 1-14
Tóm tắt
Fine particulate matter (PM $$_{2.5}$$ ) poses a significant public health risk due to its association with respiratory and cardiovascular diseases. Given the limited availability of PM $$_{2.5}$$ monitoring stations, there is a pressing need for reliable real-time forecasting models. This study introduces TSPPM25, a novel Transformer-based model designed for spatiotemporal prediction of PM $$_{2.5}$$ levels. TSPPM25 leverages multiple data embedding techniques and various attention layers to effectively capture the intricate spatiotemporal relationships in multivariate data. The model’s performance is evaluated using a California Aerosol Optical Depth PM $$_{2.5}$$ dataset and compared with several baseline models, including LSTM, Bi-LSTM, Linear Regression, and basic heuristics models. The results demonstrate that TSPPM25 exhibits superior spatiotemporal learning capabilities, robustness, and stability, outperforming other models across MSE, MAE, and SMAPE metrics. Furthermore, the study explores the underlying patterns in PM $$_{2.5}$$ data through harmonic analysis, revealing that TSPPM25 performs exceptionally well even in complex scenarios characterized by mixed wave patterns. Conclusively, TSPPM25 emerges as a valuable tool for predicting PM $$_{2.5}$$ levels demonstrating its efficacy in the California region, and thereby contributing significantly to the field of air quality forecasting.
Tài liệu tham khảo
Brokamp C, Jandarov R, Rao M et al (2017) Exposure assessment models for elemental components of particulate matter in an urban environment: a comparison of regression and random forest approaches. Atmos Environ 151:1–11.
https://doi.org/10.1016/j.atmosenv.2016.11.066
Chu Y, Liu Y, Li X et al (2016) A review on predicting ground pm2.5 concentration using satellite aerosol optical depth. Atmosphere 7(10):129.
https://doi.org/10.3390/atmos7100129
citation_journal_title=IEEE Trans Geosci Remote Sens; citation_title=Aerosol optical depth retrieval over the nasa stennis space center: Mti, modis, and aeronet; citation_author=P Chylek, BG Henderson, G Lesins; citation_volume=43; citation_issue=9; citation_publication_date=2005; citation_pages=1978-1983; citation_doi=10.1109/TGRS.2005.851773; citation_id=CR3
Devlin J, Chang MW, Lee K et al (2018) Bert: pre-training of deep bidirectional transformers for language understanding.
https://doi.org/10.48550/arXiv.1810.04805
.
arXiv:1810.04805
Di Q, Kloog I, Koutrakis P et al (2016) Assessing pm2.5 exposures with high spatiotemporal resolution across the continental united states. Environ Sci Technol 50(9):4712–4721.
https://doi.org/10.1021/acs.est.5b06121
citation_journal_title=New England J Med; citation_title=An association between air pollution and mortality in six us cities; citation_author=DW Dockery, CA Pope, X Xu; citation_volume=329; citation_issue=24; citation_publication_date=1993; citation_pages=1753-1759; citation_doi=10.1056/NEJM199312093292401; citation_id=CR6
EPA (2006) Air quality guidelines: global update 2005: particulate matter, ozone, nitrogen dioxide, and sulfur dioxide. World Health Organization
Fan J, Li S, Fan C et al (2016) The impact of pm2.5 on asthma emergency department visits: a systematic review and meta-analysis. Environ Sci Pollut Res 23(1):843–850.
https://doi.org/10.1007/s11356-015-5321-x
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778.
https://doi.org/10.1109/CVPR.2016.90
Holben BN, Eck TF, Ia Slutsker et al (1998) Aeronet-a federated instrument network and data archive for aerosol characterization. Remote Sens Environ 66(1):1–16.
https://doi.org/10.1016/S0034-4257(98)00031-5
Holben BN, Tanre D, Smirnov A et al (2001) An emerging ground-based aerosol climatology: aerosol optical depth from aeronet. J Geophys Res: Atmos 106(D11):12,067–12,097.
https://doi.org/10.1029/2001JD900014
Hu X, Belle JH, Meng X et al (2017) Estimating pm2.5 concentrations in the conterminous united states using the random forest approach. Environ Sci Technol 51(12):6936–6944.
https://doi.org/10.1021/acs.est.7b01210
citation_journal_title=Catena; citation_title=Comparisons of heuristic, general statistical and machine learning models for landslide susceptibility prediction and mapping; citation_author=F Huang, Z Cao, J Guo; citation_volume=191; citation_issue=104; citation_publication_date=2020; citation_pages=580; citation_doi=10.1016/j.catena.2020.104580; citation_id=CR13
Kazemi SM, Goel R, Eghbali S et al (2019) Time2vec: learning a vector representation of time.
https://doi.org/10.48550/arXiv.1907.05321
.
arXiv:1907.05321
Lee KH, Chen X, Hua G et al (2018) Stacked cross attention for image-text matching. In: Proceedings of the european conference on computer vision (ECCV), pp 201–216,
https://doi.org/10.1007/978-3-030-01225-0_13
Limperis J, Tong W, Hamza-Lup F et al (2023) Pm2.5 forecasting based on transformer neural network and data embedding. Earth Sci Inf 1–14.
https://doi.org/10.1007/s12145-023-01002-x
citation_journal_title=Atmospheric Environ: X; citation_title=No one knows which city has the highest concentration of fine particulate matter; citation_author=RV Martin, M Brauer, A Donkelaar; citation_volume=3; citation_issue=100; citation_publication_date=2019; citation_pages=040; citation_doi=10.1016/j.aeaoa.2019.100040; citation_id=CR17
citation_journal_title=Environ Int; citation_title=Space-time trends of pm2.5 constituents in the conterminous united states estimated by a machine learning approach, 2005–2015; citation_author=X Meng, JL Hand, BA Schichtel; citation_volume=121; citation_publication_date=2018; citation_pages=1137-1147; citation_doi=10.1016/j.envint.2018.10.029; citation_id=CR18
Morain SA, Budge AM (2012) Environmental tracking for public health surveillance. CRC Press.
https://doi.org/10.1201/b12680
Nussbaumer HJ (1981) The fast fourier transform. In: Fast fourier transform and convolution algorithms. Springer, pp 80–111.
https://doi.org/10.1007/978-3-662-00551-4_4
Oord Avd, Dieleman S, Zen H et al (2016) Wavenet: a generative model for raw audio.
arXiv:1609.03499
.
https://doi.org/10.48550/arXiv.1609.03499
Qi Y, Li Q, Karimian H et al (2019) A hybrid model for spatiotemporal forecasting of pm2. 5 based on graph convolutional neural network and long short-term memory. Sci Total Environ 664:1–10.
https://doi.org/10.1016/j.scitotenv.2019.01.333
Samet JM, Dominici F, Curriero FC et al (2000) Fine particulate air pollution and mortality in 20 us cities, 1987–1994. New England J Med 343(24):1742–1749.
https://doi.org/10.1056/NEJM200012143432401
citation_journal_title=Adv Space Res; citation_title=Terra and aqua modis products available from nasa ges daac; citation_author=A Savtchenko, D Ouzounov, S Ahmad; citation_volume=34; citation_issue=4; citation_publication_date=2004; citation_pages=710-714; citation_doi=10.1016/j.asr.2004.03.012; citation_id=CR24
Schliep E, Gelfand A, Holland D (2015) Autoregressive spatially varying coefficients model for predicting daily pm2.5 using viirs satellite aot. Adv Stat Climatol Meteorol Oceanograph 1(1):59–74.
https://doi.org/10.5194/ascmo-1-59-2015
Shen S, Yao Z, Gholami A et al (2020) Powernorm: rethinking batch normalization in transformers. In: International conference on machine learning, pp 8741–8751.
https://doi.org/10.48550/arXiv.2003.07845
citation_title=Fourier analysis: an introduction,; citation_publication_date=2011; citation_id=CR27; citation_author=EM Stein; citation_author=R Shakarchi; citation_publisher=Princeton University Press
citation_journal_title=Atmosph Environ; citation_title=Emissions estimation from satellite retrievals: a review of current capability; citation_author=DG Streets, T Canty, GR Carmichael; citation_volume=77; citation_publication_date=2013; citation_pages=1011-1042; citation_doi=10.1016/j.atmosenv.2013.05.051; citation_id=CR28
Sun W, Zhang H, Palazoglu A et al (2013) Prediction of 24-hour-average pm2.5 concentrations using a hidden markov model with different emission distributions in northern California. Sci Total Environ 443:93–103.
https://doi.org/10.1016/j.scitotenv.2012.10.070
Tobler WR (1970) A computer movie simulating urban growth in the Detroit region. Economic Geograph 46:234–240.
https://doi.org/10.2307/143141
Tong W, Li L, Zhou X et al (2019) Efficient spatiotemporal interpolation with spark machine learning. Earth Sci Inf 12:87–96.
https://doi.org/10.1007/s12145-018-0364-4
Tong W, Li L, Zhou X et al (2019) Deep learning pm2.5 concentrations with bidirectional lstm rnn. Air Q Atmos Health 12:411–423.
https://doi.org/10.1007/s11869-018-0647-4
Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008.
https://doi.org/10.5555/3295222.3295349
Xi X, Wei Z, Xiaoguang R et al (2015) A comprehensive evaluation of air pollution prediction improvement by a machine learning method. In: 2015 IEEE international conference on service operations and logistics, and informatics (SOLI), IEEE, pp 176–181.
https://doi.org/10.1109/SOLI.2015.7367615
Xing YF, Xu YH, Shi MH et al (2016) The impact of pm2.5 on the human respiratory system. J Thoracic Disease 8(1):E69.
https://doi.org/10.3978/j.issn.2072-1439.2016.01.19
Zerveas G, Jayaraman S, Patel D et al (2021) A transformer-based framework for multivariate time series representation learning. In: The 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 2114–2124.
https://doi.org/10.1145/3447548.3467401
Zhou X, Tong W, Li L (2020) Deep learning spatiotemporal air pollution data in china using data fusion. Earth Sci Inf 13:859–868.
https://doi.org/10.1007/s12145-020-00470-9