Weather Forecasting Using Ensemble of Spatial-Temporal Attention Network and Multi-Layer Perceptron

Springer Science and Business Media LLC - Tập 57 - Trang 533-546 - 2020
Yuanpeng Li1, Junwei Lang1, Lei Ji2, Jiqin Zhong2, Zaiwen Wang2, Yang Guo1,3, Sailing He1,3
1National Engineering Research Center for Optical Instruments, Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou, China
2Institute of Urban Meteorology (IUM), China Meteorological Administration, Beijing, China
3Ningbo Research Institute, Zhejiang University, Ningbo, China

Tóm tắt

Weather forecasting is a challenging task, which is especially suited for artificial intelligence due to the large amount of data involved. This paper proposed an end-to-end hybrid regression model, called Ensemble of Spatial-Temporal Attention Network and Multi-Layer Perceptron (E-STAN-MLP), to forecast surface temperature, humidity, wind speed, and wind direction at 24 automatic weather stations in Beijing. Combining the data from historical observations with the data from the numerical weather prediction (NWP) system, our proposed model give better results than the NWP system or previously reported algorithms. Our E-STAN-MLP model consists of two parts. One is to use the spatial-temporal attention based recurrent neural network to model the time series of meteorological elements. The other is a simple but efficient multi-layer perceptron architecture forecasts the regression value while ignoring time dependence. Results at each time stamp are integrated together using a step-wise fusion strategy. Moreover, we use a joint loss step integrating both the regression loss function and the classification loss function to simultaneously forecast the wind speed and direction. Experiments demonstrate that our proposed E-STAN-MLP model achieves state-of-the-art results in weather forecasting.

Tài liệu tham khảo

Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv:1409.0473 (2014) Chen, N., Qian, Z., Nabney, I.T., Meng, X.: Short-term wind power forecasting using gaussian processes. In: Twenty-third international joint conference on artificial intelligence (2013) Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp 785–794 (2016) Grover, A., Kapoor, A., Horvitz, E.: A deep hybrid model for weather forecasting. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 379–386 (2015) Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based Spatial-Temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp 922–929 (2019) Hernández, E., Sanchez-Anguix, V., Julian, V., Palanca, J., Duque, N.: Rainfall prediction: A deep learning approach. In: International Conference on Hybrid Artificial Intelligence Systems, pp 151–162 (2016) Lai, G., Chang, W.-C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp 95–104 (2018) Liang, Y., Ke, S., Zhang, J., Yi, X., Zheng, Y.: GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction. In: IJCAI, pp 3428–3434 (2018) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988 (2017) Liu, H., Tian, H.-Q., Li, Y.-F.: Comparison of two new ARIMA-ANN and ARIMA-kalman hybrid methods for wind speed prediction. Appl. Energy 98, 415–424 (2012) Lorenz, E.N.: Deterministic nonperiodic flow, journal of the atmospheric sciences vol. 20, No. In. XX (1963) Luo, P., Ren, J., Peng, Z., Zhang, R., Li, J.: Differentiable learning-to-normalize via switchable normalization. arXiv:1806.10779 (2018) Lynch, P.: Weather prediction by numerical process weather prediction by numerical process (2006) Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings icml, p 3 (2013) Marchuk, G.: Numerical methods in weather prediction. Numer. Meth. Weather Predict. 259–273 (1974) Percival, D.B., Walden, A.T.: Spectral analysis for physical applications. Cambridge University Press, Cambridge (1993) Shih, S.-Y., Sun, F.-K., Lee, H.-Y.: Temporal pattern attention for multivariate time series forecasting. Mach Learn 108 (2019) Song, D., Chen, H., Jiang, G., Qin, Y.: Dual Stage Attention Based Recurrent Neural Network for Time Series Prediction. In: Google Patents (2018) Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112 (2014) Taylor, S., Letham, B.: Forecasting at scale. The American Statistician 72 (2017) Tolstykh, M., Frolov, A.: Some current problems in numerical weather prediction. Izve. Atmos. Ocean. Phys. 41, 285–295 (2005) Voyant, C., Muselli, M., Paoli, C., Nivet, M. -L.: Numerical Weather Prediction (NWP) and hybrid ARMA/ANN model to predict global radiation. Energy 39 (2012) Wang, B., Lu, J., Yan, Z., Luo, H., Li, T., Zheng, Y., Zhang, G.: Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 2087–2095. ACM, New York (2019) Xie, Y., Fan, S., Chen, M., Shi, J., Zhong, J., Zhang, X.: An assessment of satellite radiance data assimilation in RMAPS. Remote Sens 11(1), 54 (2019) Xu, W., Peng, H., Zeng, X., Zhou, F., Tian, X., Peng, X.: A hybrid modelling method for time series forecasting based on a linear regression model and deep learning. Appl Intell 1–14 (2019)