Weather Forecasting Using Ensemble of Spatial-Temporal Attention Network and Multi-Layer Perceptron
Tóm tắt
Weather forecasting is a challenging task, which is especially suited for artificial intelligence due to the large amount of data involved. This paper proposed an end-to-end hybrid regression model, called Ensemble of Spatial-Temporal Attention Network and Multi-Layer Perceptron (E-STAN-MLP), to forecast surface temperature, humidity, wind speed, and wind direction at 24 automatic weather stations in Beijing. Combining the data from historical observations with the data from the numerical weather prediction (NWP) system, our proposed model give better results than the NWP system or previously reported algorithms. Our E-STAN-MLP model consists of two parts. One is to use the spatial-temporal attention based recurrent neural network to model the time series of meteorological elements. The other is a simple but efficient multi-layer perceptron architecture forecasts the regression value while ignoring time dependence. Results at each time stamp are integrated together using a step-wise fusion strategy. Moreover, we use a joint loss step integrating both the regression loss function and the classification loss function to simultaneously forecast the wind speed and direction. Experiments demonstrate that our proposed E-STAN-MLP model achieves state-of-the-art results in weather forecasting.
Tài liệu tham khảo
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv:1409.0473 (2014)
Chen, N., Qian, Z., Nabney, I.T., Meng, X.: Short-term wind power forecasting using gaussian processes. In: Twenty-third international joint conference on artificial intelligence (2013)
Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp 785–794 (2016)
Grover, A., Kapoor, A., Horvitz, E.: A deep hybrid model for weather forecasting. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 379–386 (2015)
Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based Spatial-Temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp 922–929 (2019)
Hernández, E., Sanchez-Anguix, V., Julian, V., Palanca, J., Duque, N.: Rainfall prediction: A deep learning approach. In: International Conference on Hybrid Artificial Intelligence Systems, pp 151–162 (2016)
Lai, G., Chang, W.-C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp 95–104 (2018)
Liang, Y., Ke, S., Zhang, J., Yi, X., Zheng, Y.: GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction. In: IJCAI, pp 3428–3434 (2018)
Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988 (2017)
Liu, H., Tian, H.-Q., Li, Y.-F.: Comparison of two new ARIMA-ANN and ARIMA-kalman hybrid methods for wind speed prediction. Appl. Energy 98, 415–424 (2012)
Lorenz, E.N.: Deterministic nonperiodic flow, journal of the atmospheric sciences vol. 20, No. In. XX (1963)
Luo, P., Ren, J., Peng, Z., Zhang, R., Li, J.: Differentiable learning-to-normalize via switchable normalization. arXiv:1806.10779 (2018)
Lynch, P.: Weather prediction by numerical process weather prediction by numerical process (2006)
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings icml, p 3 (2013)
Marchuk, G.: Numerical methods in weather prediction. Numer. Meth. Weather Predict. 259–273 (1974)
Percival, D.B., Walden, A.T.: Spectral analysis for physical applications. Cambridge University Press, Cambridge (1993)
Shih, S.-Y., Sun, F.-K., Lee, H.-Y.: Temporal pattern attention for multivariate time series forecasting. Mach Learn 108 (2019)
Song, D., Chen, H., Jiang, G., Qin, Y.: Dual Stage Attention Based Recurrent Neural Network for Time Series Prediction. In: Google Patents (2018)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112 (2014)
Taylor, S., Letham, B.: Forecasting at scale. The American Statistician 72 (2017)
Tolstykh, M., Frolov, A.: Some current problems in numerical weather prediction. Izve. Atmos. Ocean. Phys. 41, 285–295 (2005)
Voyant, C., Muselli, M., Paoli, C., Nivet, M. -L.: Numerical Weather Prediction (NWP) and hybrid ARMA/ANN model to predict global radiation. Energy 39 (2012)
Wang, B., Lu, J., Yan, Z., Luo, H., Li, T., Zheng, Y., Zhang, G.: Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 2087–2095. ACM, New York (2019)
Xie, Y., Fan, S., Chen, M., Shi, J., Zhong, J., Zhang, X.: An assessment of satellite radiance data assimilation in RMAPS. Remote Sens 11(1), 54 (2019)
Xu, W., Peng, H., Zeng, X., Zhou, F., Tian, X., Peng, X.: A hybrid modelling method for time series forecasting based on a linear regression model and deep learning. Appl Intell 1–14 (2019)