Neural network-based anomalous diffusion parameter estimation approaches for Gaussian processes

Jacobs, M.H.: Diffusion Processes. Ergebnisse der Biologie. Springer (1967) Nezhadhaghighi, M.G., Rajabpour, M.A., Rouhani, S.: First-passage-time processes and subordinated Schramm-0-Loewner evolution. Phys. Rev. E 84, 011134 (2011) Failla, R., Grigolini, P., Ignaccolo, M., Schwettmann, A.: Random growth of interfaces as a subordinated process. Phys. Rev. E 70, 010101(R) (2004) Stanislavsky, A., Weron, K.: Two-time scale subordination in physical processes with long-term memory. Ann. Phys. 323(3), 643–653 (2008) Gan, Y., Sun, L., Banhart, F.: One- and two-dimensional diffusion of metal atoms in graphene. Small 4(5), 587–591 (2008) Lee, S.T., Gao, Z.Q., Hung, L.S.: Metal diffusion from electrodes in organic light-emitting diodes. Applied Physics Letters 75(10), 1404–1406 (1999) Gabaix, X., Gopikrishnan, P., Plerou, V., Stanley, H.: A Theory of Power Law Distributions in Financial Market Fluctuations. Nature 423, 267–270 (2003) Ivanov, P.C., Yuen, A., Podobnik, B., Lee, Y.: Common scaling patterns in intertrade times of US stocks. Phys. Rev. E 69, 05610 (2004) Scher, H., Margolin, G., Metzler, R., Klafter, J.: The dynamical foundation of fractal stream chemistry: The origin of extremely long retention times. Geophysical Research Letters 29, 1061 (2002) Doukhan, P., Oppenheim, G., (Eds.), M.S.T.: Theory and Applications of Long-Range Dependence. Birkhäuser Boston, Inc., Boston (2003) Golding, I., Cox, E.C.: Physical nature of bacterial cytoplasm. Phys. Rev. Lett. 96, 098 (2006) Muir, D.: Bulk flow and diffusion in the airways of the lung. Br. J. Dis. Chest 60(4), 169–176 (1966) Mandelbrot, B.B., Van Ness, J.W.: Fractional Brownian Motions, Fractional Noises and Applications. SIAM Review 10(4), 422–437 (1968) Stanislavsky, A., Burnecki, K., Magdziarz, M., Weron, A., Weron, K.: FARIMA modelling of solar flare activity from empirical time series of soft X-Ray Solar emission. Astrophys. J. 693, 1877–1882 (2009) Smit, D.F.B.: Understanding molecular simulation: from algorithms to applications, 2nd edn, pp. 87–90. Computational science (San Diego, Calif.). Academic Press (2002) Beran, J.: Statistics for Long-Memory Processes. Chapman & Hall, pp. 55–59 (1994) Thiel, F., Sokolov, I.: Scaled brownian motion as a mean-field model for continuous-time random walks. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. 89, 012115 (2014) Thiel, F., Sokolov, I.M.: Scaled Brownian motion as a mean-field model for continuous-time random walks. Phys. Rev. E 89, 012 (2014) Safdari, H., Cherstvy, A.G., Chechkin, A.V., Bodrova, A., Metzler, R.: Aging underdamped scaled Brownian motion: Ensemble- and time-averaged particle displacements, nonergodicity, and the failure of the overdamping approximation. Phys. Rev. E 95, 012 (2017) Samorodnitsky, G., Taqqu, M.: Stable Non-Gaussian Random Processes. Chapman & Hall, New York, pp. 349–352 (1994) Metzler, R., Klafter, J.: The random walk’s guide to anomalous diffusion: a fractional dynamics approach. Phys. Rep. 339, 1–77 (2000) Metzler, R., Klafter, J.: The restaurant at the end of the random walk: recent developments in the description of anomalous transport by fractional dynamics. J. Phys. A Gen. Phys. 37, R161–R208 (2004) Sato, K.I.: Lévy Processes and Infinitely Divisible Distributions. Cambridge University Press, Cambridge (1999) Wyłomańska, A.: Arithmetic Brownian motion subordinated by tempered stable and inverse tempered stable processes. Phys. A 391(22), 5685–5696 (2012) Wyłomańska, A.: Tempered stable process with infinitely divisible inverse subordinators. J. Stat. Mech. Theory Exp. P10011 (2013) Wyłomańska, A., Gajda, J.: Stable continuous-time autoregressive process driven by stable subordinator. Physica A 444, 1012–1026 (2016) Magdziarz, M., Weron, A.: Fractional Fokker–Planck dynamics: stochastic representation and computer simulation. Phys. Rev. E 75, 056 (2007) Magdziarz, M.: Langevin Picture of Subdiffusion with Infinitely Divisible Waiting Times. J. Stat. Phys. 135, 763–772 (2009) Gajda, J., Magdziarz, M.: Fractional Fokker–Planck equation with tempered alpha-stable waiting times: Langevin picture and computer simulation. Phys. Rev. E 82, 011 (2010) Gajda, J., Wyłomańska, A.: Fokker-Planck type equations associated with fractional Brownian motion controlled by infinitely divisible processes. Physica A 405, 104–113 (2014) Maraj, K., Szarek, D., Sikora, G., Balcerek, M., Wyłomańska, A., Jabłoński, I.: Measurement instrumentation and selected signal processing techniques for anomalous diffusion analysis. Meas. Sens. 7-9, 100017 (2020) Balcerek, M., Burnecki, K.: Testing of fractional Brownian motion in a noisy environment. Chaos Solitons Fractals 140, 110 (2020) Balcerek, M., Burnecki, K., Sikora, G., Wyłomańska, A.: Discriminating gaussian processes via quadratic form statistics. Chaos An Interdiscip. J. Nonlinear Sci. 31(6), 063 (2021) Szarek, D., Sikora, G., Balcerek, M., Jabłoński, I., Wyłomańska, A.: Fractional dynamics identification via intelligent unpacking of the sample autocovariance function by neural networks. Entropy 22(11) (2020) Luiz Roberto Evangelista, E.K.L.: Fractional Diffusion Equations and Anomalous Diffusion, 1 edn. Cambridge University Press, Cambridge, pp. 86–116 (2018) Hassler, U.: Stochastic Processes and Calculus: An Elementary Introduction with Applications. Springer Texts in Business and Economics. Springer (2015) Garra, R., Issoglio, E., Taverna, G.S.: Fractional brownian motions ruled by nonlinear equations. Appl. Math. Lett. 102, 106160 (2020) Nourdin, I.: Selected aspects of fractional Brownian motion. Bocconi & Springer series 4. Springer, pp. 21–31 (2012) Gut, A.: An Intermediate Course in Probability, 2 edn. Springer texts in statistics. Springer-Verlag New York, pp. 117–127 (2009) Mitrinović, D.D.S.: Analytic Inequalities, 1 edn. Die Grundlehren der mathematischen Wissenschaften 165. Springer-Verlag Berlin Heidelberg, p. 109 (1970) George Bachman, L.N.: Functional Analysis, 2nd edn, p. 29. Dover Books on Mathematics. Dover Publications (1998) Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997) Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. Neural Comput. 12, 2451–2471 (1999) Pascanu, R., Mikolov, T., Bengio, Y.: Understanding the exploding gradient problem. arXiv preprint arXiv:1211.5063 (2012) Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford, pp. 116–161, 195–208 (1996) Dekking, F., Kraaikamp, C., Lopuhaä, H., Meester, L.: A modern introduction to probability and statistics: understanding why and how. Springer Texts in Statistics. Springer, pp. 181–194 (2005) Elfwing, S., Uchibe, E., Doya, K.: Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Netw. 107, 3–11 (2018) Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. arXiv preprint arXiv:1710.05941 (2017) Hendrycks, D., Gimpel, K.: Gaussian error linear units (GELUS). arXiv preprint arXiv:1606.08415 (2020) Clevert, D., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus). In: Bengio, Y., LeCun, Y. (eds.) 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2–4, 2016, Conference Track Proceedings (2016) Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings ICML, vol. 30, p. 3 (2013) Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (2014) Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260–270. Association for Computational Linguistics, San Diego, California (2016) Sabour, S., Frosst, N., Hinton, G.: Dynamic routing between capsules. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30, pp. 3856–3866. Curran Associates, Inc. (2017) Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., Brox, T.: Flownet: Learning optical flow with convolutional networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2758–2766 (2015) Mao, X., Li, Q., Xie, H., Lau, R., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2813–2821 (2017) Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2(4), 303–314 (1989) Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge, pp. 224–270 (2016) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017) Hoedt, P.J., Kratzert, F., Klotz, D., Halmich, C., Holzleitner, M., Nearing, G., Hochreiter, S., Klambauer, G.: MC-LSTM: mass-conserving LSTM. In: ICML (2021) Neil, D., Pfeiffer, M., Liu, S.C.: Phased LSTM: accelerating recurrent network training for long or event-based sequences. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, pp. 3889–3897. Curran Associates Inc., Red Hook, NY, USA (2016) Dey, R., Salem, F.M.: Gate-variants of gated recurrent unit (GRU) neural networks. In: 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS), pp. 1597–1600 (2017) Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016) Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers. Association for Computational Linguistics, Minneapolis, Minnesota), pp. 4171–4186 (2019) Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2021) Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S.: GLUE: A multi-task benchmark and analysis platform for natural language understanding. In: Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353–355. Association for Computational Linguistics, Brussels, Belgium (2018) Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2018) Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: Self-attention generative adversarial networks. In: International Conference on Machine Learning, pp. 7354–7363. PMLR (2019) Wu, Y., Lian, D., Gong, N.Z., Yin, L., Yin, M., Zhou, J., Yang, H.: Linear-time self attention with codeword histogram for efficient recommendation. In: Proceedings of the Web Conference 2021, WWW ’21, p. 1262–1273. Association for Computing Machinery, New York, NY, USA (2021) Thapak, P., Hore, P.: Transformer++. arXiv preprint arXiv:2003.04974 (2020) Roy, A., Saffar, M., Vaswani, A., Grangier, D.: Efficient content-based sparse attention with routing transformers. Trans. Assoc. Comput. Linguist. 9, 53–68 (2021) Fedus, W., Zoph, B., Shazeer, N.: Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961 (2021)