Optimal fractional-order PID controller based on fractional-order actor-critic algorithm

Aguilar CZ, Gómez-Aguilar J, Alvarado-Martínez V, Romero-Ugalde H (2020) Fractional order neural networks for system identification. Chaos, Solitons & Fractals 130:109444 Akbarimajd A (2015) Reinforcement learning adaptive pid controller for an under-actuated robot arm. Int J Integrated Eng 7(2) Al-Saggaf UM, Mehedi IM, Mansouri R, Bettayeb M (2017) Rotary flexible joint control by fractional order controllers. Int J Control Autom Syst 15(6):2561–2569 Arya Y (2020) A novel cffopi-fopid controller for agc performance enhancement of single and multi-area electric power systems. ISA Trans 100:126–135 Badri V, Tavazoei MS (2013) On tuning fractional order [proportional-derivative] controllers for a class of fractional order systems. Automatica 49(7):2297–2301 Bai W, Li T, Tong S (2020) Nn reinforcement learning adaptive control for a class of nonstrict-feedback discrete-time systems. IEEE Trans Cybernet 50(11):4573–4584 Barth JM, Condomines JP, Bronz M, Lustosa LR, Moschetta JM, Join C, Fliess M (2018) Fixed-wing uav with transitioning flight capabilities: model-based or model-free control approach? a preliminary study. In: 2018 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1157–1164. IEEE Boubertakh H, Tadjine M, Glorennec PY, Labiod S (2010) Tuning fuzzy pd and pi controllers using reinforcement learning. ISA Trans 49(4):543–551 Busoniu L, Babuska R, De Schutter B, Ernst D (2017) Reinforcement learning and dynamic programming using function approximators. CRC press, US Carlucho I, De Paula M, Acosta GG (2020) An adaptive deep reinforcement learning approach for mimo pid control of mobile robots. ISA Trans 102:280–294 Chen M, Lam HK, Shi Q, Xiao B (2019) Reinforcement learning-based control of nonlinear systems using lyapunov stability concept and fuzzy reward scheme. IEEE Trans Circuits Syst II Exp Briefs 67(10):2059–2063 Dwivedi P, Pandey S, Junghare A (2017) Performance analysis and experimental validation of 2-dof fractional-order controller for underactuated rotary inverted pendulum. Arab J Sci Eng 42(12):5121–5145 Fei J, Wang Z (2020) Multi-loop recurrent neural network fractional-order terminal sliding mode control of mems gyroscope. IEEE Access 8:167965–167974 Fu X, Li S, Fairbank M, Wunsch DC, Alonso E (2014) Training recurrent neural networks with the levenberg-marquardt algorithm for optimal control of a grid-connected converter. IEEE Trans Neural Netw Learn Syst 26(9):1900–1912 George MA, Kamath DV (2020) Design and tuning of fractional order pid (fopid) controller for speed control of electric vehicle on concrete roads. In: 2020 IEEE International Conference on Power Electronics, Smart Grid and Renewable Energy (PESGRE2020), pp. 1–6. IEEE Gheisarnejad M, Khooban MH (2020) An intelligent non-integer pid controller-based deep reinforcement learning: Implementation and experimental results. IEEE Trans Industr Electron 68(4):3609–3618 Görges D (2017) Relations between model predictive control and reinforcement learning. IFAC-PapersOnLine 50(1):4920–4928 Hassan RO, Mostafa H (2021) Implementation of deep neural networks on fpga-cpu platform using xilinx sdsoc. Analog Integr Circ Sig Process 106(2):399–408 Hou Z, Xiong S (2019) On model-free adaptive control and its stability analysis. IEEE Trans Autom Control 64(11):4555–4569 Huang L, Deng L, Li A, Gao R, Zhang L, Lei W (2021) A novel approach for solar greenhouse air temperature and heating load prediction based on laplace transform. J Build Eng 44:102682 Huang X, Guo Z, Song M, Zeng X (2021) Accelerating the sm3 hash algorithm with cpu-fpga co-designed architecture. IET Comput Dig Tech 15(6):427–436 Huang X, Naghdy F, Du H, Naghdy G, Todd C (2015) Reinforcement learning neural network (rlnn) based adaptive control of fine hand motion rehabilitation robot. In: 2015 IEEE Conference on Control Applications (CCA), pp. 941–946. IEEE Ibraheem GAR, Azar AT, Ibraheem IK, Humaidi AJ (2020) A novel design of a neural network-based fractional pid controller for mobile robots using hybridized fruit fly and particle swarm optimization. Complexity 2020 Jang B, Kim M, Harerimana G, Kim JW (2019) Q-learning algorithms: a comprehensive classification and applications. IEEE Access 7:133653–133667 Johnson T, Zhou S, Cheah W, Mansell W, Young R, Watson S (2020) Implementation of a perceptual controller for an inverted pendulum robot. J Intell Robot Syst 99(3):683–692 Karakoyun M, Ozkis A, Kodaz H (2020) A new algorithm based on gray wolf optimizer and shuffled frog leaping algorithm to solve the multi-objective optimization problems. Appl Soft Comput 96:106560 Khater AA, El-Nagar AM, El-Bardini M, El-Rabaie N (2019) A novel structure of actor-critic learning based on an interval type-2 tsk fuzzy neural network. IEEE Trans Fuzzy Syst 28(11):3047–3061 Kokologiannaki CG (2010) Properties and inequalities of generalized k-gamma, beta and zeta functions. Int J Contemp Math Sci 5(14):653–660 Kumar P, Hati AS (2021) Deep convolutional neural network based on adaptive gradient optimizer for fault detection in scim. ISA Trans 111:350–359 Kumar R, Sinha N (2021) Voltage stability of solar dish-stirling based autonomous dc microgrid using grey wolf optimised fopid-controller. Int J Sustain Energ 40(5):412–429 Lawrence NP, Forbes MG, Loewen PD, McClement DG, Backström JU, Gopaluni RB (2022) Deep reinforcement learning with shallow controllers: An experimental application to pid tuning. Control Eng Pract 121:105046 Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105 Liu JG, Yang XJ, Feng YY, Cui P (2020) New fractional derivative with sigmoid function as the kernel and its models. Chin J Phys 68:533–541 Liu TK, Juang JG (2009) A single neuron pid control for twin rotor mimo system. In: 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 186–191. IEEE Ma J, Coogler K, Suh M (2020) Inquiry-based learning: development of an introductory manufacturing processes course based on a mobile inverted pendulum robot. Int J Mech Eng Educ 48(4):371–390 Mahmoud TA, Abdo MI, Elsheikh EA, Elshenawy LM (2021) Direct adaptive control for nonlinear systems using a tsk fuzzy echo state network based on fractional-order learning algorithm. J Franklin Inst 358(17):9034–9060 Matouk AE, Elsadany A, Ahmed E, Agiza H (2015) Dynamical behavior of fractional-order hastings-powell food chain model and its discretization. Commun Nonlinear Sci Numer Simul 27(1–3):153–167 Mbihi MD, Moffo BL, Nneme LN (2021) Design and virtual simulation of an optimal pid/lqrt-pso control system for 2wd mobile robots. algerian journal of signals and systems (ajss) 6(2):98–111 Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61 Mohammadzadeh A, Sabzalian MH, Zhang W (2019) An interval type-3 fuzzy system and a new online fractional-order learning algorithm: theory and practice. IEEE Trans Fuzzy Syst 28(9):1940–1950 Mokhtari SA (2022) Fopid control of quadrotor based on neural networks optimization and path planning through machine learning and pso algorithm. Int J Aeronaut Space Sci pp. 1–16 Monje CA, Chen Y, Vinagre BM, Xue D, Feliu-Batlle V (2010) Fractional-order systems and controls: fundamentals and applications. Springer Science & Business Media, Berlin Mughees A, Mohsin SA (2020) Design and control of magnetic levitation system by optimizing fractional order pid controller using ant colony optimization algorithm. IEEE Access 8:116704–116723 Munagala VK, Jatoth RK (2021) Design of fractional-order pid/pid controller for speed control of dc motor using harris hawks optimization. In: Intelligent Algorithms for Analysis and Control of Dynamical Systems, pp. 103–113. Springer Norsahperi N, Danapalasingam K (2020) Particle swarm-based and neuro-based fopid controllers for a twin rotor system with improved tracking performance and energy reduction. ISA Trans 102:230–244 Okazaki R, Tabata T, Sakashita S, Kitamura K, Takagi N, Sakata H, Ishibashi T, Nakamura T, Ajima Y (2020) Supercomputer fugaku cpu a64fx realizing high performance, high-density packaging, and low power consumption. Fujitsu Technical Review pp. 2020–03 Oostwal E, Straat M, Biehl M (2021) Hidden unit specialization in layered neural networks: Relu vs. sigmoidal activation. Physica A 564:125517 Patel R, Deb D, Modi H, Shah S (2017) Adaptive backstepping control scheme with integral action for quanser 2-dof helicopter. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 571–577. IEEE Petráš I (2011) Fractional-order nonlinear systems: modeling, analysis and simulation. Springer Science & Business Media, Berlin Pezol NS, Rahiman MHF, Adnan R, Tajjudin M (2021) Comparison of the crone-1 and fopid controllers for steam temperature control of the essential oil extraction process. In: 2021 IEEE International Conference on Automatic Control & Intelligent Systems (I2CACIS), pp. 253–258. IEEE Pirasteh-Moghadam M, Saryazdi MG, Loghman E, Kamali A, Bakhtiari-Nejad F (2020) Development of neural fractional order pid controller with emulator. ISA Trans 106:293–302 Podlubny I (1998) Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications. Elsevier Rais MC, Dekhandji FZ, Recioui A, Rechid MS, Djedi L (2022) Comparative study of optimization techniques based pid tuning for automatic voltage regulator system. Engineering Proceedings 14(1):21 Sedighizadeh M, Rezazadeh A (2008) Adaptive pid controller based on reinforcement learning for wind turbine control. In: Proceedings of world academy of science, engineering and technology, vol. 27, pp. 257–262. Citeseer Shah P, Agashe S (2016) Review of fractional pid controller. Mechatronics 38:29–41 Shalaby R, El-Hossainy M, Abo-Zalam B (2019) Fractional order modeling and control for under-actuated inverted pendulum. Commun Nonlinear Sci Numer Simul 74:97–121 Sharma R, Pfeiffer CF (2017) Comparison of control strategies for a 2 dof helicopter. In: Proceedings of the 58th Conference on Simulation and Modelling (SIMS 58), pp. 271–279 Shi JZ (2020) A fractional order general type-2 fuzzy pid controller design algorithm. IEEE Access 8:52151–52172 Singh AP, Deb D, Agarwal H (2019) On selection of improved fractional model and control of different systems with experimental validation. Commun Nonlinear Sci Numer Simul 79:104902 Song D, Han J, Liu G (2012) Active model-based predictive control and experimental investigation on unmanned helicopters in full flight envelope. IEEE Trans Control Syst Technol 21(4):1502–1509 Song R, Xiao W, Zhang H, Sun C (2014) Adaptive dynamic programming for a class of complex-valued nonlinear systems. IEEE Trans Neural Netw Learn Syst 25(9):1733–1739 Su X, Xia F, Liu J, Wu L (2018) Event-triggered fuzzy control of nonlinear systems with its application to inverted pendulum systems. Automatica 94:236–248 Sun Q, Du C, Duan Y, Ren H, Li H (2019) Design and application of adaptive pid controller based on asynchronous advantage actor–critic learning method. Wireless Networks pp. 1–11 Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT press, USA Tepljakov A (2017) Fractional-order modeling and control of dynamic systems. Springer, Berlin Tepljakov A, Alagoz BB, Yeroglu C, Gonzalez E, HosseinNia SH, Petlenkov E (2018) Fopid controllers and their industrial applications: a survey of recent results. IFAC-PapersOnLine 51(4):25–30 Tepljakov A, Alagoz BB, Yeroglu C, Gonzalez EA, Hosseinnia SH, Petlenkov E, Ates A, Cech M (2021) Towards industrialization of fopid controllers: a survey on milestones of fractional-order control and pathways for future developments. IEEE Access 9:21016–21042 Tripathi S, Shrivastava A, Jana KC (2020) Self-tuning fuzzy controller for sun-tracker system using gray wolf optimization (gwo) technique. ISA Trans 101:50–59 Wang J, Wen Y, Gou Y, Ye Z, Chen H (2017) Fractional-order gradient descent learning of bp neural networks with caputo derivative. Neural Netw 89:19–30 Wang J, Yang G, Zhang B, Sun Z, Liu Y, Wang J (2017) Convergence analysis of caputo-type fractional order complex-valued neural networks. IEEE Access 5:14560–14571 Wang S, Diao R, Xu C, Shi D, Wang Z (2020) On multi-event co-calibration of dynamic model parameters using soft actor-critic. IEEE Transactions on Power Systems Xiao B, Lam HK, Yu Y, Li Y (2019) Sampled-data output-feedback tracking control for interval type-2 polynomial fuzzy systems. IEEE Trans Fuzzy Syst 28(3):424–433 Xie Y, Tang X, Song B, Zhou X, Guo Y (2019) Model-free tuning strategy of fractional-order pi controller for speed regulation of permanent magnet synchronous motor. Trans Inst Meas Control 41(1):23–35 Xiong Y, Guo L, Huang Y, Chen L (2020) Intelligent thermal control strategy based on reinforcement learning for space telescope. J Thermophys Heat Trans 34(1):37–44 Yakoub Z, Amairi M, Chetoui M, Saidi B, Aoun M (2017) Model-free adaptive fractional order control of stable linear time-varying systems. ISA Trans 67:193–207 Yang LH, Huang BH, Hsu CY, Chen SL (2019) Performance analysis of an earth-air heat exchanger integrated into an agricultural irrigation system for a greenhouse environmental temperature-control system. Energy and Build 202:109381 Yao X, Wang Z, Huang Z (2021) A stability criterion for discrete-time fractional-order echo state network and its application. Soft Comput 25(6):4823–4831 Yeroğlu C, Ateş A (2014) A stochastic multi-parameters divergence method for online auto-tuning of fractional order pid controllers. J Franklin Inst 351(5):2411–2429 Zennir Y, Mechhoud EA, Seboui A, Bendib R (2018) Optimal pso-pi\(\lambda\)d\(\mu\) multi-controller for a robotic wrist. Algerian J Sig Syst 3(1):22–34