Intelligent navigation for the cruise phase of solar system boundary exploration based on Q-learning EKF
Complex & Intelligent Systems - Trang 1-20 - 2023
Tóm tắt
With the continuous advancement of deep space exploration missions, the solar system boundary exploration mission is established as one of the China's most important deep space scientific exploration missions. The mission of the solar system boundary exploration has many challenges such as ultra-remote detection distance, ultra-long operation time, and ultra-long communication delay. Therefore, the problem of high-precision autonomous navigation needs to be solved urgently. This paper designs an autonomous intelligent navigation method based on X-ray pulsars in the cruise phase, which estimate the motion state of the probe in real time. The proposed navigation method employs the Q-learning Extended Kalman filter (QLEKF) to improve navigation accuracy during long periods of self-determining running. The QLEKF selects automatically the error covariance matrix parameter of the process noise and the measurement noise by the reward mechanism of reinforcement learning. Compared to the traditional EKF and AEKF, the QLEKF improves the estimation accuracy of position and velocity. Finally, the simulation result demonstrates the effectiveness and the superiority of the intelligent navigation algorithm based on QLEKF, which can satisfy the high-precision navigation requirements in the cruise phase of the solar system boundary exploration.
Tài liệu tham khảo
Dyal P, Fimmel RO (1984) Exploring beyond the planets—the Pioneer 10 and 11 missions. J Br Interplanet Soc 37(10):469–479
Decker RB, Krimigis SM, Roelof EC et al (2005) Voyager 1 in the foreshock, termination shock, and heliosheath. Science 309:2020–2024
Burlaga LF, Ness NF, Acuña MH et al (2009) Observations of the heliosheath and solar wind near the termination shock by Voyager 2. Astrophys J 692(2):1125–1130
Fountain GH, Kusnierkiewicz DY, Hersman CB et al (2008) The new horizons spacecraft. Space Sci Rev 140:23–47
Wu WR, Yu DY, Huang JC et al (2019) Exploring the solar system boundary (in Chinese). Sci Sin Inf 49:1–16
Tian BY, Wang YD, Zhang XY et al (2021) Flight mission planning for solar system boundary exploration. J Astronaut 42:284–294
Song YQ, Wu WR, Hu H et al (2023) Gravity assist space pruning and global optimization of spacecraft trajectories for solar system boundary exploration. Complex Intell Syst 2023:01123–01132
Wang C, Li H, Guo XC et al (2020) Scientific objectives for the exploration of the boundary of solar system. J Deep Space Explor 7(6):517–524, 535
Ma X, Ning XL, Chen X et al (2019) Geometric coplanar constraints-aided autonomous celestial navigation for spacecraft in deep space exploration. IEEE Access 7:112424–112434
Xia Y, Li J, Zhai R et al (2021) Application prospect of fission-powered spacecraft in solar system exploration missions. Space Sci Technol 2021:5245136
Wang YS, Wang YD, Zheng W et al (2022) X-ray pulsar-based navigation scheme for solar system boundary exploration. J Phys Conf Ser 2224(1):012127
Wu WR, Li HT, Li Z et al (2020) Status and prospect of China’s deep space TT&C network. Sci Sin Inf 50(1):93–114
Ma X, Fang JC, Ning XL (2013) An overview of the autonomous navigation for a gravity-assist interplanetary spacecraft. Prog Aerosp Sci 63:56–66
Zheng W, Wang YD (2020) X-ray pulsar-based navigation: theory and applications, Chap. 1. Springer, Singapore, pp 1–3
Reichley P, Downs G, Morris G (1971) Use of pulsar signals as clocks. NASA Jet Propuls Lab Q Tech Rev 1(2):80–86
Chester TJ, Butman SA (1981) Navigation using X-ray pulsars. NASA Technical Report 22-25
Ray Paul S, Sheikh Suneel I, Graven Paul H et al (2008) Deep space navigation using celestial X-ray sources. In: Proceedings of the 2008 national technical meeting of The Institute of Navigation, San Diego, CA, pp 101–109
Xu Q, Fan XH, Zhao AG et al (2021) Pre-correction X-ray pulsar navigation algorithm based on asynchronous overlapping observation method. Adv Space Res 67(1):583–596
Wang YQ, Zheng W (2016) Pulse phase estimation of X-ray pulsar with the aid of vehicle orbital dynamics. J Navig 69(2):414–432
Ning X, Yang YQ, Gui MZ et al (2017) Pulsar navigation using time of arrival (TOA) and time differential TOA (TDTOA). Acta Astronaut 142(1):57–63
Yang B, Liu LK (2012) Deep space navigation using X-ray pulsars. Adv Mater Res 433–440:6325–6331
Wei EH, Jin SG, Zhang Q et al (2013) Autonomous navigation of Mars probe using X-ray pulsars: modeling and results. Adv Space Res 51(5):849–857
Feng DZ, Guo HH, Xin W et al (2014) Autonomous orbit determination and its error analysis for deep space using X-ray pulsar. Aerosp Sci Technol 32(1):35–41
Runnels JT, Gebre-Egziabher D (2021) Estimator for deep-space position and attitude using X-ray pulsars. IEEE Trans Aerosp Electron Syst 57(4):2149–2166
Zhang HH, Li J, Wang ZG et al (2021) Guidance navigation and control for Chang’E-5 powered descent. Space Sci Technol 2021:9823609
Huang XY, Li MD, Wang XL et al (2021) The Tianwen-1 guidance, navigation, and control for mars entry, descent, and landing. Space Sci Technol 2021:9846185
Jiang YG, Huang Y, Xue WC et al (2017) On designing consistent extended Kalman filter. J Syst Sci Complex 30(4):751–764
Yang BJ, Huang H, Gao L (2022) Centered error entropy-based sigma-point Kalman filter for spacecraft state estimation with non-Gaussian noise. Space Sci Technol 2022:9854601
Song QP, Liu RK (2015) Weighted adaptive filtering algorithm for carrier tracking of deep space signal. Chin J Aeronaut 28(4):1236–1244
Fu K, Zhang D, Tang P et al (2015) Adaptive extended Kalman filter for a red shift navigation system. In: 2015 34th Chinese control conference (CCC). IEEE, pp 5194–5199
Xiong K, Wei CL, Zhang HY (2021) Q-learning for noise covariance adaptation in extended KALMAN filter. Asian J Control 23(4):1803–1816
Gosavi A (2009) Reinforcement learning: a tutorial survey and recent advances. INFORMS J Comput 21(2):178–192
Kim D, Lee T, Kim S et al (2018) Adaptive packet scheduling in IoT environment based on Q-learning. Procedia Comput Sci 141(2018):247–254
Dai X, Nateghi V, Fourati H et al (2022) Q-learning-based noise covariance adaptation in Kalman filter for MARG sensors attitude estimation. In: 2022 IEEE international symposium on inertial sensors and systems (INERTIAL). IEEE, 2022
Xiong K, Wei CW (2020) Integrated celestial navigation for spacecraft using interferometer and earth sensor. Proc Inst Mech Eng Part G J Aerosp Eng 234(16):2248–2262
Xiong K, Wei CL, Zhou P (2022) Integrated autonomous optical navigation using Q-Learning extended Kalman filter. Aircr Eng Aerosp Technol 94(6):848–861
Xiong K, Zhou P, Wei CL (2022) Autonomous navigation of unmanned aircraft using space target LOS measurements and QLEKF. Sensors 22(18):6992
Liu J, Fang JC, Kang ZW et al (2015) Novel algorithm for X-ray pulsar navigation against Doppler effects. IEEE Trans Aerosp Electron Syst 51(1):228–241
Emadzadeh AA, Speyer JL (2010) On modeling and pulse phase estimation of X-ray pulsars. IEEE Trans Signal Process 58(9):4484–4495
Shen LR, Sun HF, Li XP et al (2019) A robust filtering method for X-ray pulsar navigation in the situation of strong noises and large state model errors. IEEE Access 7:161141–161151
Sheikh SI, Pines DJ, Ray PS et al (2006) Spacecraft navigation using X-ray pulsars. J Guid Control Dyn 29(1):49–63
Ma PB, Wang TS, Jiang FH et al (2017) Autonomous navigation of Mars probes by single X-ray pulsar measurement and optical data of viewing Martian moons. J Navig 70(1):18–32
Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8:279–292
Zhao FQ, Wang QY, Wang L (2023) An inverse reinforcement learning framework with the Q-learning mechanism for the metaheuristic algorithm. Knowl Based Syst 265:110368
Dai X, Fourati H, Prieur C (2022) A dynamic grid-based Q-learning for noise covariance adaptation in EKF and its application in navigation. In: 2022 IEEE 61st conference on decision and control (CDC), pp 4984–4989
Ren XB, Nie GG, Li LY (2020) An improved augmented algorithm for direction error in XPNAV. Symmetry 12(7):1059