Optimizing energy efficiency of LoRaWAN-based wireless underground sensor networks: A multi-agent reinforcement learning approach

Akyildiz, 2006, Wireless underground sensor networks: Research challenges, Ad Hoc Netw., 4, 669, 10.1016/j.adhoc.2006.04.003 Zhang, 2017, Thoreau: A subterranean wireless sensing network for agriculture and the environment, 78 Silva, 2015, Experimental link quality characterization of wireless sensor networks for underground monitoring, IEEE Trans. Ind. Inform., 11, 1099, 10.1109/TII.2015.2471263 Lin, 2020, Experimental link quality analysis for lora-based wireless underground sensor networks, IEEE Internet Things J., 8, 6565, 10.1109/JIOT.2020.3044647 Banaseka, 2021, Signal propagation models in soil medium for the study of wireless underground sensor networks: a review of current trends, Wirel. Commun. Mob. Comput., 2021 Wu, 2020, Long range wide area network for agricultural wireless underground sensor networks, J. Ambient Intell. Humaniz. Comput., 1 Lin, 2019, A preliminary study of UG2AG link quality in LoRa-based wireless underground sensor networks, 51 Moiroux-Arvis, 2022, Evaluation of LoRa technology in 433-MHz and 868-MHz for underground to aboveground data transmission, Comput. Electron. Agric., 194, 10.1016/j.compag.2022.106770 Ebi, 2019, Synchronous LoRa mesh network to monitor processes in underground infrastructure, IEEE Access, 7, 57663, 10.1109/ACCESS.2019.2913985 Lin, 2022, Throughput optimization in backscatter-assisted wireless-powered underground sensor networks for smart agriculture, Internet Things, 20, 10.1016/j.iot.2022.100637 Kufakunesu, 2020, A survey on adaptive data rate optimization in LoRaWAN: Recent solutions and major challenges, Sensors, 20, 5044, 10.3390/s20185044 Sornin, 2017 Park, 2020, EARN: Enhanced ADR with coding rate adaptation in LoRaWAN, IEEE Internet Things J., 7, 11873, 10.1109/JIOT.2020.3005881 Sandoval, 2019, Optimizing and updating LoRa communication parameters: A machine learning approach, IEEE Trans. Netw. Serv. Manag., 16, 884, 10.1109/TNSM.2019.2927759 Sandoval, 2020, Deriving and updating optimal transmission configurations for Lora networks, IEEE Access, 8, 38586, 10.1109/ACCESS.2020.2973252 Sutton, 2018 Mammeri, 2019, Reinforcement learning based routing in networks: Review and classification of approaches, IEEE Access, 7, 55916, 10.1109/ACCESS.2019.2913776 Cuomo, 2017, EXPLoRa: Extending the performance of LoRa by suitable spreading factor allocations, 1 Su, 2020, Energy efficient uplink transmissions in LoRa networks, IEEE Trans. Commun., 68, 4960, 10.1109/TCOMM.2020.2993085 Marini, 2020, A novel collision-aware adaptive data rate algorithm for LoRaWAN networks, IEEE Internet Things J., 8, 2670, 10.1109/JIOT.2020.3020189 F. Cuomo, J.C.C. Gámez, A. Maurizio, L. Scipione, M. Campo, A. Caponi, G. Bianchi, G. Rossini, P. Pisani, Towards traffic-oriented spreading factor allocations in LoRaWAN systems, in: 2018 17th Annual Mediterranean Ad Hoc Networking Workshop (Med-Hoc-Net), 2018, pp. 1–8, http://dx.doi.org/10.23919/MedHocNet.2018.8407091. K.Q. Abdelfadeel, V. Cionca, D. Pesch, Fair Adaptive Data Rate Allocation and Power Control in LoRaWAN, in: 2018 IEEE 19th International Symposium on ”a World of Wireless, Mobile and Multimedia Networks” (WoWMoM), 2018, pp. 14–15, http://dx.doi.org/10.1109/WoWMoM.2018.8449737. Slabicki, 2018, Adaptive configuration of LoRa networks for dense IoT deployments, 1 Li, 2018, How agile is the adaptive data rate mechanism of LoRaWAN?, 206 N. Benkahla, H. Tounsi, Y.-Q. Song, M. Frikha, Enhanced ADR for LoRaWAN networks with mobility, in: 2019 15th International Wireless Communications Mobile Computing Conference, IWCMC, 2019, pp. 1–6, http://dx.doi.org/10.1109/IWCMC.2019.8766738. Y. Li, J. Yang, J. Wang, DyLoRa: Towards Energy Efficient Dynamic LoRa Transmission Control, in: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications, 2020, pp. 2312–2320, http://dx.doi.org/10.1109/INFOCOM41043.2020.9155407. D.-T. Ta, K. Khawam, S. Lahoud, C. Adjih, S. Martin, LoRa-MAB: A Flexible Simulator for Decentralized Learning Resource Allocation in IoT Networks, in: 2019 12th IFIP Wireless and Mobile Networking Conference, WMNC, 2019, pp. 55–62, http://dx.doi.org/10.23919/WMNC.2019.8881393. I. Ilahi, M. Usama, M.O. Farooq, M. Umar Janjua, J. Qadir, LoRaDRL: Deep Reinforcement Learning Based Adaptive PHY Layer Transmission Parameters Selection for LoRaWAN, in: 2020 IEEE 45th Conference on Local Computer Networks, LCN, 2020, pp. 457–460, http://dx.doi.org/10.1109/LCN48667.2020.9314772. Y. Yu, L. Mroueh, S. Li, M. Terré, Multi-Agent Q-Learning Algorithm for Dynamic Power and Rate Allocation in LoRa Networks, in: 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, 2020, pp. 1–5, http://dx.doi.org/10.1109/PIMRC48278.2020.9217291. Watkins, 1992, Q-learning, Mach. Learn., 8, 279, 10.1007/BF00992698 Park, 2020, Network resource optimization with reinforcement learning for low power wide area networks, EURASIP J. Wireless Commun. Networking, 2020, 1, 10.1186/s13638-020-01783-5 Mai, 2022, Transfer reinforcement learning aided distributed network slicing optimization in industrial IoT, IEEE Trans. Ind. Inform., 18, 4308, 10.1109/TII.2021.3132136 Ivoghlian, 2022, Adaptive wireless network management with multi-agent reinforcement learning, Sensors, 22, 1019, 10.3390/s22031019 J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, S. Whiteson, Counterfactual multi-agent policy gradients, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, 2018. Vuran, 2010, Channel model and analysis for wireless underground sensor networks in soil medium, Phys. Commun., 3, 245, 10.1016/j.phycom.2010.07.001 King, 2012 M. Bor, J.E. Vidler, U. Roedig, LoRa for the Internet of Things, in: EWSN ’16 Proceedings of the 2016 International Conference on Embedded Wireless Systems and Networks, 2016, pp. 361–366. M.C. Bor, U. Roedig, T. Voigt, J.M. Alonso, Do LoRa low-power wide-area networks scale?, in: Proceedings of the 19th ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, 2016, pp. 59–67. Littman, 1994, Markov games as a framework for multi-agent reinforcement learning, 157 Gupta, 2017, Cooperative multi-agent control using deep reinforcement learning, 66 Mnih, 2015, Human-level control through deep reinforcement learning, Nature, 518, 529, 10.1038/nature14236 Zhang, 1997, Multivariate nonlinear modelling of fluorescence data by neural network with hidden node pruning algorithm, Anal. Chim. Acta, 344, 29, 10.1016/S0003-2670(96)00628-9 K. Mikhaylov, J. Petaejaejaervi, T. Haenninen, Analysis of capacity and scalability of the LoRa low power wide area network technology, in: European Wireless 2016 22th European Wireless Conference, 2016, pp. 1-6. Schaefer, 2007, The USDA natural resources conservation service soil climate analysis network (SCAN), J. Atmos. Ocean. Technol., 24, 2073, 10.1175/2007JTECHA930.1