WARA-PS: a research arena for public safety demonstrations and autonomous collaborative rescue robotics experimentation
Tóm tắt
A research arena (WARA-PS) for sensing, data fusion, user interaction, planning and control of collaborative autonomous aerial and surface vehicles in public safety applications is presented. The objective is to demonstrate scientific discoveries and to generate new directions for future research on autonomous systems for societal challenges. The enabler is a computational infrastructure with a core system architecture for industrial and academic collaboration. This includes a control and command system together with a framework for planning and executing tasks for unmanned surface vehicles and aerial vehicles. The motivating application for the demonstration is marine search and rescue operations. A state-of-art delegation framework for the mission planning together with three specific applications is also presented. The first one concerns model predictive control for cooperative rendezvous of autonomous unmanned aerial and surface vehicles. The second project is about learning to make safe real-time decisions under uncertainty for autonomous vehicles, and the third one is on robust terrain-aided navigation through sensor fusion and virtual reality tele-operation to support a GPS-free positioning system in marine environments. The research results have been experimentally evaluated and demonstrated to industry and public sector audiences at a marine test facility. It would be most difficult to do experiments on this large scale without the WARA-PS research arena. Furthermore, these demonstrator activities have resulted in effective research dissemination with high public visibility, business impact and new research collaborations between academia and industry.
Tài liệu tham khảo
WASP, WASP Home Page (2021). https://wasp-sweden.org/. [Online; accessed June 2021].
WASP, WARA-PS Arena (2021). https://wasp-sweden.org/research/research-arenas/wara-ps-public-safety/. [Online; accessed June 2021].
Wikipedia, Wikipedia Pubic Safety and Security Home Page (2021). https://en.wikipedia.org/wiki/Public_security. [Online; accessed July 2021].
WASP, WASP Publications (2021). https://wasp-sweden.org/research/publications/f. [Online; accessed July 2021].
SaabNaval, Saab Naval Home Page (2021). https://www.saab.com/products/naval. [Online; accessed July 2021].
SaabAir, Saab Air Home Page (2021). https://www.saab.com/products/air. [Online; accessed July 2021].
Combitech, Combitech Home Page (2021). https://www.combitech.se/. [Online; accessed July 2021].
SaabAB, Saab AB Home Page (2021). https://www.saab.com/. [Online; accessed July 2021].
Axis, Axis Communications Home Page (2021). https://www.axis.com/en-us/. [Online; accessed July 2021].
Skeldar, UMS Skeldar Home Page (2021). https://umsskeldar.aero/. [Online; accessed July 2021].
AIICS, Artificial Intelligence and Integrated Computer Systems Division, IDA, LiU Home Page (2021). https://www.ida.liu.se/divisions/aiics/. [Online; accessed June 2021].
SMaRC, Swedish Maritime Robotics Centre Home Page (2021). https://smarc.se/. [Online; accessed July 2021].
SSRS, SSRS Home Page (2021). https://www.sjoraddning.se/. [Online; accessed July 2021].
M. Dastani, J. -J. C. Meyer, in Proceedings of the AAMAS07 Workshop on Programming Multi-Agent Systems (ProMAS2007). A practical agent programming language, (2007), pp. 72–87.
R. J. Firby, in Proceedings of the National Conference on Artificial Intelligence, 1. An investigation into reactive planning in complex domains (AAAI PressSeattle, 1987), pp. 202–206.
K. Konolige, in KI-97: Advances in Artificial Intelligence, 1303. Colbert: A language for reactive control in Sapphira (SpringerBerlin Heidelberg, 1997), pp. 31–52.
D. C. MacKenzie, R. Arkin, J. M. Cameron, Multiagent mission specification and execution. Auton. Robots. 4(1), 29–52 (1997).
R. Simmons, D. Apfelbaum, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). A task description language for robot control, (1998).
P. Ulam, Y. Endo, A. Wagner, R. C. Arkin, in ICRA. Integrated mission specification and task allocation for robot teams – design and implementation, (2007).
A. Marzinotto, M. Colledanchise, C. Smith, P. Ögren, Towards a unified behavior trees framework for robot control, (2014).
P. Doherty, D. Landén, F. Heintz, in Proceedings of the International Conference on Principles and Practice of Multi-Agent Systems (PRIMA). A distributed task specification language for mixed-initiative delegation, (2010).
P. Doherty, J. Kvarnström, A. Szalas, in Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning (KR). Temporal composite actions with constraints, (2012), pp. 478–488.
P. Doherty, F. Heintz, J. Kvarnström, High-level Mission Specification and Planning for Collaborative Unmanned Aircraft Systems using Delegation. Unmanned Syst.1:, 75–119 (2013).
P. Doherty, J. Kvarnström, M. Wzorek, P. Rudol, F. Heintz, G. Conte, in Handbook of Unmanned Aerial Vehicles, ed. by K. P. Valavanis, G. J. Vachtsevanos. HDRC3 - A Distributed Hybrid Deliberative/Reactive Architecture for Unmanned Aircraft Systems (SpringerNetherlands, 2014), pp. 849–952. This article exists in revised form in the 2nd edition of the handbook 2016.
L. Marconi, C. Melchiorri, M. Beetz, D. Pangercic, R. Siegwart, S. Leutenegger, R. Carloni, S. Stramigioli, H. Bruyninckx, P. Doherty, A. Kleiner, V. Lippiello, A. Finzi, B. Siciliano, A. Sala, N. Tomatis, in Proceedings of the IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). The SHERPA project: Smart collaboration between humans and ground-aerial robots for improving rescuing activities in alpine environments (IEEEUSA, 2012), pp. 1–4. https://doi.org/10.1109/SSRR.2012.6523905.
P. Doherty, P. Rudol, in AI’07: Proceedings of the Australian Joint Conference on Advances in Artificial Intelligence. A UAV search and rescue scenario with human body detection and geolocalization (SpringerBerlin Heidelberg, 2007), pp. 1–13.
P. Doherty, J. Kvarnström, P. Rudol, M. Wzorek, G. Conte, C. Berger, T. Hinzmann, T. Stastny, in PRIMA 2016: Principles and Practice of Multi-Agent Systems, 9862. A Collaborative Framework for 3D Mapping using Unmanned Aerial Vehicles, (2016).
T. Hinzmann, T. Stastny, G. Conte, P. Doherty, P. Rudol, M. Wzorek, I. Gilitschenski, E. Galceran, R. Siegwart, in International Symposium on Robotics. Collaborative 3D reconstruction using heterogeneous unmanned aerial vehicles, (2016).
C. Berger, M. Wzorek, J. Kvarnström, G. Conte, P. Doherty, A. Eriksson, in 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). Area Coverage with Heterogeneous UAVs using Scan Patterns (IEEE Robotics and Automation SocietyUSA, 2016), pp. 342–349.
P. Doherty, J. Kvarnström, F. Heintz, A Temporal Logic-based Planning and Execution Monitoring Framework for Unmanned Aircraft Systems. Auton. Agent. Multi-Agent Syst.19(3), 332–377 (2009). https://doi.org/10.1007/s10458-009-9079-8.
P. Doherty, J. -J. C. Meyer, in The Goals of Cognition: Essays in Honour of Cristiano Castelfranchi, ed. by F. Paglieri, L. Tummolini, R. Falcone, and M. Miceli. On the logic of delegation - relating theory and practice (College Publications LtdLondon, 2012), pp. 467–496.
Foundation for Intelligent Physical Agents, FIPA Communicative Act Library Specification (2002). http://www.fipa.org.
Foundation for Intelligent Physical Agents, FIPA Contract Net Interaction Protocol Specification (2002). http://www.fipa.org.
CORBA, CORBA Home Page (2021). https://www.corba.org/. [Online; accessed May 2021].
ROS, ROS Homepage (2021). http://wiki.ros.org/. [Online; accessed May 2021].
ROS-Industrial, ROS Industrial Home Page (2021). https://rosindustrial.org. [Online; accessed May 2021].
MQTT, http://mqtt.org/. [Online; accessed May 2021] (2021). MQTT Home Page.
WARA-Portal, WARA-PS Research Portal (2021). https://portal.waraps.org/. [Online; accessed June 2021].
Airpelago, Airpelago Home Page (2021). https://www.airpelago.com/. [Online; accessed July 2021].
L. Persson, B. Wahlberg, in AIAA Scitech 2019 Forum. Model predictive control for autonomous ship landing in a search and rescue scenario, (2019). https://doi.org/10.2514/6.2019-1169. https://arc.aiaa.org/doi/abs/10.2514/6.2019-1169.
L. Persson, B. Wahlberg, in 2018 European Control Conference (ECC). Verification of cooperative maneuvers in flightgear using MPC and backwards reachable sets, (2018), pp. 1411–1416. https://doi.org/10.23919/ECC.2018.8550247.
R. Bereza-Jarocinski, L. Persson, B. Wahlberg, Distributed model predictive control for cooperative landing. IFAC-PapersOnLine. 53(2), 15180–15185 (2019). https://doi.org/10.1016/j.ifacol.2020.12.2290.
L. Persson, B. Wahlberg, in 2021 IEEE Aerospace Conference. Variable prediction horizon control for cooperative landing on moving target, (2021).
L. Persson, A. Hansson, B. Wahlberg, in (Submitted to Control Engineering Practice). A computationally fast variable horizon MPC algorithm with application in rendezvous of autonomous unmanned vehicles, (2021).
B. Stellato, G. Banjac, P. Goulart, A. Bemporad, S. Boyd, OSQP: an operator splitting solver for quadratic programs. Math. Program. Comput.12(4), 637–672 (2020). https://doi.org/10.1007/s12532-020-00179-2.
D. P. Bertsekas, Dynamic Programming and Optimal Control. (3rd edn., ed.), vol. I (Athena Scientific, Belmont, 2005).
O. Andersson, Learning to make safe real-time decisions under uncertainty for autonomous robots. PhD thesis (Linköping University, Sweden, 2020). https://doi.org/10.3384/diss.diva-163419.
O. Andersson, M. Wzorek, P. Rudol, P. Doherty, in 2016 IEEE International Conference on Robotics and Automation (ICRA). Model-predictive control with stochastic collision avoidance using bayesian policy optimization, (2016), pp. 4597–4604. https://doi.org/10.1109/ICRA.2016.7487661.
A. Domahidi, A. Zgraggen, M. N. Zeilinger, M. Morari, C. N. Jones, in IEEE Conference on Decision and Control (CDC). Efficient interior point methods for multistage problems arising in receding horizon control (IEEEMaui, 2012), pp. 668–674.
B. Houska, H. J. Ferreau, M. Diehl, ACADO toolkit–An open-source framework for automatic control and dynamic optimization. Optim. Control Appl. Methods. 32(3), 298–312 (2011).
M. Gelbart, J. Snoek, R. Adams, in Proceedings of the Thirtieth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-14). Bayesian optimization with unknown constraints (AUAI PressCorvallis, Oregon, 2014), pp. 250–259.
J. M. Hernández-Lobato, M. A. Gelbart, M. W. Hoffman, R. P. Adams, Z. Ghahramani, in Proceedings of the 32nd International Conference on Machine Learning (ICML). Predictive entropy search for bayesian optimization with unknown constraints, (2015), pp. 1699–1707.
O. Andersson, M. Wzorek, P. Doherty, in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI). Deep Learning Quadcopter Control Via Risk-Aware Active Learning (AAAI PressCalifornia, 2017), pp. 3812–3818.
O. Andersson, O. Ljungqvist, M. Tiger, D. Axehill, F. Heintz, in 2018 IEEE Conference on Decision and Control (CDC). Receding-horizon lattice-based motion planning with dynamic obstacle avoidance (IEEEUSA, 2018), pp. 4467–4474.
O. Andersson, P. Sidén, J. Dahlin, P. Doherty, M. Villani, in Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019. Real-time robotic search using structural spatial point processes (AUAI PressArlington, 2019).
I. Stenius, P. Sigray, G. Linn, Smarc mid-term evauation report. Technical Report TRITA-SCI-RAP-2020.006, KTH Royal Institute of Technology, 54 (2020). https://doi.org/10.30746/TRITA-SCI-RAP-2020:006.
M. Lager, E. A. Topp, J. Malec, in Multisensor Fusion and Integration for Intelligent Systems. Underwater terrain navigation during realistic scenarios (Springer International Publishing, 2017), pp. 186–209. https://doi.org/10.1007/978-3-319-90509-9_11.
M. Lager, E. A. Topp, J. Malec, in 2020 IEEE 23rd International Conference on Information Fusion (FUSION). Robust terrain-aided navigation through sensor fusion (IEEEUSA, 2020), pp. 1–8. https://doi.org/10.23919/fusion45008.2020.9190578.
M. Lager, E. A. Topp, J. Malec, in Proceedings of the 1st International Workshop on Virtual, Augmented, and Mixed Reality for HRI (VAM-HRI). Remote operation of unmanned surface vessel through virtual reality-a low cognitive load approach, (2018). https://doi.org/10.6084/m9.figshare.13537019.
M. Lager, E. A. Topp, J. Malec, VR teleoperation to support a GPS-free positioning system in a marine environment. Int. J. Mar. Navig. Saf. Sea Transp.14(4), 789–798 (2020).
T. Porathe, in Usability Professionals’ Association Conference. User-centered map design, (2007).
Unity, Unity (2021). https://unity.com/. [Online; accessed May 2021].
J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, J. Gall, Semantickitti: A dataset for semantic scene understanding of lidar sequences, (2019).
K. Geng, G. Dong, G. Yin, J. Hu, Deep dual-modal traffic objects instance segmentation method using camera and lidar data for autonomous driving. Remote Sens.12(20), 3274 (2020).
D. Phan, S. Rosenqvist, Semi-Automatic Annotation and Gathering of Marine Images. Student Paper (Lund University, Sweden, 2021). https://lup.lub.lu.se/student-papers/search/publication/9064600.
I. H. Organization, Standards and Specifications (2021). https://iho.int/en/standards-and-specifications.
IALAAISM (International Association of Marine Aids to Navigation and Lighthouse Authorities), IALA maritime buoyage system. https://www.iala-aism.org/.
US department of Defense, Global Positioning System Standard Positioning Service Performance Standard (4th Edition, September 2008). https://www.gps.gov/technical/ps/2008-SPS-performance-standard.pdf.
A. Bochkovskiy, C. -Y. Wang, H. -Y. M. Liao, Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020). https://arxiv.org/abs/2004.10934.
R. Girshick, J. Donahue, T. Darrell, J. Malik, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Rich feature hierarchies for accurate object detection and semantic segmentation, (2014), pp. 580–587.
H. Grabner, C. Leistner, H. Bischof, in European Conference on Computer Vision. Semi-supervised on-line boosting for robust tracking (SpringerBerlin Heidelberg, 2008), pp. 234–247.
B. Babenko, M. -H. Yang, S. Belongie, in 2009 IEEE Conference on Computer Vision and Pattern Recognition. Visual tracking with online multiple instance learning (IEEEUSA, 2009), pp. 983–990.
Y. Li, J. Zhu, in European Conference on Computer Vision. A scale adaptive kernel correlation filter tracker with feature integration (SpringerCham, 2014), pp. 254–265.
Z. Kalal, K. Mikolajczyk, J. Matas, Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell.34(7), 1409–1422 (2011).
Z. Kalal, K. Mikolajczyk, J. Matas, in 2010 20th International Conference on Pattern Recognition. Forward-backward error: Automatic detection of tracking failures (IEEEUSA, 2010), pp. 2756–2759.
D. S. Bolme, J. R. Beveridge, B. A. Draper, Y. M. Lui, in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Visual object tracking using adaptive correlation filters (IEEEUSA, 2010), pp. 2544–2550.
A. Lukezic, T. Vojir, L. Čehovin Zajc, J. Matas, M. Kristan, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Discriminative correlation filter with channel and spatial reliability, (2017), pp. 6309–6318.
Tzutalin, Git code. LabelImg. GitHub (2015). https://github.com/tzutalin/labelImg.
J. A. E. Andersson, J. Gillis, G. Horn, J. B. Rawlings, M. Diehl, CasADi – A software framework for nonlinear optimization and optimal control. Math. Program. Comput.11:, 1–36 (2018).