Things in the air: tagging wearable IoT information on drone videos
Tóm tắt
Drones have been applied to a wide range of security and surveillance applications recently. With drones, Internet of Things are extending to 3D space. An interesting question is: Can we conduct person identification (PID) in a drone view? Traditional PID technologies such as RFID and fingerprint/iris/face recognition have their limitations or require close contact to specific devices. Hence, these traditional technologies can not be easily deployed to drones due to dynamic change of view angle and height. In this work, we demonstrate how to retrieve IoT data from users’ wearables and correctly tag them on the human objects captured by a drone camera to identify and track ground human objects. First, we retrieve human objects from videos and conduct coordination transformation to handle the change of drone positions. Second, a fusion algorithm is applied to measure the correlation of video data and inertial data based on the extracted human motion features. Finally, we can couple human objects with their wearable IoT devices, achieving our goal of tagging wearable device data (such as personal profiles) on human objects in a drone view. Our experimental evaluation shows a recognition rate of 99.5% for varying walking paths, and 98.6% when the drone’s camera angle is within 37°. To the best of our knowledge, this is the first work integrating videos from drone cameras and IoT data from inertial sensors.
Tài liệu tham khảo
Van LD, Chang CH, Tong KL, Wu KR, Zhang LY, Tseng YC. Demo: tagging IOT data in a drone view. In: Proceedings of the international conference on mobil computing and networking; 2019. p. 1–3.
Perera T, Priyankara A, Jayasinghe G. Unmanned arial vehicles (UAV) in smart agriculture: trends, benefits and future perspectives. In: Proceedings of the international research conference of Uva Wellassa University
Roberge V, Tarbouchi M, Labonté G. Fast genetic algorithm path planner for fixed-wing military UAV using GPU. IEEE Trans Aerospace Electr Syst. 2018;54(5):2105–17.
Qu T, Zang W, Peng Z, Liu J, Loh P. Construction site monitoring using uav oblique photogrammeter and bim technologies. In: Proceedings of the 22nd association for computer-aided architectural design research in asia conference; 2017. p. 654–62.
Manfreda S, McCabe M, Miller P, Lucas R, Pajuelo Madrigal V, Mallinis G, Ben Dor E, Helman D, Estes L, Ciraolo G. On the use of unmanned aerial systems for environmental monitoring. Remote Sens. 2018;641:1–28.
Nath RK, Bajpai R, Thapliyal H. IoT based indoor location detection system for smart home environment. In: Proceedings of the IEEE international conference on consumer electronics; 2018. p. 1–3.
Hassanalieragh M, Page A, Soyata T, Sharma G, Aktas M, Mateos G, Kantarci B, Andreescu S. Health monitoring and management using internet-of-things (IoT) sensing with cloud-based processing: Opportunities and challenges. In: Proceedings of the IEEE international conference on services computing; 2015. p. 285–92.
Paul A, Ahmad A, Rathore MM, Jabbar S. Smartbuddy: defining human behaviors using big data analytics in social internet of things. IEEE Wireless Commun. 2016;23(5):68–74.
Davis N, Pittaluga F, Panetta K. Facial recognition using human visual system algorithms for robotic and uav platforms. In: Proceedings of the IEEE conference on technologies for practical robot applications; 2013. p. 1–5.
Fysh M, Bindemann M. Person identification from drones by humans: insights from cognitive psychology. Drones. 2018;2(4):1–11.
Liu F, Zhang D, Shen L. Study on novel curvature features for 3D fingerprint recognition. Neurocomputing. 2015;168:599–608.
Chen J, Shen F, Chen DZ, Flynn PJ. Iris recognition based on human-interpretable features. IEEE Trans Inf For Secur. 2016;11(7):1476–85.
Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 2014;47(6):2280–92.
Chen B, Chen C, Wang J. Smart homecare surveillance system: behavior identification based on state-transition support vector machines and sound directivity pattern analysis. IEEE Trans Syst Man Cybernetics Syst. 2013;43(6):1279–89.
Qi M, Han J, Jiang J, Liu H, et al. Deep feature representation and multiple metric ensembles for person re-identification in security surveillance system. Multimedia Tools Appl. 2019;78:1–15.
Hsu SC, Wang YW, Huang CL. Human object identification for human-robot interaction by using fast r-cnn. In: Proceedings of the IEEE international conference on robotic computing; 2018. p. 201–4.
Wang T, Gong S, Zhu X, Wang S. Person re-identification by discriminative selection in video ranking. IEEE Trans Pattern Anal Mach Intell. 2016;38(12):2501–14.
Tang S, Andriluka M, Andres B, Schiele B. Multiple people tracking by lifted multicut and person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 3701–10.
Gong XW, Zhu SG. Person re-identification based on two-stream network with attention and pose features. IEEE Access. 2019;7(99):374–82.
Din C, Ta D. Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell. 2018;40(4):1002–144.
Ge S, Zhao S, Li C, Li J. Low-resolution face recognition in the wild via selective knowledge distillation. IEEE Trans Image Process. 2019;28(4):2051–62.
Parkhi OM, Vedaldi A, Zisserman A. Deep face recognition. In: Proceedings of the British machine vision conference; 2015. p. 1–12.
Hsu HJ, Chen KT. Face recognition on drones: Issues and limitations. In: Proceedings of the workshop on micro aerial vehicle networks, systems, and applications for civilian use; 2015. p. 39–44.
face++. https://www.faceplusplus.com/.
Rekognition. https://aws.amazon.com/tw/rekognition/.
OpenCV. https://opencv.org/https://opencv.org/.
Das R, Piciucco E, Maiorana E, Campisi P. Convolutional neural network for finger-vein-based biometric identification. IEEE Trans Inf For Secur. 2019;14(2):360–73.
O’Toole AJ, Phillips PJ, Weimer S, Roark DA, Ayyad J, Barwick R, Dunlop J. Recognizing people from dynamic and static faces and bodies: dissecting identity with a fusion approach. Vision Res. 2011;51(1):74–83.
Henschel R, Marcard TV, Rosenhahn B. Simultaneous identification and tracking of multiple people using video and imus. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops; 2020. p. 780–9.
Masullo A, Burghardt T, Damen D, Perrett T, Mirmehdi M. Who goes there? exploiting silhouettes and wearable signals for subject identification in multi-person environments. In: Proceedings IEEE/CVF international conference on computer vision workshop; 2019. p. 1599–607.
Masullo A, Burghardt T, Damen D, Perrett T, Mirmehdi M. Person re-id by fusion of video silhouettes and wearable signals for home monitoring applications. Sensors. 2020;20(9):2576–95.
Korany B, Karanam CR, Cai H, Mostofi Y. Xmodal-id: using wifi for through-wall person identification from candidate video footage. In: Proceedings of the international conference on mobile computing and networking; 2019. p. 1–15.
Naegeli T, Oberholzer S, Pluess S, Alonso-Mora J, Hilliges O. Flycon: real-time environment-independent multi-view human pose estimation with aerial vehicles. ACM Trans Graph. 2018;37(6):182.1–182.14.
Zhu P, Wen L, Bian X, Ling H, Hu Q. Vision meets drones: a challenge. Comput Res Reposit; 2018. p. 1–11.
Tzelepi M, Tefas A. Graph embedded convolutional neural networks in human crowd detection for drone flight safety. IEEE Trans Emerg Topics Comput Intell. 2019;99:1–14.
Layne R, Hospedales TM, Gong S. Investigating open-world person re-identification using a drone. In: European conference on computer vision; 2015. p. 225–40.
Portmann J, Lynen S, Chli M, Siegwart R. People detection and tracking from aerial thermal views. In: Proceedings IEEE international conference on robotics and automation; 2014. p. 1794–800.
Fradi H, Bracco L, Canino F, Dugelay JL. Autonomous person detection and tracking framework using unmanned aerial vehicles (uavs). In: Proceedings European signal processing conference; 2018. p. 1047–51.
Shen Q, Jiang L, Xiong H. Person tracking and frontal face capture with uav. In: Proceedings of the international conference on communication technology; 2018. p. 1412–6.
Nguyen HD, Na IS, Kim SH, Lee GS, Yang HJ, Choi JH. Multiple human tracking in drone image. Multimedia Tools Appl. 2019;78(4):4563–77.
Redmon J, Farhadi A. Yolov3: an incremental improvement; 2018. CoRR. abs/1804.02767. arxiv:1804.02767
Bewley A, Ge Z, Ott L, Ramos F, Upcroft B. Simple online and realtime tracking. In: Proceedings of the IEEE international conference on image processing; 2016. p. 3464–8.
Lowe DG et al. Object recognition from local scale-invariant features. In: Proceedings of the IEEE international conference on computer vision. 1999; 2:1150–7.
Berndt DJ, Clifford J. Using dynamic time warping to find patterns in time series. In: Proceedings KDD workshop; 1994. p. 359–70.
Bernardin K, Elbs A, Stiefelhagen R. Multiple object tracking performance metrics and evaluation in a smart room environment. In: Proceedings IEEE international workshop on visual surveillance, in conjunction with ECCV; 2006. p. 1–8.
Teixeira T, Jung D, Savvides A. Tasking networked cctv cameras and mobile phones to identify and localize multiple people. In: Proceedings of the ACM international conference on ubiquitous computing; 2010. p. 213–22.