Multi-sensor 3D volumetric reconstruction using CUDA
Tóm tắt
This paper presents a full-body volumetric reconstruction of a person in a scene using a sensor network, where some of them can be mobile. The sensor network is comprised of couples of camera and inertial sensor (IS). Taking advantage of IS, the 3D reconstruction is performed using no planar ground assumption. Moreover, IS in each couple is used to define a virtual camera whose image plane is horizontal and aligned with the earth cardinal directions. The IS is furthermore used to define a set of inertial planes in the scene. The image plane of each virtual camera is projected onto this set of parallel-horizontal inertial-planes, using some adapted homography functions. A parallel processing architecture is proposed in order to perform human real-time volumetric reconstruction. The real-time characteristic is obtained by implementing the reconstruction algorithm on a graphics processing unit (GPU) using Compute Unified Device Architecture (CUDA). In order to show the effectiveness of the proposed algorithm, a variety of the gestures of a person acting in the scene is reconstructed and demonstrated. Some analyses have been carried out to measure the performance of the algorithm in terms of processing time. The proposed framework has potential to be used by different applications such as smart-room, human behavior analysis and 3D teleconference.
Tài liệu tham khảo
H. Aliakbarpour J. Dias (2010) IMU-aided 3D Reconstruction based on Multiple Virtual Planes, DICTA'10 (the Australian Pattern Recognition and Computer Vision Society Conference), IEEE Pr, 1–3 December, Sydney, Australia.
S. M. Khan, P. Yan, M. Shah (2007) A Homographic Framework for the Fusion of Multi-view Silhouettes, Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference.
B. Michoud, E. Guillou, S. Bouakaz (2007) Real-Time and Markerless 3D Human Motion Capture Using Multiple Views, Human Motion-Understanding, Modeling, Capture and Animation, Springer Berlin/Heidelberg, 4814/2007:88–103
Q. Zhang, H. Wang, S. Wei (2003) A NEW ALGORITHM FOR 3D PROJECTIVE RECONSTRUCTION BASED ON INFINITE HOMOGRAPHY. Machine Learning and Cybernetics, 2003 International Conference on, IEEE.
Z. Zhang, A. R. Hanson (1996) 3D Reconstruction Based on Homography Mapping, In ARPA Image Understanding Workshop.
M. Sormann, C. Zach, J. Bauer, K. Karner, H. Bishof (2007) Watertight Multi-view Reconstruction Based on Volumetric Graph-Cuts. In Ersball, Bjarne and Pedersen, Kim, editors, Image Analysis in Lecture Notes in Computer Science, Springer Berlin, Heidelberg, 393–402.
P. Lai, A. Yilmaz (2008) PROJECTIVE RECONSTRUCTION OF BUILDING SHAPE FROM SILHOUETTE IMAGES ACQUIRED FROM UNCALIBRATED CAMERAS. ISPRS Congress Beijing 2008, Proceedings of Commission III.
T. Feldmann, I. Mihailidis, S. Schulz, D. Paulus, A. Worner (2010) Online Full Body Human Motion Tracking Based on Dense Volumetric 3DÂ Reconstructions from Multi Camera Setups. In Dillmann, Rudiger and Beyerer, Jurgen and Hanebeck, Uwe and Schultz, Tanja, editors, KI 2010, Advances in Artificial Intelligence in Lecture Notes in Computer Science, Springer Berlin/Heidelberg, 74–81.
R. Guerchouche, O. Bernier, T. Zaharia (2008) Multiresolution volumetric 3D object reconstruction for collaborative interactions. Pattern Recognition and Image Analysis, 18:621–637. 10.1134/S1054661808040147.
T. Azevedo, J. Tavares, M. Vaz (2009) 3D Object Reconstruction from Uncalibrated Images Using an Off-the-Shelf Camera. Advances in Computational Vision and Medical Image Processing; in series of Computational Methods in Applied Sciences, Springer Netherlands, 13:117–136, Universidade do Porto.
H. Lee, A. Yilmaz (2010) 3D RECONSTRUCTION USING PHOTO CONSISTENCY FROM UNCALIBRATED MULTIPLE VIEWS. VISAPP 2010 — The International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
B. Zhang, Y.F. Li (2005) An efficient method for dynamic calibration and 3D reconstruction using homographic transformation. Sensors and Actuators A: Physical, 119(2):349–357
H. Lin, J. Wu (2008) 3D Reconstruction by Combining Shape from Silhouette with Stereo. IEEE.
B. Michoud, S. Bouakaz, E. Guillou. H. Briceno (2008) Largest Silhouette-Equivalent Volume for 3D Shapes Modeling without Ghost Object. M2SFA2 2008: Workshop on Multi-camera and Multi-modal Sensor Fusion, Marseille, France.
H. Aliakbarpour, J. Dias. (2010) Human Silhouette Volume Reconstruction Using a Gravity-based Virtual Camera Network. Proceedings of the 13th International Conference on Information Fusion, 26–29 July 2010 EICC Edinburgh, UK.
A. Calbi, C. Regazzoni, L. Marcenaro (2006) Dynamic Scene Reconstruction for Efficient Remote Surveillance. IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'06).
J. Franco, E. Boyer (2005) Fusion of Multi-View Silhouette Cues Using a Space Occupancy Grid. Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV05).
J. F. Ferreira, J. Lobo, J. Dias (2010) Bayesian real-time perception algorithms on GPU — Real-time implementation of Bayesian models for multimodal perception using CUDA. Journal of Real-Time Image Processing, Special Issue.
L. Almeida, P. Menezes, J. Dias (2011) Stereo Vision Head Vergence Using GPU Cepstral Filtering. Proceedings of the Fifth International Conference on Computer Vision Theory and Applications (VISAPP), Vilamoura, Algarve, Portugal, March, 5–7.
A. Griesser, S. D. Roeck, A. Neubeck, L. V. Gool (2005) Gpu-based foreground-background segmentation using an extended colinearity criterion. In Proc. Vision, Modeling, and Visualization (VMV) 2005. Amsterdam, The Netherlands: IOS, Nov. 2005., 319–326
G. Ziegler (2010) GPU Data Structures for Graphics and Vision. PhD thesis, Max-Planck-Institut für Informatik.
C. Nitschke, A. Nakazawa, H. Takemura (2007) Real-Time Space Carving Using Graphics Hardware. IEICE — Trans. Inf. Syst., E90-D:1175–1184
S.S. Stone, J.P. Haldar, S.C. Tsao, W.-m.W. Hwu, B.P. Sutton, Z.-P. Liang (2008) Accelerating advanced MRI reconstructions on GPUs. Journal of Parallel and Distributed Computing, 68(10):1307–1318, General-Purpose Processing using Graphics Processing Units.
W. Waizenegger, I. Feldmann, P. Eisert, P. Kauff (2009) Parallel high resolution real-time Visual Hull on GPU. Image Processing (ICIP), 2009 16th IEEE International Conference on, 430–4304
S. Yous, H. Laga, M. Kidode, K. Chihara (2007) GPU-based shape from silhouettes. Proceedings of the 5th international conference on Computer graphics and interactive techniques in Australia and Southeast Asia in GRAPHITE’ 07, 71–77, New York, NY, USA, 2007. ACM.
D. Knoblauch, F. Kuester (2009) Focused Volumetric Visual Hull with Color Extraction. In Bebis, George and Boyle, Richard and Parvin, Bahram and Koracin, Darko and Kuno, Yoshinori and Wang, Junxian and Pajarola, Renato and Lindstrom, Peter and Hinkenjann, Andre and Encarnacao, Miguel and Silva, Claudio and Coming, Daniel, editors, Advances in Visual Computing in Lecture Notes in Computer Science, Springer Berlin-Heidelberg, 208–217
A. Ladikos, S. Benhimane, N. Navab (2008) Efficient visual hull computation for real-time 3D reconstruction using CUDA. Computer Vision and Pattern Recognition Workshops, 2008. CVPRW’ 08. IEEE Computer Society Conference on, 1–8.
M. Yguel, O. Aycard, C. Laugier (2006) Efficient GPU-based Construction of Occupancy Grids Using several Laser Range-finders. Oct. 2006. Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on.
M. Kalantari, A. Hashemi, F. Jung, J. Guedon (2011) A New Solution to the Relative Orientation Problem Using Only 3 Points and the Vertical Direction. Journal of Mathematical Imaging and Vision, 39:259–268
H. Aliakbarpour, J. Dias (2011) Inertial-Visual Fusion For Camera Network Calibration. IEEE 9th International Conference on Industrial Informatics (INDIN 2011).
J. Lobo, J. Dias (2003) Vision and inertial sensor cooperation using gravity as a vertical reference. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 25(12):1597–1608
M. Labrie, P. Hebert, (2007) Efficient camera motion and 3D recovery using an inertial sensor. Computer and Robot Vision, 2007. CRV’ 07. Fourth Canadian Conference on, 55–62
T. Okatani, K. Deguchi (2002) Robust estimation of camera translation between two images using a camera with a 3D orientation sensor. Pattern Recognition, 2002. Proceedings. 16th International Conference on, 1:275–278
M. A. Brodie, A. Walmsley, W. Page (2008) The static accuracy and calibration of inertial measurement units for 3D orientation. Computer Methods in Biomechanics and Biomedical Engineering, 11:641–648
R. Hartley, A. Zisserman (2003) Multiple View Geometry in Computer Vision. CAMBRIDGE UNIVERSITY PRESS
Y. Ma, S. Soatta, J. Kosecka, S. S. Sastry (2004) An invitation to 3D vision. Springer.
L. G. B. Mirisola, J. Dias, A. Traca de Almeida (2007) Trajectory Recovery and 3D Mapping from Rotation-Compensated Imagery for an Airship. Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems San Diego, CA, USA, Oct 29–Nov 2.
L. G. B. Mirisola (2009) Exploiting attitude sensing in vision-based navigation, mapping and tracking including results from an airship. PhD thesis.
L. G. B. Mirisola, J. M. M. Dias (2007) Exploiting inertial sensing in mosaicing and visual navigation. In 6th IFAC Symposium on Inteligent Autonomous Vehicles (IAV07), Toulouse, France, Sep. 2007.
MRL, http://paloma.isr.uc.pt/mrl/.
Prosilica, http://www.1stvision.com/cameras/Prosilica/GC650-GC650C.html.
Xsens Motion Technologies. http://www.xsens.com.
J. Bouguet (2003) Camera Calibration Toolbox for Matlab. www.vision.caltech.edu/bouguetj.
J. Lobo, J. Dias (2007) Relative Pose Calibration Between Visual and Inertial Sensors. International Journal of Robotics Research, Special Issue 2nd Workshop on Integration of Vision and Inertial Sensors, 26:561–575
P. Kakumanu, S. Makrogiannis, N. Bourbakis (2007) A survey of skin-color modeling and detection methods. Pattern Recogn, 40:1106–1122
G. R. Bradski (1998) Computer Vision Face Tracking For Use in a Perceptual User Interface. Intel Technology Journal, (Q2).
OpenCV. http://opencv.willowgarage.com/.
NVIDIA. http://www.nvidia.com/.
T. Wada, X. Wu, S. Tokaim. T. Matsuyama (2000) Homography Based Parallel Volume Intersection: Toward Real-Time Volume Reconstruction Using Active Cameras. Computer Architectures for Machine Perception, 2000. Proceedings. Fifth IEEE International Workshop on 11–13 Sept. 2000, 331–339
P. Lai, A. Yilmaz (2008) Efficient object shape recovery via slicing planes. Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 1–6