BundleFusion
Tóm tắt
Real-time, high-quality, 3D scanning of large-scale scenes is key to mixed reality and robotic applications. However, scalability brings challenges of drift in pose estimation, introducing significant errors in the accumulated model. Approaches often require hours of offline processing to globally correct model errors. Recent online methods demonstrate compelling results but suffer from (1) needing minutes to perform online correction, preventing true real-time use; (2) brittle frame-to-frame (or frame-to-model) pose estimation, resulting in many tracking failures; or (3) supporting only unstructured point-based representations, which limit scan quality and applicability. We systematically address these issues with a novel, real-time, end-to-end reconstruction framework. At its core is a robust pose estimation strategy, optimizing per frame for a global set of camera poses by considering the complete history of RGB-D input with an efficient hierarchical approach. We remove the heavy reliance on temporal tracking and continually localize to the globally optimized frames instead. We contribute a parallelizable optimization framework, which employs correspondences based on sparse features and dense geometric and photometric matching. Our approach estimates globally optimized (i.e., bundle adjusted) poses in real time, supports robust tracking with recovery from gross tracking failures (i.e., relocalization), and re-estimates the 3D model in real time to ensure global consistency, all within a single framework. Our approach outperforms state-of-the-art online systems with quality on par to offline methods, but with unprecedented speed and scan completeness. Our framework leads to a comprehensive online scanning solution for large indoor environments, enabling ease of use and high-quality results. 1
Từ khóa
Tài liệu tham khảo
S. Agarwal K. Mierle and Others. 2013. Ceres Solver. Retrieved from http://ceres-solver.org. (2013). S. Agarwal K. Mierle and Others. 2013. Ceres Solver. Retrieved from http://ceres-solver.org. (2013).
S. Choi , Q.-Y. Zhou , and V. Koltun . 2015. Robust reconstruction of indoor scenes . In Proc. CVPR. 5556--5565 . S. Choi, Q.-Y. Zhou, and V. Koltun. 2015. Robust reconstruction of indoor scenes. In Proc. CVPR. 5556--5565.
Z. DeVito , M. Mara , M. Zollhöfer , G. Bernstein , J. Ragan-Kelley , C. Theobalt , P. Hanrahan , M. Fisher , and M. Nießner . 2016 . Opt: A domain specific language for non-linear least squares optimization in graphics and imaging. arXiv Preprint arXiv:1604.06525 (2016). Z. DeVito, M. Mara, M. Zollhöfer, G. Bernstein, J. Ragan-Kelley, C. Theobalt, P. Hanrahan, M. Fisher, and M. Nießner. 2016. Opt: A domain specific language for non-linear least squares optimization in graphics and imaging. arXiv Preprint arXiv:1604.06525 (2016).
P. Henry , M. Krainin , E. Herbst , X. Ren , and D. Fox . 2010 . RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments . In Proc. Int. Symp. Experimental Robotics , Vol. 20. 22 -- 25 . P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox. 2010. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In Proc. Int. Symp. Experimental Robotics, Vol. 20. 22--25.
R. Kümmerle , G. Grisetti , H. Strasdat , K. Konolige , and W. Burgard . 2011. g 2 o: A general framework for graph optimization . In Proc. ICRA. IEEE, 3607--3613 . R. Kümmerle, G. Grisetti, H. Strasdat, K. Konolige, and W. Burgard. 2011. g 2 o: A general framework for graph optimization. In Proc. ICRA. IEEE, 3607--3613.
R. M. Murray S. S. Sastry and L. Zexiang. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press. R. M. Murray S. S. Sastry and L. Zexiang. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press.
M. Nießner A. Dai and M. Fisher. 2014. Combining inertial navigation and ICP for real-time 3d surface reconstruction. In Eurographics (Short Papers). 13--16. M. Nießner A. Dai and M. Fisher. 2014. Combining inertial navigation and ICP for real-time 3d surface reconstruction. In Eurographics (Short Papers). 13--16.
F. Reichl J. Weiss and R. Westermann. 2015. Memory-efficient interactive online reconstruction from depth image streams. In Computer Graphics Forum. Wiley Online Library. F. Reichl J. Weiss and R. Westermann. 2015. Memory-efficient interactive online reconstruction from depth image streams. In Computer Graphics Forum. Wiley Online Library.
K. M. Wurm , A. Hornung , M. Bennewitz , C. Stachniss , and W. Burgard . 2010. OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems . In Proc. ICRA , Vol. 2 . K. M. Wurm, A. Hornung, M. Bennewitz, C. Stachniss, and W. Burgard. 2010. OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems. In Proc. ICRA, Vol. 2.