BundleFusion

ACM Transactions on Graphics - Tập 36 Số 4 - Trang 1 - 2017
Angela Dai1, Matthias Nießner1, Michael Zollhöfer2, Shahram Izadi3, Christian Theobalt2
1Stanford University
2Max-Planck-Institute for Informatics
3Microsoft Research

Tóm tắt

Real-time, high-quality, 3D scanning of large-scale scenes is key to mixed reality and robotic applications. However, scalability brings challenges of drift in pose estimation, introducing significant errors in the accumulated model. Approaches often require hours of offline processing to globally correct model errors. Recent online methods demonstrate compelling results but suffer from (1) needing minutes to perform online correction, preventing true real-time use; (2) brittle frame-to-frame (or frame-to-model) pose estimation, resulting in many tracking failures; or (3) supporting only unstructured point-based representations, which limit scan quality and applicability. We systematically address these issues with a novel, real-time, end-to-end reconstruction framework. At its core is a robust pose estimation strategy, optimizing per frame for a global set of camera poses by considering the complete history of RGB-D input with an efficient hierarchical approach. We remove the heavy reliance on temporal tracking and continually localize to the globally optimized frames instead. We contribute a parallelizable optimization framework, which employs correspondences based on sparse features and dense geometric and photometric matching. Our approach estimates globally optimized (i.e., bundle adjusted) poses in real time, supports robust tracking with recovery from gross tracking failures (i.e., relocalization), and re-estimates the 3D model in real time to ensure global consistency, all within a single framework. Our approach outperforms state-of-the-art online systems with quality on par to offline methods, but with unprecedented speed and scan completeness. Our framework leads to a comprehensive online scanning solution for large indoor environments, enabling ease of use and high-quality results. 1

Từ khóa


Tài liệu tham khảo

S. Agarwal K. Mierle and Others. 2013. Ceres Solver. Retrieved from http://ceres-solver.org. (2013). S. Agarwal K. Mierle and Others. 2013. Ceres Solver. Retrieved from http://ceres-solver.org. (2013).

10.1109/34.121791

10.1145/2461912.2461940

S. Choi , Q.-Y. Zhou , and V. Koltun . 2015. Robust reconstruction of indoor scenes . In Proc. CVPR. 5556--5565 . S. Choi, Q.-Y. Zhou, and V. Koltun. 2015. Robust reconstruction of indoor scenes. In Proc. CVPR. 5556--5565.

10.1145/237170.237269

Z. DeVito , M. Mara , M. Zollhöfer , G. Bernstein , J. Ragan-Kelley , C. Theobalt , P. Hanrahan , M. Fisher , and M. Nießner . 2016 . Opt: A domain specific language for non-linear least squares optimization in graphics and imaging. arXiv Preprint arXiv:1604.06525 (2016). Z. DeVito, M. Mara, M. Zollhöfer, G. Bernstein, J. Ragan-Kelley, C. Theobalt, P. Hanrahan, M. Fisher, and M. Nießner. 2016. Opt: A domain specific language for non-linear least squares optimization in graphics and imaging. arXiv Preprint arXiv:1604.06525 (2016).

10.1109/CDC.1987.272800

10.1109/ICRA.2012.6225199

10.1007/978-3-319-10605-2_54

10.1109/ICCV.2013.183

10.1109/CVPR.2015.7299077

10.1109/ICRA.2014.6906584

10.1145/2601097.2601163

10.1007/978-3-642-15986-2_1

10.1109/TVCG.2014.2360403

10.1007/BF02291478

10.1109/ICRA.2014.6907054

P. Henry , M. Krainin , E. Herbst , X. Ren , and D. Fox . 2010 . RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments . In Proc. Int. Symp. Experimental Robotics , Vol. 20. 22 -- 25 . P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox. 2010. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In Proc. Int. Symp. Experimental Robotics, Vol. 20. 22--25.

10.1007/BFb0015528

10.1145/2047196.2047270

10.1107/S0567739476001873

10.1109/3DV.2013.9

10.1109/IROS.2013.6696650

10.1109/ISMAR.2007.4538852

R. Kümmerle , G. Grisetti , H. Strasdat , K. Konolige , and W. Burgard . 2011. g 2 o: A general framework for graph optimization . In Proc. ICRA. IEEE, 3607--3613 . R. Kümmerle, G. Grisetti, H. Strasdat, K. Konolige, and W. Burgard. 2011. g 2 o: A general framework for graph optimization. In Proc. ICRA. IEEE, 3607--3613.

10.1145/344779.344849

10.1145/2508363.2508407

10.1023/B:VISI.0000029664.99615.94

10.1007/978-3-319-11752-2_5

10.1109/IROS.2013.6696881

10.5244/C.25.45

10.1109/ICCV.2007.4408984

R. M. Murray S. S. Sastry and L. Zexiang. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press. R. M. Murray S. S. Sastry and L. Zexiang. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press.

10.1109/ISMAR.2011.6092378

10.1109/ICCV.2011.6126513

M. Nießner A. Dai and M. Fisher. 2014. Combining inertial navigation and ICP for real-time 3d surface reconstruction. In Eurographics (Short Papers). 13--16. M. Nießner A. Dai and M. Fisher. 2014. Combining inertial navigation and ICP for real-time 3d surface reconstruction. In Eurographics (Short Papers). 13--16.

10.1145/2508363.2508374

10.1109/ISMAR.2013.6671767

F. Reichl J. Weiss and R. Westermann. 2015. Memory-efficient interactive online reconstruction from depth image streams. In Computer Graphics Forum. Wiley Online Library. F. Reichl J. Weiss and R. Westermann. 2015. Memory-efficient interactive online reconstruction from depth image streams. In Computer Graphics Forum. Wiley Online Library.

10.5244/C.26.112

10.1145/566654.566600

10.1109/IM.2001.924423

10.1007/978-3-642-33715-4_54

10.1109/ICCV.2013.405

10.1109/ICRA.2014.6907127

10.1016/j.jvcir.2013.02.008

10.1109/IROS.2012.6385773

10.1007/3-540-44480-7_21

10.1109/CVPR.2015.7299069

10.1109/ICCVW.2009.5457479

10.1109/ICRA.2013.6631400

10.1109/IROS.2013.6696405

10.15607/RSS.2015.XI.001

10.1145/2661229.2661232

K. M. Wurm , A. Hornung , M. Bennewitz , C. Stachniss , and W. Burgard . 2010. OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems . In Proc. ICRA , Vol. 2 . K. M. Wurm, A. Hornung, M. Bennewitz, C. Stachniss, and W. Burgard. 2010. OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems. In Proc. ICRA, Vol. 2.

10.1109/ICCV.2013.458

10.1016/j.gmod.2012.09.002

10.1145/2768821

10.1145/2461912.2461919

10.1145/2601097.2601134

10.1109/ICCV.2013.65

10.1145/2766887

10.1145/2601097.2601165

Thông tin
Thông tin xuất bản

ACM Transactions on Graphics

Tập 36 Số 4

1

Thông tin tác giả