Deep photo
Tóm tắt
In this paper, we introduce a novel system for browsing, enhancing, and manipulating casual outdoor photographs by combining them with already existing georeferenced digital terrain and urban models. A simple interactive registration process is used to align a photograph with such a model. Once the photograph and the model have been registered, an abundance of information, such as depth, texture, and GIS data, becomes immediately available to our system. This information, in turn, enables a variety of operations, ranging from dehazing and relighting the photograph, to novel view synthesis, and overlaying with geographic information. We describe the implementation of a number of these applications and discuss possible extensions. Our results show that augmenting photographs with already available 3D models of the world supports a wide variety of new ways for us to experience and interact with our everyday snapshots.
Từ khóa
Tài liệu tham khảo
Ashikhmin , M. 2001 . Synthesizing natural textures . Proceedings of the 2001 symposium on Interactive 3D graphics (I3D), 217--226 . 10.1145/364338.364405 Ashikhmin, M. 2001. Synthesizing natural textures. Proceedings of the 2001 symposium on Interactive 3D graphics (I3D), 217--226. 10.1145/364338.364405
Chen B. Ramos G. Ofek E. Cohen M. Drucker S. and Nister D. 2008. Interactive techniques for registering images to digital terrain and building models. Microsoft Research Technical Report MSR-TR-2008-115. Chen B. Ramos G. Ofek E. Cohen M. Drucker S. and Nister D. 2008. Interactive techniques for registering images to digital terrain and building models. Microsoft Research Technical Report MSR-TR-2008-115 .
Cho , P. L. 2007 . 3D organization of 2D urban imagery . Proceedings of the 36th Applied Imagery Pattern Recognition Workshop, 3--8. 10 .1109/AIPR.2007.12 Cho, P. L. 2007. 3D organization of 2D urban imagery. Proceedings of the 36th Applied Imagery Pattern Recognition Workshop, 3--8. 10.1109/AIPR.2007.12
Debevec , P. E. , Taylor , C. J. , and Malik , J . 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach . Proceedings of SIGGRAPH '96 , 11--20. 10.1145/237170.237191 Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. Proceedings of SIGGRAPH '96, 11--20. 10.1145/237170.237191
Debevec , P. , Hawkins , T. , Tchou , C. , Duiker , H.-P. , Sarokin , W. , and Sagar , M . 2000. Acquiring the reflectance field of a human face . Proceedings of SIGGRAPH 2000 , 145--156. 10.1145/344779.344855 Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. Proceedings of SIGGRAPH 2000, 145--156. 10.1145/344779.344855
Drori , I. , Cohen-Or , D. , and Yeshurun , H . 2003. Fragment-based image completion . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2003) 22, 3, 303--312. 10.1145/882262.882267 Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based image completion. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2003) 22, 3, 303--312. 10.1145/882262.882267
E-on Software 2008. Vue 6 Infinite. http://www.e-onsoftware.com/products/vue/vue_6_infinite. E-on Software 2008. Vue 6 Infinite. http://www.e-onsoftware.com/products/vue/vue_6_infinite.
Efros , A. A. , and Leung , T. K . 1999. Texture synthesis by non-parametric sampling . Proceedings of IEEE International Conference on Computer Vision (ICCV) '99 2, 1033--1038 . Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. Proceedings of IEEE International Conference on Computer Vision (ICCV) '99 2, 1033--1038.
Fattal , R. 2008 . Single image dehazing . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008) 27, 3, 73. 10.1145/1360612.1360671 Fattal, R. 2008. Single image dehazing. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008) 27, 3, 73. 10.1145/1360612.1360671
Gruen A. and Huang T. S. 2001. Calibration and Orientation of Cameras in Computer Vision. Springer-Verlag Secaucus NJ USA. Gruen A. and Huang T. S. 2001. Calibration and Orientation of Cameras in Computer Vision. Springer-Verlag Secaucus NJ USA.
Hays , J. , and Efros , A. A . 2007. Scene completion using millions of photographs . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 4. 10.1145/1276377.1276382 Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 4. 10.1145/1276377.1276382
Hertzmann , A. , Jacobs , C. E. , Oliver , N. , Curless , B. , and Salesin , D. H . 2001. Image analogies . Proceedings of SIGGRAPH 2001 , 327--340. 10.1145/383259.383295 Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. Proceedings of SIGGRAPH 2001, 327--340. 10.1145/383259.383295
Hoiem , D. , Efros , A. A. , and Hebert , M . 2005. Automatic photo pop-up . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005) 24, 3, 577--584. 10.1145/1073204.1073232 Hoiem, D., Efros, A. A., and Hebert, M. 2005. Automatic photo pop-up. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005) 24, 3, 577--584. 10.1145/1073204.1073232
Horry , Y. , Anjyo , K.-I. , and Arai , K . 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image . Proceedings of SIGGRAPH '97 , 225--232. 10.1145/258734.258854 Horry, Y., Anjyo, K.-I., and Arai, K. 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image. Proceedings of SIGGRAPH '97, 225--232. 10.1145/258734.258854
Kaftory , R. , Schechner , Y. Y. , and Zeevi , Y. Y . 2007. Variational distance-dependent image restoration . Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2007 , 1--8. Kaftory, R., Schechner, Y. Y., and Zeevi, Y. Y. 2007. Variational distance-dependent image restoration. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2007, 1--8.
Kang , S. B. 1998. Depth painting for image-based rendering applications. Tech. rep ., Compaq Cambridge Research Lab . Kang, S. B. 1998. Depth painting for image-based rendering applications. Tech. rep., Compaq Cambridge Research Lab.
Lalonde , J.-F. , Hoiem , D. , Efros , A. A. , Rother , C. , Winn , J. , and Criminisi , A . 2007. Photo clip art . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 3. 10.1145/1276377.1276381 Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 3. 10.1145/1276377.1276381
McCartney , E. J. 1976. Optics of the Atmosphere: Scattering by Molecules and Particles . John Wiley and Sons , New York, NY, USA . McCartney, E. J. 1976. Optics of the Atmosphere: Scattering by Molecules and Particles. John Wiley and Sons, New York, NY, USA.
Narasimhan , S. G. , and Nayar , S. K . 2003. Interactive (de)weathering of an image using physical models . IEEE Workshop on Color and Photometric Methods in Computer Vision. Narasimhan, S. G., and Nayar, S. K. 2003. Interactive (de)weathering of an image using physical models. IEEE Workshop on Color and Photometric Methods in Computer Vision.
NASA 2008. The landsat program. http://landsat.gsfc.nasa.gov/. NASA 2008. The landsat program. http://landsat.gsfc.nasa.gov/.
NASA 2008. Shuttle radar topography mission. http://www2.jpl.nasa.gov/srtm/. NASA 2008. Shuttle radar topography mission. http://www2.jpl.nasa.gov/srtm/.
Nayar , S. K. , and Narasimhan , S. G . 1999. Vision in bad weather . Proceedings of IEEE International Conference on Computer Vision (ICCV) '99 , 820--827. Nayar, S. K., and Narasimhan, S. G. 1999. Vision in bad weather. Proceedings of IEEE International Conference on Computer Vision (ICCV) '99, 820--827.
Reichmann M. 2001. The art of photography. http://www.luminous-landscape.com/essays/theartof.shtml. Reichmann M. 2001. The art of photography. http://www.luminous-landscape.com/essays/theartof.shtml.
Shade , J. , Gortler , S. , He , L.-W. , and Szeliski , R . 1998. Layered depth images . Proceedings of SIGGRAPH '98 , 231--242. 10.1145/280814.280882 Shade, J., Gortler, S., He, L.-W., and Szeliski, R. 1998. Layered depth images. Proceedings of SIGGRAPH '98, 231--242. 10.1145/280814.280882
Shum , H.-Y. , Han , M. , and Szeliski , R . 1998. Interactive construction of 3-d models from panoramic mosaics . Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1998 , 427--433. Shum, H.-Y., Han, M., and Szeliski, R. 1998. Interactive construction of 3-d models from panoramic mosaics. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1998, 427--433.
Snavely , N. , Seitz , S. M. , and Szeliski , R . 2006. Photo tourism: exploring photo collections in 3d . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006) 25, 3, 835--846. 10.1145/114 1911 .1141964 Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006) 25, 3, 835--846. 10.1145/1141911.1141964
Stamos , I. , and Allen , P. K . 2000. 3-D model construction using range and image data . Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1998 , 531--536. Stamos, I., and Allen, P. K. 2000. 3-D model construction using range and image data. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1998, 531--536.
Sunkavalli , K. , Matusik , W. , Pfister , H. , and Rusinkiewicz , S . 2007. Factored time-lapse video . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 101. 10.1145/1276377.1276504 Sunkavalli, K., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2007. Factored time-lapse video. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 101. 10.1145/1276377.1276504
Toyama , K. , Logan , R. , and Roseway , A . 2003. Geographic location tags on digital images . Proceedings of the 11th ACM international conference on Multimedia, 156--166 . 10.1145/957013.957046 Toyama, K., Logan, R., and Roseway, A. 2003. Geographic location tags on digital images. Proceedings of the 11th ACM international conference on Multimedia, 156--166. 10.1145/957013.957046
Wang , J. , Agrawala , M. , and Cohen , M. F . 2007. Soft scissors: an interactive tool for realtime high quality matting . ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3. 10.1145/1276377.1276389 Wang, J., Agrawala, M., and Cohen, M. F. 2007. Soft scissors: an interactive tool for realtime high quality matting. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3. 10.1145/1276377.1276389
Yu , Y. , and Malik , J . 1998. Recovering photometric properties of architectural scenes from photographs . Proceedings of SIGGRAPH '98 , 207--217. 10.1145/280814.280874 Yu, Y., and Malik, J. 1998. Recovering photometric properties of architectural scenes from photographs. Proceedings of SIGGRAPH '98, 207--217. 10.1145/280814.280874
Yu , Y. , Debevec , P. , Malik , J. , and Hawkins , T . 1999. Inverse global illumination: recovering reflectance models of real scenes from photographs . Proceedings of SIGGRAPH '99 , 215--224. 10.1145/311535.311559 Yu, Y., Debevec, P., Malik, J., and Hawkins, T. 1999. Inverse global illumination: recovering reflectance models of real scenes from photographs. Proceedings of SIGGRAPH '99, 215--224. 10.1145/311535.311559