An Optical Digital Twin for Underwater Photogrammetry
PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science - Tập 90 - Trang 69-81 - 2022
Tóm tắt
Most parts of the Earth’s surface are situated in the deep ocean. To explore this visually rather adversarial environment with cameras, they have to be protected by pressure housings. These housings, in turn, need interfaces to the world, enduring extreme pressures within the water column. Commonly, a flat window or a half-sphere of glass, called flat-port or dome-port, respectively is used to implement such kind of interface. Hence, multi-media interfaces, between water, glass and air are introduced, entailing refraction effects in the images taken through them. To obtain unbiased 3D measurements and to yield a geometrically faithful reconstruction of the scene, it is mandatory to deal with the effects in a proper manner. Hence, we propose an optical digital twin of an underwater environment, which has been geometrically verified to resemble a real water lab tank that features the two most common optical interfaces. It can be used to develop, evaluate, train, test and tune refractive algorithms. Alongside this paper, we publish the model for further extension, jointly with code to dynamically generate samples from the dataset. Finally, we also publish a pre-rendered dataset ready for use at
https://git.geomar.de/david-nakath/geodt
.
Tài liệu tham khảo
Agrafiotis P, Karantzalos K, Georgopoulos A, Skarlatos D (2021) Learning from synthetic data: Enhancing refraction correction accuracy for airborne image-based bathymetric mapping of shallow coastal waters. PFG-J Photogram Remote Sens Geoinf Sci 2:1–19
Agrawal A, Taguchi Y, Ramalingam S (2010) Analytical forward projection for axial non-central dioptric and catadioptric cameras. In: European Conference on Computer Vision, pp. 129–143. Springer
Akkaynak D, Treibitz T (2019) Sea-thru: A method for removing water from underwater images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1682–1691
Akkaynak D, Treibitz T, Shlesinger T, Loya Y, Tamir R, Iluz D (2017) What is the space of attenuation coefficients in underwater computer vision? In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 568–577. IEEE
Community BO (2018) Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam http://www.blender.org
Cozman F, Krotkov E (1997) Depth from scattering. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp. 801–806. IEEE
Drap P (2012) Underwater photogrammetry for archaeology. In: D.C. da Silva (ed.) Special Applications of Photogrammetry, chap. 6. IntechOpen, Rijeka. https://doi.org/10.5772/33999
Duda A, Gaudig C (2016) Refractive forward projection for underwater flat port cameras. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2022–2027. IEEE
Eakins B, Sharman G (2012) Hypsographic curve of earth’s surface from etopo1. NOAA National Geophysical Data Center, Boulder, CO 5
Fong J, Wrenninge M, Kulla C, Habel R (2017) Production volume rendering: Siggraph 2017 course. In: ACM SIGGRAPH 2017 Courses 1–79
Fryer JG, Fraser CS (1986) On the calibration of underwater cameras. Photogram Rec 12:73–85
Glassner AS (1989) An introduction to ray tracing. Elsevier, Amstterdam
Harvey ES, Shortis MR (1998) Calibration stability of an underwater stereo-video system : Implications for measurement accuracy and precision. Mar Technol Soc J 32:3–17
Henyey LG, Greenstein JL (1941) Diffuse radiation in the galaxy. Astrophys J 93:70–83
Hu X, Lauze F, Pedersen KS, Melou J (2021) Absolute and relative pose estimation in refractive multi view. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2569–2578
Jaffe JS (1990) Computer modeling and the design of optimal underwater imaging systems. IEEE J Oceanic Eng 15(2):101–111
Jaffe JS, Moore KD, McLean J, Strand MP (2001) Underwater optical imaging: Status and prospects. Oceanography 14:2
Jordt A (2014) Underwater 3d reconstruction based on physical models for refraction and underwater light propagation. Ph.D. thesis, Christian-Albrechts-Universtät zu Kiel, Germany
Jordt A, Koch R (2011) Fast tracking of deformable objects in depth and colour video. In: McKenna S, Hoey J, Trucco M (eds.) Proceedings of the British Machine Vision Conference, BMVC 2011. British Machine Vision Association
Jordt A, Köser K, Koch R (2016) Refractive 3d reconstruction on underwater images. Methods Oceanogr 15–16:90–113. https://doi.org/10.1016/j.mio.2016.03.001
Jordt-Sedlazeck A, Koch R (2012) Refractive calibration of underwater cameras. In: European conference on computer vision, pp. 846–859. Springer
Kahmen O, Rofallski R, Conen N, Luhmann T (2019) On scale definition within calibration of multi-camera systems in mulimedia photogrammetry. Remote Sensing & Spatial Information Sciences, International Archives of the Photogrammetry
Kajiya JT (1986) The rendering equation. In: Proceedings of the 13th annual conference on Computer graphics and interactive techniques, pp. 143–150
Kirk JT (1994) Light and photosynthesis in aquatic ecosystems. Cambridge University Press, Cambridge
Koenig N, Howard A (2004) Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), vol. 3, pp. 2149–2154. IEEE
Kotowski R (1988) Phototriangulation in multi-media photogrammetry. Int’l Archives of Photogrammetry and Remote Sensing XXVII
Kunz C, Singh H (2008) Hemispherical refraction and camera calibration in underwater vision. In: OCEANS 2008, pp. 1–7. IEEE
Li B, Heng L, Koser K, Pollefeys M (2013) A multiple-camera system calibration toolbox using a feature descriptor-based calibration pattern. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1301–1307. IEEE
Manhães MMM, Scherer SA, Voss M, Douat LR, Rauschenbach T (2016) Uuv simulator: A gazebo-based package for underwater intervention and multi-robot simulation. In: OCEANS 2016 MTS/IEEE Monterey, pp. 1–8. IEEE
McGlamery BL (1975) Computer analysis and simulation of underwater camera system performance. Tech. rep., Visibility Laboratory, Scripps Institution of Oceanography, University of California in San Diego
Menna F, Nocerino E, Fassi F, Remondino F (2016) Geometric and optic characterization of a hemispherical dome port for underwater photogrammetry. Sensors 16(1). http://www.mdpi.com/1424-8220/16/1/48
Menna F, Nocerino E, Remondino F (2017). Optical aberrations in underwater photogrammetry with flat and hemispherical dome ports https://doi.org/10.1117/12.2270765
Menna F, Nocerino E, Ural S, Gruen A (2020) Mitigating image residuals systematic patterns in underwater photogrammetry. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLIII-B2-2020: 977–984. https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-977-2020
Millard R, Seaver G (1990) An index of refraction algorithm for seawater over temperature, pressure, salinity, density, and wavelength. Deep Sea Res Part A Oceanogr Res Pap 37(12):1909–1926
Mobley C, Boss E, Roesler C (2021) Ocean optics web book. URL http://www.oceanopticsbook.info
Moore EJ (1976) Underwater photogrammetry. Photogram Rec 8(48):748–763. https://doi.org/10.1111/j.1477-9730.1976.tb00852.x
Mulsow C, Maas HG (2014) A universal approach for geometric modelling in underwater stereo image processing. In: 2014 ICPR Workshop on Computer Vision for Analysis of Underwater Imagery, pp. 49–56. IEEE
Nakath D, She M, Song Y, Koser K (2021) In-situ joint light and medium estimation for underwater color restoration. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3731–3740
Nayar SK, Narasimhan SG (1999) Vision in bad weather. In: Proceedings of the Seventh IEEE International Conference on Computer Vision 2: 820–827 vol.2. https://doi.org/10.1109/ICCV.1999.790306
Nocerino E, Menna F, Gruen A (2021) Bundle adjustment with polynomial point-to-camera distance dependent corrections for underwater photogrammetry. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLIII-B2-2021: 673–679. https://doi.org/10.5194/isprs-archives-XLIII-B2-2021-673-2021
Novák J, Georgiev I, Hanika J, Krivánek J, Jarosz W (2018) Monte carlo methods for physically based volume rendering. In: SIGGRAPH Courses, pp. 14–1
Parker SG, Bigler J, Dietrich A, Friedrich H, Hoberock J, Luebke D, McAllister D, McGuire M, Morley K, Robison A, Stich M (2010) Optix: A general purpose ray tracing engine. ACM Trans Gr 2:2
Pharr M, Jakob W, Humphreys G (2016) Physically based rendering: From theory to implementation. Morgan Kaufmann
Prats M, Perez J, Fernández JJ, Sanz PJ (2012) An open source tool for simulation and supervision of underwater intervention missions. In: 2012 IEEE/RSJ international conference on Intelligent Robots and Systems, pp. 2577–2582. IEEE
Preisendorfer R (1964) Physical aspect of light in the sea. Univ Hawai Press Honolulu Hawaii 51:60
Quigley M, Gerkey B, Conley K, Faust J, Foote T, Leibs J, Berger E, Wheeler R, Ng A (2009) Ros: an open-source robot operating system. In: Proc. of the IEEE Intl. Conf. on Robotics and Automation (ICRA) Workshop on Open Source Robotics. Kobe, Japan
Radziszewski M, Boryczko K, Alda W (2009) An improved technique for full spectral rendering. J WSCG 17:9–16
Sedlazeck A, Koch R (2011) Simulating deep sea underwater images using physical models for light attenuation, scattering, and refraction. In: Eisert P, Hornegger J, Polthier K (eds) VMV 2011: Vision, Modeling & Visualization, 978-3-905673-85-2. Eurographics Association, Berlin, Germany, pp 49–56
She M, Nakath D, Song Y, Köser K (2022) Refractive geometry for underwater domes. ISPRS J Photogramm Remote Sens 183:525–540. https://doi.org/10.1016/j.isprsjprs.2021.11.006
She M, Song Y, Mohrmann J, Köser K (2019) Adjustment and calibration of dome port camera systems for underwater vision. In: German Conference on Pattern Recognition, pp. 79–92. Springer
Shmutter LB (1967) Orientation problems in two-media photogrammetry. Photogrammetric Engineering pp. 1421–1428
Song Y, Nakath D, She M, Elibol F, Köser K (2021) Deep sea robotic imaging simulator. In: Del Bimbo A, Cucchiara R, Sclaroff S, Farinella GM, Mei T, Bertini M, Escalante HJ, Vezzani R (eds) Pattern Recognition. ICPR International Workshops and Challenges. Springer International Publishing, Cham, pp 375–389
Treibitz T, Schechner YY, Singh H (2008) Flat refractive geometry. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition CVPR 2008, pp. 1–8
Veach E (1998) Robust Monte Carlo methods for light transport simulation. Stanford University, Stanford
Whitted T (1980) An improved illumination model for computer graphics. Comm. ACM 23(6):343–349
Zwilgmeyer PGO, Yip M, Teigen AL, Mester R, Stahl A (2021) The varos synthetic underwater data set: Towards realistic multi-sensor underwater data with ground truth. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3722–3730