The fundamental matrix: Theory, algorithms, and stability analysis

Springer Science and Business Media LLC - Tập 17 - Trang 43-75 - 1996
Quan-Tuan Luong1, Olivier D. Faugeras1
1I.N.R.I.A., Sophia-Antipolis, France

Tóm tắt

In this paper we analyze in some detail the geometry of a pair of cameras, i.e., a stereo rig. Contrarily to what has been done in the past and is still done currently, for example in stereo or motion analysis, we do not assume that the intrinsic parameters of the cameras are known (coordinates of the principal points, pixels aspect ratio and focal lengths). This is important for two reasons. First, it is more realistic in applications where these parameters may vary according to the task (active vision). Second, the general case considered here, captures all the relevant information that is necessary for establishing correspondences between two pairs of images. This information is fundamentally projective and is hidden in a confusing manner in the commonly used formalism of the Essential matrix introduced by Longuet-Higgins (1981). This paper clarifies the projective nature of the correspondence problem in stereo and shows that the epipolar geometry can be summarized in one 3×3 matrix of rank 2 which we propose to call the Fundamental matrix. After this theoretical analysis, we embark on the task of estimating the Fundamental matrix from point correspondences, a task which is of practical importance. We analyze theoretically, and compare experimentally using synthetic and real data, several methods of estimation. The problem of the stability of the estimation is studied from two complementary viewpoints. First we show that there is an interesting relationship between the Fundamental matrix and three-dimensional planes which induce homographies between the images and create unstabilities in the estimation procedures. Second, we point to a deep relation between the unstability of the estimation procedure and the presence in the scene of so-called critical surfaces which have been studied in the context of motion analysis. Finally we conclude by stressing the fact that we believe that the Fundamental matrix will play a crucial role in future applications of three-dimensional Computer Vision by greatly increasing its versatility, robustness and hence applicability to real difficult problems.

Tài liệu tham khảo

Adiv, G. 1989. Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11:477–489.

Barrett, E.B., Brill, M.H., Haag, N.N., and Payton, P.M. 1992. Invariant linear methods in photogrammetry and model-matching. In J.L. Mundy and A. Zisserman (Eds.), Geometric Invariance in Computer Vision, Chap. 14, pp. 277–292, MIT Press.

Beardsley, P.A. 1992. Applications of Projective Geometry to Computer vision. Ph.D. Thesis, University of Oxford.

Beardsley, P.A., Zisserman, A., and Murray, D.W. 1994. Navigation using affine structure from motion. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 85–96.

Bookstein, F.L. 1979. Fitting conic sections to scattered data. Computer Graphics and Image Processing, 9(1):56–71.

Coxeter, H.S.M. 1987. Projective Geometry. Springer Verlag (Second Edition).

Daniilidis, K. 1992. Zur Fehlerempfindlichkeit in der Ermittlung von Objektbeschreibungen und relativen Bewegugen aus monokularen Bildfolgen. Ph.D. Thesis, University of Karsruhe.

Daniilidis, K. and Nagel, H.-H. 1990. Analytical results on error sensitivity of motion estimation from two views. Image and Vision Computing, 8:297–303.

Demey, S., Zisserman, A., and Beardsley, P.A. 1992. Affine and projective structure from motion. In Proc. British Machine Vision Conference, Leeds, UK, pp. 49–58.

Deriche, R., Zhang, Z., Luong, Q.-T., and Faugeras, O.D. 1994. Robust recovery of the epipolar geometry for an uncalibrated stereo rig. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 567–576.

Devernay, F. and Faugeras, O.D. 1994. Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In cvpr, Seattle, WA, pp. 208–213.

Fang, J.Q. and Huang, T.S. 1984. Some experiments on estimating the 3D motion parameters of a rigid body from two consecutive image frames. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:545–554.

Faugeras, O.D. 1992. What can be seen in three dimensions with an uncalibrated stereo rig. In Proc. European Conference on Computer Vision, pp. 563–578.

Faugeras, O.D. 1993. Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press.

Faugeras, O.D. Hotz, B. 1994. Real time correlation-based stereo: algorithm, implementations and applications. The International Journal of Computer Vision. To appear.

Faugeras, O.D., Luong, Q.-T., and Maybank, S.J. 1992. Camera self-calibration: theory and experiments. In Proc. European Conference on Computer Vision, Santa-Margerita, Italy, pp. 321–334.

Faugeras, O.D. and Lustman, F. 1988. Motion and structure from motion in a piecewise planar environment. International Journal of Pattern Recognition and Artificial Intelligence, 2(3):485–508.

Faugeras, O.D., Lustman, F., and Toscani, G. 1987. Motion and structure from point and line matches. In Proc. International Conference on Computer Vision, pp. 25–34.

Faugeras, O.D. and Robert, L. 1993. What can two images tell us about a third one? Technical Report RR-2018, INRIA. To appear in IJCV.

Faugeras, O.D. and Toscani, G. 1986. The calibration problem for stereo. In Proceedings of CVPR'86, pp. 15–20.

Garner, L.E. 1981. An Outline of Projective Geometry. Elsevier: North Holland.

Gros, P. and Quan, L. 1993. 3D projective invariants from two images. In Geometric Methods in Computer Vision II, SPIE Optical Instrumentation and Applied Science, San Diego.

Hartley, R.I. 1992. Estimation of relative camera positions for uncalibrated cameras. In Proc. European Conference on Computer Vision, pp. 579–587.

Hartley, R.I. 1993. Cheirality invariants. In Proc. DARPA Image Understanding Workshop, University of Maryland, pp. 745–753.

Hartley, R.I. 1994a. An algorithm for self calibration from several views. In Proc. Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 908–912.

Hartley, R.I. 1994b. Projective reconstruction from line correspondences. In Proc. Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 903–907.

Hartley, R.I. and Gupta R. 1993. Computing matched-epipolar projections. In Proc. Conference on Computer Vision and Pattern Recognition, New York, pp. 549–555.

Horn, B.K.P. 1990. Relative orientation. The International Journal of Computer Vision, 4(1):59–78.

Horn, B.K.P. and Weldon, E.J. 1988. Direct methods for recovering motion. The International Journal of Computer Vision, 2(1):51–76.

Huang, T.S. and Faugeras, O.D. 1989. Some properties of the E-matrix in two view motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11:1310–1312.

Jerian, C.P. and Jain, R. 1991. Structure from motion. A critical analysis of methods. IEEE Transactions on Systems, Man and Cybernetics, 21(3):572–587.

Jones, D.G. and Malik, J. 1992. A computational framework for determining stereo correspondence from a set of linear spatial filters. In Proc. European Conference on Computer Vision, pp. 395–410.

Kanatani, K. 1991. Computational projective geometry. Computer Vision, Graphics, and Image Processing. Image Understanding, 54(3).

Kanatani, K. 1992. Geometric Computation for Machine Vision. Oxford University Press.

Laveau, S. and Faugeras, O.D. 1994. 3-D scene representation as a collection of images. In Proc. International Conference on Pattern Recognition, Jerusalem, Israel. To appear.

Lawn, J.M. and Cipolla, R. 1994. Robust egomotion estimation from affine motion parallax. Technical Report CUED/F-INFENG/TR 160, University of Cambridge. A shorter version appeared at ECCV'94.

Lee, C.H. 1991. Time-varying images: the effect of finite resolution on uniqueness. Computer Vision, Graphics, and Images Processing. Image Understanding, 54(3):325–332.

Longuet-Higgins, H.C. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133–135.

Longuet-Higgins, C. 1984. The reconstruction of a scene from two projections: configurations that defeat the 8-point algorithm. In Proc. 1st Conf. on Artificial Intelligence Applications, Denver, pp. 395–397.

Longuet-Higgins, H.C. 1988. Multiple interpretations of a pair of images of a surface. Proc. of the Royal Society London A, 418:1–15.

Longuet-Higgins, H.C. and Prazdny, K. 1980. The interpretation of moving retinal images. Proceedings of the Royal Society of London B, 208:385–387.

Luong, Q.-T. 1992. Matrice fondamentale et auto-calibration en vision par ordinateur. Ph.D. Thesis, Universite de Paris-Sud, Orsay.

Luong, Q.-T. and Faugeras, O.D. 1993a. Camera calibration, scene motion and structure recovery from point correspondences and fundamental matrices. Submitted to IJCV.

Luong, Q.-T. and Faugeras, O.D. 1993b. Determining the fundamental matrix with planes: unstability and new algorithms. In Proc. Conference on Computer Vision and Pattern Recognition, New York, pp. 489–494.

Luong, Q.-T., Deriche, R., Faugeras, O.D., and Papadopoulo, T. 1993. On determining the fundamental matrix: analysis of different methods and experimental results. Technical Report RR-1894, INRIA. A shorter version appeared in the Israelian Conf. on Artificial Intelligence and Computer Vision.

Luong, Q.-T. and Faugeras, O.D. 1994. Stratified projective motion analysis: Fundamental matrix and self-calibration. In preparation.

Luong, Q.-T. and Vieville, T. 1994. Canonical representations for the geometries of multiple projective views. CVGIP: image understanding. To appear.

Maybank, S.J. 1985. The angular velocity associated with the optical flow field arising from motion through a rigid environment. Proc. of the Royal Society London A, 401:317–326.

Maybank, S.J. 1990a. The projective geometry of ambiguous surfaces. Proc. of the Royal Society London A, 332:1–47.

Maybank, S.J. 1990b. Properties of essential matrices. International Journal of Imaging Systems and Technology, 2:380–384.

Maybank, S.J. and Faugeras, O.D. 1992. A theory of self-calibration of a moving camera. The International Journal of Computer Vision, 8(2):123–151.

McLauchlan, P.F., Reid, I.D., and Murray, D.W. 1994. Recursive affine structure and motion from image sequences. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 217–224.

Mitiche, A., Zhuang, X., and Haralick, R. 1987. Interpretation of optical flow by rotation decoupling. In Proc IEEE Workshop on Computer Vision, Miami Beach, FL, pp. 195–200.

Mundy, J.L. and Zisserman, A. (Eds.) 1992, Geometric Invariance in Computer Vision. MIT Press.

Mundy, J., Welty, R.P., Brill, M.H., Payton, P.M., and Barrett. 1992. 3-D model alignment without computing pose. In Proc. DARPA Image Understanding Workshop, San Mateo, CA, pp. 727–735.

Nishimura, E., Xu, G., and Tsuji, S. 1993. Motion segmentation and correspondence using epipolar constraint. In Proc. 1st Asian Conf. Computer Vision, Osaka, Japan, pp. 199–204.

Olsen, S.I. 1992. Epipolar line estimation. In Proc. European Conference on Computer Vision, pp. 307–311.

Philip, J. 1991. Estimation of three-dimensional motion of rigid objects from noisy observations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(1):61–66.

Ponce, J., Marimont, D.H., and Cass, T.A. 1994. Analytical methods for uncalibrated stereo and motion reconstruction. In Proc. European Conference on Computer Vision, Stockholm, Sweden. pp. 463–470.

Quan, L. and Mohr, R. 1992. Affine shape representation from motion through reference points. Journal of Mathematical Imaging and Vision, 1:145–151.

Robert, L. 1993. Reconstruction de Courbs et de Surfaces par Vision Stéréoscopique. Applications a la Robotique Mobile. Ph.D. Thesis, Ecole Polytechnique.

Robert, L. and Faugeras, O.D. 1993. Relative 3D positioning and 3D convex hull computation from a weakly calibrated Stereo pair. In Proc. International Conference on Computer Vision, Berlin, Germany, pp. 540–543.

Robert, L. and Hebert, M. 1994. Deriving orientation cues from stereo images. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 377–388.

Sampson, P.D. 1982. Fitting conic sections to very scattered data: an iterative refinement of the Bookstein algorithm. Computer Graphics and Image Processing, 18(1):97–108.

Semple, J.G. and Kneebone, G.T. 1952. Algebraic Projective Geometry. Clarendon Press: Oxford. Reprinted 1979.

Shapiro, L.S., Zisserman, A., and Brady, M. 1993. Motion from point matches using affine epipolar geometry. Technical Report OUEL 1994/93, Oxford University. A shorter version appeared at ECCV'94.

Shashua, A. 1993a. On geometric and algebraic aspects of 3D affine and projective structures from perspective 2D views. In Proceedings of the 2nd European Workshop on Invariants, Ponta Delagada, Azores. Also MIT AI Memo No. 1405.

Shashua, A. 1993b. Projective depth: a geometric invariant for 3d reconstruction from two perspective/orthographic views and for visual recognition. In Proc. International Conference on Computer Vision, Berlin, Germany, pp. 583–590.

Shashua, A. and Navab, N. 1994. Relative affine structure: theory and application to 3D reconstruction from perspective views. In Proc. Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 483–489.

Sinclair, D., Blake, A., Smith, S., and Rothwell, C. 1992. Planar region detection and motion recovery. In Proc. British Machine Vision Conf., pp. 59–68.

Spetsakis, M.E. 1992. A linear algorithm for point and line-based structure from motion. Computer Vision, Graphics, and Image Processing. Image Understanding, 56(2):230–241.

Torr, P.H.S. and Murray, D.W. 1994. Stochastic motion clustering. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 328–337.

Tsai, R.Y. 1986. An efficient and accurate camera calibration technique for 3D machine vision. In IEEE Proceedings CVPR'86. Miami Beach, Florida, pp. 364–374.

Tsai, R.Y. and Huang, T.S. 1982. Estimating three-dimensional motion parameters of a rigid planar patch, II: singular value decomposition. IEEE Transactions on Acoustic, Speech and Signal Processing, 30.

Tsai, R.Y. and Huang, T.S. 1984. Uniqueness and estimation of three-dimensional motion parameters of rigid objects wirth curved surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:13–27.

Viéville, T. and Sander, P. 1992. Using pseudo kalman-filters in the presence of constraints. Technical Report RR-1669, INRIA.

Viéville, T., Luong, Q.-T., and Faugeras, O.D. 1994. Motion of points and lines in the uncalibrated case. Intl. Journal of Computer Vision. To appear.

Weber, J. and Malik, J. 1994. Rigid body segmentation and shape description from dense optical flow under weak perspective. Dept. of EECS, University of California at Berkeley.

Weng, J., Huang, T.S., and Ahuja, N., 1989. Motion and structure from two perspective views: algorithms, error analysis and error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):451–476.

Zhang, Z. and Faugeras, O.D. 1992. 3D Dynamic Scene Analysis. Springer-Verlag.

Thông tin
Thông tin xuất bản

Springer Science and Business Media LLC

Tập 17

43-75

Thông tin tác giả