A Survey of Computer Vision-Based Human Motion Capture

Aggarwal, 1999, Human motion analysis: a review, Comput. Vision Image Understanding, 73, 10.1006/cviu.1998.0744 J. K. Aggarwal, Q. Cai, W. Liao, and B. Sabata, Articulated and elastic non-rigid motion: a review, in Workshop on Motion of Non-Rigid and Articulated Objects, Austin, TX, 1994, pp. 2–14. Akita, 1984, Image sequence analysis of real world human motion, Pattern Recognition, 17, 73, 10.1016/0031-3203(84)90036-0 J. Amat, M. Casals, and, M. Frigola, Stereoscopic system for human body tracking in natural scenes, in, International Workshop on Modeling People at ICCV'99, Corfu, Greece, September 1999. Andersen, 1999, Human Motion Capture C. I. Attwood, G. D. Sullivan, and, K. D. Baker, Model-based recognition of human posture using single synthetic images, in, Fifth Alvey Vision Conference, University of Reading, UK, 1989. A. Azarbayejani, C. R. Wren, and, A. P. Pentland, Real-time 3-D tracking of the human body, in, IMAGE'COM 96, Bordeaux, France, May 1996. N. Badler, Virtual humans for animation, ergonomics, and simulation, in, Workshop on Motion of Non-Rigid and Articulated Objects, Puerto Rico, 1997. A. M. Baumberg and D. C. Hogg, An efficient method for contour tracking using active shape models, in Workshop on Motion of Non-Rigid and Articulated Objects, Austin, TX, 1994, pp. 2–14. D. A. Becker, and, A. Pentland, Staying alive: a virtual reality visualization tool for cancer patients, in, AAAI'96 Workshop on Entertainment and Alife/AI, Portland, OR, August 1996. Bernat, 1987, Security applications of computer motion detection, Appl. Artificial Intell, V, 786 A. G. Bharatkumar, K. E. Daigle, M. G. Pandy, Q. Cai, and, J. K. Aggarwal, Lower limb kinematics of human walking with the medial axis transformation, in, Workshop on Motion of Non-Rigid and Articulated Objects, Austin, TX, 1994. Blake, 1998, Active Contours, 10.1007/978-1-4471-1555-7 A. F. Bobick, and, J. W. Davis, An appearance-based representation of action, in, International Conference on Pattern Recognition, 1996. A. Bottino, A. Laurentini, and, P. Zuccone, Toward non-intrusive motion capture, in, Asian Conference on Computer Vision, 1998. M. Brand, Shadow puppetry, in, International Conference on Computer Vision, Corfu, Greece, September 1999. C. Bregler, Learning and recognizing human dynamics in video sequences, in, Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, 1997. C. Bregler, and, J. Malik, Tracking people with twists and exponential maps, in, International Conference on Computer Vision and Pattern Recognition, 1998. Q. Cai, and, J. K. Aggarwal, Tracking human motion using multiple cameras, in, International Conference on Pattern Recognition, 1996. Q. Cai, A. Mitiche, and, J. K. Aggarwal, Tracking human motion in an indoor environment, in, International Conference on Image Processing, 1995. L. Campbell, and, A. Bobick, Recognition of human body motion using phase space constraints, in, International Conference on Computer Vision, Cambridge, MA, 1995. L. Campbell, and, A. Bobick, Using phase space constraints to represent human body motion, in, International Workshop on Automatic Face- and Gesture-Recognition, Zurich, Switzerland, 1995. Cedras, 1995, Motion-based recognition: a survey, Image Vision Comput, 13, 129, 10.1016/0262-8856(95)93154-K T. J. Cham, and, J. M. Rehg, Multiple hypothesis approach to figure tracking, in, Conference on Computer Vision and Pattern Recognition, Fort Collins, CO, June 23–25 1999. O. Chomat, and, J. L. Crowley, Recognizing motion using local appearance, in, International Symposium on Intelligent Robotic Systems, University of Edinburgh, 1998. Christensen, 1997, Tracking of Articulated Objects Using Model-Based Computer Vision Christensen, 1997, Visualization of Human Motion Using Model-based Vision J. M. Chung, and, N. Ohnishi, Cue circles: image feature for measuring 3-D motion of articulated objects using sequential image pair, in, International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998. Corlin, 1998, Real Time Tracking of a Human Arm A. Cretual, F. Chaumette, and, P. Bouthemy, Complex object tracking by visual serving based on 2D image motion, in, International Conference on Pattern Recognition, 1998. R. Cutler, and, L. Davis, Real-time periodic motion detection, analysis, and applications, in, Conference on Computer Vision and Pattern Recognition, Fort Collins, CO, June 23–25, 1999. T. Darrell, P. Maes, B. Blumberg, and, A. P. Pentland, A novel environment for situated vision and behavior, in, Workshop for Visual Behaviors at CVPR-94, 1994. J. W. Davis, and, A. Bobick, The representation and recognition of action using temporal templates, in, Conference on Computer Vision and Pattern Recognition, 1997. Davis, 1998, SIDEshow: A Silhouette-based Interactive Dual-screen Environment J. W. Davis, and, A. Bobick, Virtual PAT: A virtual personal aerobics trainer, in, Workshop on Perceptual User Interfaces, San Francisco, November 1998. L. Davis, S. Fejes, D. Harwood, Y. Yacoob, I. Hariatoglu, and, M. J. Black, Visual surveillance of human activity, in, Asian Conference on Computer Vision, Mumbai, India, 1998. Q. Delamarre, and, O. Faugeras, 3D articulated models and multi-view tracking with silhouettes, in, International Conference on Computer Vision, Corfu, Greece, September 1999. Delaney, 1998, On the trail of the shadow women: the mystery of motion capture, Comput. Graphics Appl, 18, 14, 10.1109/38.708556 Dempster, 1977, Maximum likelihood estimation from incomplete data via the EM algorithm, J. Roy. Stat. Soc. (B), 39, 1 I. Douros, L. Dekker, and, B. F. Buxton, An improved algorithm for reconstruction of the surface of the human body from 3D scanner data using local B-spline patches, in, International Workshop on Modeling People at ICCV'99, Corfu, Greece, September 1999. W. T. Freeman, K. Tanaka, J. Ohta, and, K. Kyuma, Computer vision for computer games, in, International Workshop on Automatic Face- and Gesture-Recognition, Zurich, Switzerland, 1995. P. Fua, A. Gruen, R. Plänkers, N. D'Apuzzo, and, D. Thalmann, Human body modeling and motion analysis from video sequences, in, International Symposium on Real Time Imaging and Dynamic Analysis, Hakodate, Japan, June 1998. H. Fujiyoshi, and, A. J. Lipton, Real-time human motion analysis by image skeletonization, in, Workshop on Applications of Computer Vision, 1998. Gavrila, 1999, The visual analysis of human movement: a survey, Comput. Vision Image Understanding, 73, 82, 10.1006/cviu.1998.0716 D. M. Gavrila, and, L. S. Davis, 3-D model-based tracking of humans in action: a multi-view approach, in, Conference on Computer Vision and Pattern Recognition, San Francisco, CA, 1996. L. Goncalves, E. D. Bernardo, and, P. Perona, Reach out and touch space (motion learning), in, International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998. L. Goncalves, E. D. Bernardo, E. Ursella, and, P. Perona, Monocular tracking of the human arm in 3D, in, International Conference on Computer Vision, Cambridge, MA, 1995. H. Gu, Y. Shirai, and, M. Asada, MDL-based spatiotemporal segmentation from motion in a long image sequence, in, Conference on Computer Vision and Pattern Recognition, 1994. Gu, 1998, A 3D reconstruction system for human body modeling, Modeling and Motion Capture Techniques for Virtual Environments, Lecture Notes in Artificial Intelligence, Vol. 1537 Guo, 1994, Tracking human body motion based on a stick figure model, J. Visual Comm. Image Representation, 5, 1, 10.1006/jvci.1994.1001 Haritaoglu, 1999, Backpack: Detection of people carrying objects using silhouettes, International Conference on Computer Vision, Corfu, Greece Haritaoglu, 1998, Ghost: A human body part labeling system using silhouettes, International Conference on Pattern Recognition I. Haritaoglu, D. Harwood, and, L. S. Davis, W4: Who? When? Where? What? - A real time system for detecting and tracking people, in, International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998. B. Heisele, and, C. Wohler, Motion-based recognition of pedestrians, in, International Conference on Pattern Recognition, 1998. A. Hilton, Towards model-based capture of a persons shape, appearance and motion, in, International Workshop on Modeling People at ICCV'99, Corfu, Greece, September 1999. Hilton, 1999, Virtual people: Capturing human models to populate virtual worlds, International Conference on Computer Animation, 174 Hilton, 2000, Whole-body modelling of people from multi-view images to populate virtual worlds, The Visual Computer, 16, 411, 10.1007/PL00013395 Hogg, 1983, Model-based vision: A program to see a walking person, Image and Vision Computing, 1, 5, 10.1016/0262-8856(83)90003-3 Hogg, 1984, Interpreting Images of a Known Moving Object C. Hu, Q. Tu, Y. Li, and, S. Ma, Extraction of parametric human model for posture recognition using genetic algorithm, in, The fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. Hu, 1962, Visual pattern recognition by moment invariants, IRE Trans. Inform. Theory, 8, 179, 10.1109/TIT.1962.1057692 Hunter, 1997, Estimation of articulated motion using kinematically constrained mixture densities, Workshop on Motion of Non-Rigid and Articulated Objects, Puerto Rico, 10.1109/NAMW.1997.609844 S. Ioffe, and, D. Forsyth, Finding people by sampling, in, International Conference on Computer Vision, Corfu, Greece, September 1999. Isard, 1998, CONDENSATION-conditional density propagation for visual tracking, Int. J. Comput. Vision, 5, 10.1023/A:1008078328650 Y. Iwai, K. Ogaki, and, M. Yachida, Posture estimation using structure and motion models, in, Int. Conf. Comput. Vision, Corfu, Greece, September 1999. S. Iwasawa, K. Ebihara, J. Ohya, and, S. Morishima, Real-time estimation of human body posture from monocular thermal images, in, Conference on Computer Vision and Pattern Recognition, 1997. S. Iwasawa, J. Ohya, K. Takahashi, T. Sakaguchi, S. Kawato, K. Ebihara, and, S. Morishima, Real-time estimation of human body posture from trinocular images, in, International Workshop on Modeling People at ICCV'99, Corfu, Greece, September 1999. S. Iwassawa, J. Ohya, K. Takahashi, T. Sakaguchi, K. Ebihara, and, S. Morishima, Human body postures from trinocular camera images, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. Johnsson, 1975, Visual motion perception, Scientific American, 76, 10.1038/scientificamerican0675-76 N. Jojic, B. Brumitt, B. Meyers, S. Harris, and, T. Huang, Detection and estimation of pointing gestures in dense disparity maps in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. N. Jojic, J. Gu, H. C. Shen, and, T. Huang, 3-D Reconstruction of multipart self-occluding objects, in, Asian Conference on Computer Vision, 1998. Ju, 1996, Human Motion Estimation and Recognition (Depth Oral Report) S. X. Ju, M. J. Black, and, Y. Yacoob, Cardboard people: A parameterized model of articulated image motion, in, International Conference on Automatic Face and Gesture Recognition, Killington, VT, 1996. Kahn, 1996, Gesture Recognition Using the Perseus Architecture I. Kakadiaris and D. Metaxas, Vision-based animation of digital humans, in Conference on Computer Animation, 1998, pp. 144–152. I. A. Kakadiaris and D. Metaxas, 3D human body model acquisition from multiple views, in International Conference on Computer Vision, Cambridge, MA, June 20–23, 1995, pp. 618–623. I. A. Kakadiaris, and, D. Metaxas, Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection, in, Conference on Computer Vision and Pattern Recognition, 1996. Y. Kameda, and, M. Minoh, A human motion estimation method using 3-successive video frames, in, International Conference on Virtual Systems and Multimedia, 1996. Y. Kameda, M. Minoh, and, K. Ikeda, Three dimensional pose estimation of an articulated object from its silhouette image, in, Asian Conference on Computer Vision, 1993. Y. Kameda, M. Minoh, and, K. Ikeda, Three dimensional motion estimation of a human body using a difference image sequence, in, Asian Conference on Computer Vision, 1995. T. M. Kepple, MOVE3D-Software for analyzing human motion, in, Proc. of Johns Hopkins National Search for Computing Applications to Assist Persons with Disabilities, Laurel, MD, February 1992. Koffka, 1935, Principle of Gestalt Psychology Lee, 1985, Determination of 3D human body posture from a single view, Comp. Vision, Graphics, Image Process, 30, 148, 10.1016/0734-189X(85)90094-5 Lee, 1992, Knowledge-guided visual perception of 3-D human gait from a single image sequence, Trans. Systems, Man, Cybernetics, 22, 336, 10.1109/21.148408 Lengyel, 1998, The convergence of graphics and vision, Computer, 31, 46, 10.1109/2.689676 F. Lerasle, G. Rives, and, M. Dhome, Human body limbs tracking by multi-ocular vision, in, Scandinavian Conference on Image Analysis, Lappeenranta, Finland, 1997. Lerasle, 1999, Tracking of human limbs by multiocular vision, Comp. Vision Image Understanding, 75, 229, 10.1006/cviu.1999.0759 Leung, 1987, A region based approach for human body motion analysis, Pattern Recognition, 20, 321, 10.1016/0031-3203(87)90007-0 Leung, 1987, Human body motion segmentation in a complex scene, Pattern Recognition, 20, 55, 10.1016/0031-3203(87)90017-3 Leung, 1995, First sight: A human body outline labeling system, Trans. Pattern Anal. Mach. Intelligence, 17, 359, 10.1109/34.385981 Y. Li, S. Ma, and, H. Lu, Human posture recognition using multi-scale morphological method and Kalman motion estimation, in, International Conference on Pattern Recognition, 1998. Long, 1991, Log-Tracker: an attribute-based approach to tracking human body motion, Pattern Recognition Artificial Intelligence, 5, 439, 10.1142/S0218001491000259 Luc, 1996, Real time human action recognition for virtual environments, Computer Science Postgraduate Course Luo, 1992, An automatic rotoscopy system for human motion based on a biomechanic graphical model, Comput. Graphics, 16, 355, 10.1016/0097-8493(92)90021-M Marey, 1873, Animal Mechanism: A Treatise on Terrestrial and Aerial Locomotion S. J. McKenna, S. Jabri, Z. Duric, and, H. Wechsler, Tracking interacting people, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. D. Meyer, J. Denzler, and, H. Niemann, Model based extraction of articulated objects in image sequences, in, Fourth International Conference on Image Processing, 1997. Moeslund, 1999, Summaries of 107 Computer Vision-Based Human Motion Capture Papers T. B. Moeslund, Interacting with a virtual world through motion capture, in Interaction in Virtual Inhabited 3D WorldsL. Qvortrup, Ed., chap. 11, Springer-Verlag, Berlin/New York, 2000. T. B. Moeslund, and, E. Granum, 3D human pose estimation using 2D-data and an alternative phase space representation, in, Workshop on Human Modeling, Analysis and Synthesis at CVPR, Hilton Head Island, SC, June 2000. T. B. Moeslund, and, E. Granum, Multiple cues used in model-based human motion capture, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. Moezzi, 1996, Reality modeling and visualization from multiple video sequences, Comp. Graphics Appl, 16, 58, 10.1109/38.544073 O. Munkelt, C. Ridder, D. Hansel, and, W. Hafner, A model driven 3D image interpretation system applied to person detection in video images, in, International Conference on Pattern Recognition, 1998. E. Muybridge, Animal locomotion, reprinted in Animal in MotionL. S. Brown, Ed., Dover, New York, 1957. A. Nakazawa, H. Kato, and, S. Inokuchi, Human tracking using distributed video systems, in, International Conference on Pattern Recognition, 1998. P. J. Narayanan, P. W. Rander, and, T. Kanade, Constructing virtual worlds using dense stereo, in, International Conference on Computer Vision, Bombay, India, January 1998. S. A. Niyogi, and, E. H. Adelson, Analyzing and recognizing walking figures in XYT, in, Conference on Computer Vision and Pattern Recognition, 1994. J. Njåstad, S. Grinaker, and, G. A. Storhaug, Estimating parameters in a 212D human model, in, 11th Scandinavian Conference on Image Analysis, Kangerlussuaq, Greenland, 1999. Nordlund, 1998, Figure-Ground Segmentation Using Multiple Cues Ohya, 1999, Virtual metamorphosis, MultiMedia, 6, 29, 10.1109/93.771371 R. Okada, Y. Shirai, and, J. Miura, Tracking a person with 3-D motion by integrating optical flow and depth, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. E. J. Ong, and, S. Gong, Tracking hybrid 2D-3D human models from multiple views, in, International Workshop on Modeling People at ICCV'99, Corfu, Greece, September 1999. M. Oren, C. Papageorigiou, P. Sinha, E. Osuna, and, T. Poggio, Pedestrian detection using wavelet templates, in, Conference on Computer Vision and Pattern Recognition, 1997. O'Rourke, 1980, Model-based image analysis of human motion using constraint propagation, Trans. Pattern Anal. Mach. Intelligence, 2, 522, 10.1109/TPAMI.1980.6447699 V. Pavlović, J. M. Rehg, T. J. Cham, and, K. P. Murphy, A dynamic Bayesian network approach to figure tracking using learned dynamic models, in, International Conference on Computer Vision, Corfu, Greece, September 1999. Pavlovic, 1997, Visual interpretation of hand gestures for human-computer interaction: a review, Trans. Pattern Anal. Mach. Intelligence, 19, 677, 10.1109/34.598226 F. J. Perales and J. Torres, A system for human motion matching between synthetic and real images based on a biomechanic graphical model, in Workshop on Motion of Non-Rigid and Articulated Objects Austin, TX, 1994, pp. 83–88. C. Pinhanez, and, A. Bobick, Using computer vision to control a reactive computer graphics character in a theater play, in, International Conference on Vision Systems, 1998. R. Plänkers, P. Fua, and, N. D'Apuzzo, Automated body modeling from video sequences, in, International Workshop on Modeling People at ICCV'99, Corfu, Greece, 1999. R. Polana, and, R. Nelson, Low level recognition of human motion, in, Workshop on Motion of Non-Rigid and Articulated Objects, Austin, TX, October 1994. G. Rigoll, S. Eickeler, and, S. Müller, Person tracking in real world scenarios using statiscal methods, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. J. Rittscher, and, A. Blake, Classification of human body motion, in, International Conference on Computer Vision, Corfu, Greece, September 1999. Rohr, 1997, Human Movement Analysis Based on Explicit Motion Models, 171 R. Rosales, and, S. Sclaroff, Learning and synthesizing human body motion and posture, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. Rossi, 1994, Tracking and Counting Moving People Schneider, 1994, Tracking of Human Motion H. Segawa, H. Shioya, N. Hiraki, and, T. Totsuka, Constraint-conscious smoothing framework for the recovery of 3D articulated motion from image sequences, in The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. H. Segawa, and, T. Totsuka, Torque-based recursive filtering approach to the recovery of 3D articulated motion from image sequences, in, International Conference on Computer Vision, Corfu, Greece, September 1999. Shio, 1991, Segmentation of people in motion, Workshop on Visual Motion, 325, 10.1109/WVM.1991.212768 H. Sidenbladh, F. De la Torre, and, M. J. Black, A framework for modeling the appearance of 3D articulated figures, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. M. C. Silaghi, R. Plänkers, R. Boulic, P. Fua, and, D. Thalmann, Local and global skeleton fitting techniques for optical motion capture, in, International Workshop on Modelling and Motion Capture Techniques for Virtual Environments, Geneva, Switzerland, November 1998. Sul, 1998, Virtual stage: a location-based karaoke system, Multimedia, 5, 42, 10.1109/93.682524 Sumi, 1984, Upside-down presentation of the Johnsson moving light-spot pattern, Perception, 13, 283, 10.1068/p130283 A. Tesei, G. L. Foresti, and, C. S. Regazzoni, Human body modeling for people localization and tracking from real image sequences, in, Image Processing and Its Applications, July 1995. Tsukiyama, 1985, Detection of the movements of persons from a sparse sequence of TV images, Pattern Recognition, 18, 207, 10.1016/0031-3203(85)90046-9 M. Turk, Visual Interaction with lifelike characters, in, International Conference on Automatic Face and Gesture Recognition, Killington, VT, 1996. A. Utsumi, H. Mori, J. Ohya, and, M. Yachida, Multiple-view-based tracking of multiple humans, in, International Conference on Pattern Recognition, 1998. S. Wachter, and, H.-H. Nagel, Tracking of persons in monocular image sequences, in, Workshop on Motion of Non-Rigid and Articulated Objects, Puerto Rico, 1997. Wachter, 1999, Tracking persons in monocular image sequences, Comp. Vision Image Understanding, 74, 174, 10.1006/cviu.1999.0758 J. Wang, G. Lorette, and, P. Bouthemy, Analysis of human motion: a model-based approach, in, Scandinavian Conference on Image Analysis, 1991. Wang, 1992, Human motion analysis with detection of sub-part deformations, SPIE-Biomedical Image Processing and Three-Dimensional Microscopy, 1660, 329 Wren, 1997, Pfinder: real-time tracking of the human body, Trans. Pattern Anal. Mach. Intelligence, 19, 780, 10.1109/34.598236 C. R. Wren, B. P. Clarkson, and, A. P. Pentland, Understanding purposeful human motion, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. C. R. Wren, and, A. P. Pentland, Dynaman: recursive modeling of human motion, Image Vision Comp, in press. C. R. Wren, and, A. P. Pentland, Dynamic models of human motion, in, International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998. C. R. Wren, and, A. P. Pentland, Understanding purposeful human motion, in, International Workshop on Modeling People at ICCV'99, Corfu, Greece, September 1999. C. R. Wren, et al., Perceptive spaces for performance and entertainment, in, ATR Workshop on Virtual Communication Environments: Bridges over Art/Kansei and VR Technologies, Kyoto, Japan, April 13, 1998. A. Wu, M. Shah, and, N. Lobo, A virtual 3D blackboard: 3D finger tracking using a single camera, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. Y. Yacoob, and, M. J. Black, Parameterized modeling and recognition of activities, in, International Conference on Computer Vision, Bombay, India, 1998. M. Yamada, K. Ebihara, and, J. Ohya, A new robust real-time method for extracting human silhouettes from color images, in, International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998. M. Yamamoto, T. Kondo, T. Yamagiwa, and, K. Yamanaka, Skill recognition, in, International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998. M. Yamamoto, and, K. Koshikawa, Human motion analysis based on a robot arm model, in, Conference on Computer Vision and Pattern Recognition, 1991. M. Yamamoto, Y. Ohta, T. Yamagiwa, and, K. Yagishita, Human action tracking guided by key-frames, in, The Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France, March 2000. C. Yaniz, J. Rocha, and, F. Perales, 3D region graph for reconstruction of human motion, in, Workshop on Perception of Human Motion at ECCV, 1998. J. Y. Zheng, and, S. Suezaki, A model based approach in extracting and generating human motion, in, International Conference on Pattern Recognition, 1998.