Object Tracking With a Range Camera for Augmented Reality Assembly Assistance

Rafael Radkowski1
1Mem. ASME Assistant Professor Department of Mechanical Engineering, Iowa State University, Ames, IA 50011 e-mail:

Tóm tắt

This paper introduces a 3D object tracking method for an augmented reality (AR) assembly assistance application. The tracking method relies on point clouds; it uses 3D feature descriptors and point cloud matching with the iterative closest points (ICP) algorithm. The feature descriptors identify an object in a point cloud; ICP align a reference object with this point cloud. The challenge is to achieve high fidelity while maintaining camera frame rates. The point cloud and reference object sampling density are one of the key factors to meet this challenge. In this research, three-point sampling methods and two-point cloud search algorithms were compared to assess their fidelity when tracking typical products of mechanical engineering. The results indicate that a uniform sampling maintains the best fidelity at camera frame rates.

Từ khóa


Tài liệu tham khảo

1997, A Survey of Augmented Reality, Presence Teleoperators Virtual Environ., 6, 355, 10.1561/1100000049

2015, Augmented Reality Based Manual Assembly Support With Visual Features for Different Degree of Difficulty, J. Hum. Comput. Interact., 31, 337, 10.1080/10447318.2014.994194

Henderson, S., and Feiner, S., 2011, “Augmented Reality in the Psychomotor Phase of a Procedural Task,” 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 191–200.

2013, A Study on the Benefits of Augmented Reality in Retaining Working Memory in Assembly Tasks: A Focus on Differences in Gender, Autom. Constr., 32, 38, 10.1016/j.autcon.2012.12.007

Westerfield, G., Mitrovic, A., and Billinghurst, M., 2013, “Intelligent Augmented Reality Training for Assembly Tasks,” 16th International Conference on Artificial Intelligence in Education (AIED 2013), H. Chad, Land, K. Yacef, J. Mostow, and P. Pavlik, eds., pp. 542–551.

Peniche, A., Treffetz, H., Diaz, C., and Paramo, G., 2012, “Combing Virtual and Augmented Reality to Improve the Mechanical Assembly Training Process in Manufacturing,” American Conference on Applied Mathematics, pp. 292–297.

2008, Augmented Reality for Assembly Guidance Using a Virtual Interactive Tool, Int. J. Prod. Res., 46, 1745, 10.1080/00207540600972935

2003, Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation, Int. J. Hum. Comput. Interact., 16, 497, 10.1207/S15327590IJHC1603_7

Kato, H., and Billinghurst, M., 1999, “Marker Tracking and HMD Calibration for a Video-Based Augmented Reality Conferencing System,” 2nd International Workshop on Augmented Reality (IWAR 99), pp. 85–94.

1998, Natural Feature Tracking for Augmented-Reality, IEEE Transactions on Multimedia, 53, 10.1109/6046.748171

Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., and Davison, A. J., 2013, “Slam++: Simultaneous Localisation and Mapping at the Level of Objects,” 2013 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1352–1359.

Radkowski, R., and Oliver, J., 2013, “Natural Feature Tracking Augmented Reality for On-Site Assembly Assistance Systems,” Human Computer Interaction International Conference, pp. 281–290.

1992, A Method for Registration of 3D-Shapes, PAMI., 14, 239, 10.1109/34.121791

Raczynski, A., and Gussmann, P., 2004, “Services and Training Through Augmented Reality,” 1st European Conference on Visual Media Production (CVMP’04), pp. 263–271.

Zenati, N., Zerhouni, N., and Achour, K., 2004, “Assistance to Maintenance in Industry Process Using an Augmented Reality System,” 2004 IEEE international Conference on Industrial Technology (KIT), Vol. 2, pp. 848–852.

Reiners, D., Stricker, D., Klinker, G., and Müller, S., 1998, “Augmented Reality for Construction Tasks: Doorlock Assembly,” 1st International Workshop on Augmented Reality (IWAR 98), pp. 31–46.

Neumann, U., and Majoros, A., 1998, “Cognitive, Performance, and Systems Issues for Augmented Reality Applications in Manufacturing and Maintenance,” Virtual Reality Annual International Symposium, pp. 4–11.

Robertson, C., MacIntyre, B., and Walker, B., 2008, “An Evaluation of Graphical Context When the Graphics Are Outside of the Task Area,” 7th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2008), IEEE Computer Society, Washington, DC, pp. 73–76.

2014, The Affectiveness of an AR-Based Context-Aware Assembly Support System in Object Assembly, IEEE Virtual Reality (VR), 57, 10.1109/VR.2014.6802051

Radkowski, R., and Oliver, J., 2014, “Monocular Depth Cues for Augmented Reality Applications to Enhance Spatial Perception Tasks,” Tools and Methods of Competitive Engineering (TMCE 2014), pp. 749–768.

Wang, Y., Shen, Y., Liu, D., Wei, S., and Zhu, C., 2010, “Key Technique of Assembly System in an Augmented Reality Environment,” 2010 Second International Conference on Computer Modeling and Simulation, pp. 133–137.

Song, J., Jian, Q., Sun, H., and Gao, X., 2009, “Study of the Perception Mechanisms and Method of Virtual and Real Objects in Augmented Reality Assembly Environments,” 4th IEEE Conference on Industrial Electronics and Applications, pp. 1452–1456.

1999, Evaluating the Effectiveness of Augmented Reality Displays for a Manual Assembly Task, Virtual Reality, 4, 250, 10.1007/BF01421808

Korn, O., Schmidt, A., and Hörz, T., 2013, “Augmented Manufacturing: A Study With Impaired Persons on Assistive Systems Using In-Situ Projection,” 6th International Conference on Pervasive Technologies Related to Assistive Environment (PETRA 2013), pp. 21:1–21:8.

Tang, A., Owen, C., Biocca, F., and Mou, W., 2003, “Comparative Effectiveness of Augmented Reality in Object Assembly,” Conference on Human Factors in Computing Systems (CHI’03), pp. 73–80.

2011, An Augmented Reality Training Platform for Assembly and Maintenance Skills, J. Rob. Auton. Syst., 61, 398, 10.1016/j.robot.2012.09.013

Nilsson, S., and Johansson, B., 2007, “Fun and Usable: Augmented Reality Instructions in a Hospital Setting,” 19th Australasian Conference on Computer-Human Interaction (OZCHI 07), pp. 123–130.

2012, Interactive 3D Model Acquisition and Tracking of Building Block Structures, IEEE Trans. Visualization Comput. Graphics, 18, 651, 10.1109/TVCG.2012.48

Gupta, A., Fox, D., Curless, B., and Cohen, M., 2012, “Duplotrack: A Real-Time System for Authoring and Guiding Duplo Block Assembly,” 25th Annual ACM Symposium on User Interface Software and Technology, pp. 389–402.

Newcombe, R. A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohli, P., Shotton, J., Hodges, S., and Fitzgibbon, A., 2011, “Kinectfusion: Real-Time Dense Surface Mapping and Tracking,” International Symposium on Mixed and Augmented Reality, pp. 127–136.

Garrett, T., Debernardis, S., Radkowski, R., Chang, C. K., Fiorentino, M., Uva, A. E., and Oliver, J., 2014, “Rigid-Object Tracking for Real-Time Augmented Reality,” ASME 2014 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, pp. 1–10.

Rusinkiewicz, S., and Levoy, M., 2001, “Efficient Variants of the ICP Algorithm,” IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM), pp. 145–152.

Drost, B., Ulrich, M., Navab, N., and Ilic., S., 2010, “Model Globally, Match Locally: Efficient and Robust 3D Object Recognition,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 998–1005.

Turk, G., and Levoy, M., 1994, “Zippered Polytgon Meshes From Range Images,” SIGGRAPH '94, Computer Graphics Proceedings, pp. 311–318.

1996, Registration and Integration of Multiple Range Images for 3D-Model Construction, CVPR, 879, 10.1109/ICPR.1996.546150

Greenspan, M., and Yurick, M., 2003, “Approximate k-d Tree Search for Efficient ICP,” Fourth International Conference on 3-D Digital Imaging and Modeling (3DIM 2003), pp. 442–448.

1994, Three-Dimensional Registration Using Range and Intensity Information, Proc. SPIE, 279

1994, Iterative Point Matching for Registration of Free-Form Curves and Surfaces, Int. J. Comput. Vision, 13, 119, 10.1007/BF01427149

1987, Least-Squares Fitting of Two 3-D Point Sets, IEEE Trans. Pattern Anal. Mach. Intell., PAMI-9, 698, 10.1109/TPAMI.1987.4767965