Real-time analysis of cataract surgery videos using statistical models
Tóm tắt
The automatic analysis of the surgical process, from videos recorded during surgeries, could be very useful to surgeons, both for training and for acquiring new techniques. The training process could be optimized by automatically providing some targeted recommendations or warnings, similar to the expert surgeon’s guidance. In this paper, we propose to reuse videos recorded and stored during cataract surgeries to perform the analysis. The proposed system allows to automatically recognize, in real time, what the surgeon is doing: what surgical phase or, more precisely, what surgical step he or she is performing. This recognition relies on the inference of a multilevel statistical model which uses 1) the conditional relations between levels of description (steps and phases) and 2) the temporal relations among steps and among phases. The model accepts two types of inputs: 1) the presence of surgical tools, manually provided by the surgeons, or 2) motion in videos, automatically analyzed through the Content Based Video retrieval (CBVR) paradigm. Different data-driven statistical models are evaluated in this paper. For this project, a dataset of 30 cataract surgery videos was collected at Brest University hospital. The system was evaluated in terms of area under the ROC curve. Promising results were obtained using either the presence of surgical tools (A
z
= 0.983) or motion analysis (A
z
= 0.759). The generality of the method allows to adapt it to other kinds of surgeries. The proposed solution could be used in a computer assisted surgery tool to support surgeons during the surgery.
Tài liệu tham khảo
André B, Vercauteren T, Buchner AM, Wallace MB, Ayache N (2012) Learning semantic and visual similarity for endomicroscopy video retrieval. IEEE Trans Med Imaging 31(6):1276–1288
Cao Y, Li M, Baang S, Hu S et al (2008) Medical video event classification using shared features. In: Tenth IEEE international symposium on multimedia, 2008. ISM 2008. IEEE, pp 266–273
Charrière K, Quellec G, Lamard M, Coatrieux G, Cochener B, Cazuguel G (2014) Automated surgical step recognition in norMalized cataract surgery videos. In: 36th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 4647–4650
Charrière K, Quellec G, Lamard M, Martiano D, Cazuguel G, Coatrieux G, Cochener B (2016) Real-time multilevel sequencing of cataract surgery videos. In: 2016 14th international workshop on content-based multimedia indexing (CBMI). IEEE, pp 1–6
Chattopadhyay T, Chaki A, Bhowmick B, Pal A (2008) An application for retrieval of frames from a laparoscopic surgical video based on image of query instrument. tencon 2008. In: IEEE region 10 conference, vol 11, pp 1–5
Fine S, Singer Y, Tishby N (1998) The hierarchical hidden markov model: analysis and applications. Mach Learn 32(1):41–62
Forestier G, Riffaud L, Jannin P (2015) Automatic phase prediction from low-level surgical activities. Int J Comput Assist Radiol Surg 1–9
Forney Jr GD (1973) The viterbi algorithm. Proc IEEE 61(3):268–278
Lafferty J, McCallum A, Pereira F (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning, ICML, vol 1, pp 282–289
Lalys F, Riffaud L, Bouget D, Jannin P (2012) A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans Biomed Eng 59(4):966–976
Lalys F, Bouget D, Riffaud L, Jannin P (2013) Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int J Comput Assist Radiol Surg 8(1):39–49
Laptev I (2005) On space-time interest points. Int J Comput Vis 64(2-3):107–123
Loukas C, Nikiteas N, Schizas D, Georgiou E (2016) Shot boundary detection in endoscopic surgery videos using a variational bayesian framework. Int J CARS 1–13
Lucas BD, Kanade T et al (1981) An iterative image registration technique with an application to stereo vision. In: IJCAI, vol 81, pp 674–679
Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N (2012) Statistical modeling and recognition of surgical workflow. Med Image Anal 16(3):632–641
Pearl J, Russell S (1998) Bayesian networks. Computer Science Department, University of California
Quellec G, Charrière K, Lamard M, Cochener B, Cazuguel G (2014) NorMalizing videos of anterior eye segment surgeries, In: Engineering in medicine and biology society (EMBC), 2014 36th annual international conference of the IEEE. IEEE, pp 122–125
Quellec G, Charrière K, Lamard M, Droueche Z, Roux C, Cochener B, Cazuguel G (2014) Real-time recognition of surgical tasks in eye surgery videos. Med Image Anal 18(3):579–590
Quellec G, Lamard M, Cochener B, Cazuguel G (2014) Real-time segmentation and recognition of surgical tasks in cataract surgery videos. IEEE Trans Med Imaging 33(12)
Quellec G, Lamard M, Cochener B, Cazuguel G (2015) Real-time task recognition in cataract surgery videos using adaptive spatiotemporal polynomials. IEEE Trans Med Imaging 34(4):877–887
Roberts CM (2006) Radio frequency identification (rfid). Comput Secur 25 (1):18–26
Sha F, Pereira F (2003) Shallow parsing with conditional random fields. In: Proceedings of the 2003 conference of the north american chapter of the association for computational linguistics on human language technology, vol 1, pp 134–141. Association for Computational Linguistics
Singh A, Strauss GH (2014) High-fidelity cataract surgery simulation and third world blindness. Surgical Innovation
Stanek SR, Tavanapong W, Wong J, Oh JH, De Groen PC (2012) Automatic real-time detection of endoscopic procedures using temporal features. Comput Methods Prog Biomed 108(2):524–535
Tao L, Zappella L, Hager GD, Vidal R (2013) Surgical gesture segmentation and recognition. In: Medical image computing and computer-assisted intervention–MICCAI 2013. Springer, pp 339–346
Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2016)
Wang H, Ullah MM, Klaser A, Laptev I, Schmid C (2009) Evaluation of local spatio-temporal features for action recognition. In: BMVC 2009 - British machine vision conference. London, Royaume-Uni. http://hal.inria.fr/inria-00439769. CLASS
Yao W, Chu CH, Li Z (2010) The use of rfid in healthcare: benefits and barriers. In: IEEE international conference on RFID-technology and applications (RFID-TA), 2010, pp 128–134
Zappella L, Béjar B, Hager G, Vidal R (2013) Surgical gesture classification from video and kinematic data. Med Image Anal 17(7):732–745