Surgical phase classification and operative skill assessment through spatial context aware CNNs and time-invariant feature extracting autoencoders

Biocybernetics and Biomedical Engineering - Tập 43 - Trang 700-724 - 2023
Chakka Sai Pradeep1, Neelam Sinha1
1International Institute of Information Technology Bangalore, 26/C, Hosur Rd, Electronics City Phase 1, Electronic City, Bangalore 560100, Karnataka, India

Tài liệu tham khảo

Jin, 2018, Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks, 691

Garrow, 2021, Machine learning for surgical phase recognition: A systematic review, Ann Surgery, 273, 684, 10.1097/SLA.0000000000004425

Saeidi, 2022, Autonomous robotic laparoscopic surgery for intestinal anastomosis, Sci Robot, 7, eabj2908, 10.1126/scirobotics.abj2908

Jahmunah, 2023, Endoscopy, video capsule endoscopy, and biopsy for automated celiac disease detection: A review, Biocybernet Biomed Eng, 43, 82, 10.1016/j.bbe.2022.12.002

Demir KC, Schieber H, Weise T, et al. Deep Learning in Surgical Workflow Analysis: A review; 2022. URL: https://www.techrxiv.org/articles/preprint/Surgical_Phase_Recognition_A_Review_and_Evaluation_of_Current_Approaches/19665717. https://doi.org/10.36227/techrxiv.19665717.v2.

[dataset] m2cai16 public dataset, http://camma.u-strasbg.fr/datasets.

[dataset] Cholec80 public dataset, http://camma.u-strasbg.fr/datasets.

Wagner, 2023, Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the heichole benchmark, Med Image Anal, 102770, 10.1016/j.media.2023.102770

Schoeffmann, 2018, Cataract-101: video dataset of 101 cataract surgeries, 421

Al Hajj, 2019, Cataracts: Challenge on automatic tool annotation for cataract surgery, Med Image Anal, 52, 24, 10.1016/j.media.2018.11.008

Kitaguchi, 2020, Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach, Surg Endosc, 34, 4924, 10.1007/s00464-019-07281-0

Mohammad, 2015, Brace for variability in tool positioning: Modeling and simulation of 1 dof needle insertion task under tool-braced condition, Biocybernet Biomed Eng, 35, 128, 10.1016/j.bbe.2014.10.003

Arpaia, 2022, Assessment of blood perfusion quality in laparoscopic colorectal surgery by means of machine learning, Sci Rep, 12, 14682, 10.1038/s41598-022-16030-8

Abbing, 2023, Towards an ai-based assessment model of surgical difficulty during early phase laparoscopic cholecystectomy, Comput Methods Biomech Biomed Eng: Imag Visual, 1

Zhu, 2023, A prospective study using propensity score matching to compare long-term survival outcomes after robotic-assisted, laparoscopic, or open liver resection for patients with bclc stage 0-a hepatocellular carcinoma, Ann Surg, 277, 10.1097/SLA.0000000000005380

Bu, 2020, Effect of laparoscopic surgery for colorectal cancer with n. o. s. e. on recovery and prognosis of patients, Minim Invasive Ther Allied Technol, 31, 230, 10.1080/13645706.2020.1799410

Kano, 2023, Risk factors associated with venous thromboembolism in laparoscopic surgery in non-obese patients with benign disease, Surg Endosc, 37, 592, 10.1007/s00464-022-09361-0

Huang, 2022, Laparoscopic vs open distal gastrectomy for locally advanced gastric cancer: Five-Year outcomes from the CLASS-01 randomized clinical trial, JAMA Surg, 157, 9, 10.1001/jamasurg.2021.5104

Wang, 2022, Comparative effectiveness of enhanced recovery after surgery program combined with Single-Incision laparoscopic surgery in colorectal cancer surgery: A retrospective analysis, Front Oncol, 11, 768299, 10.3389/fonc.2021.768299

Zhou, 2023, Application of a highly simulated and adaptable training system in the laparoscopic training course for surgical residents: Experience from a high-volume teaching hospital in China, Heliyon, 9, e13317, 10.1016/j.heliyon.2023.e13317

Haug, 2022, Development of a procedure-specific tool for skill assessment in left- and right-sided laparoscopic complete mesocolic excision, Colorectal Dis, 25, 31, 10.1111/codi.16317

Al-Hubaishi, 2023, Video-based assessment (VBA) of an open, simulated orthopedic surgical procedure: a pilot study using a single-angle camera to assess surgical skill and decision making, J Orthop Surg Res, 18, 90, 10.1186/s13018-023-03557-3

van den Broek, 2023, Video grading of pancreatic anastomoses during robotic pancreatoduodenectomy to assess both learning curve and the risk of pancreatic fistula - a post hoc analysis of the LAELAPS-3 training program, Ann Surg, 10.1097/SLA.0000000000005796

Fung, 2022, Outcome and learning curve for laparoscopic intra-corporeal inguinal hernia repair in children, Surg Endosc, 37, 434, 10.1007/s00464-022-09530-1

Twinanda, 2017, Endonet: A deep architecture for recognition tasks on laparoscopic videos, IEEE Trans Med Imag, 36, 86, 10.1109/TMI.2016.2593957

Jin, 2018, Sv-rcnet: Workflow recognition from surgical videos using recurrent convolutional network, IEEE Trans Med Imag, 37, 1114, 10.1109/TMI.2017.2787657

Jin, 2020, Multi-task recurrent convolutional network with correlation loss for surgical video analysis, Med Image Anal, 59, 101572, 10.1016/j.media.2019.101572

Pradeep, 2022, Multi-tasking dssd architecture for laparoscopic cholecystectomy surgical assistance systems, 1

Jalal, 2023, Analysing multi-perspective patient-related data during laparoscopic gynaecology procedures, Sci Rep, 13, 1604, 10.1038/s41598-023-28652-7

Padovan, 2022, A deep learning framework for real-time 3D model registration in robot-assisted laparoscopic surgery, Int J Med Robot, 18, e2387, 10.1002/rcs.2387

Pradeep CS, Sinha N. Spatio-temporal features based surgical phase classification using cnns. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC); 2021, pp. 3332–3335.

Czempiel, 2020, Tecno: Surgical phase recognition with multi-stage temporal convolutional networks, 343

Jin, 2021, Temporal memory relation network for workflow recognition from surgical video, IEEE Trans Med Imag, 40, 1911, 10.1109/TMI.2021.3069471

Czempiel T, Paschali M, Ostler D, et al. Opera: Attention-regularized transformers for surgical phase recognition. In: de Bruijne M, Cattin PC, Cotin S, et al. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021; 2021, pp. 604–614.

Kadkhodamohammadi, 2022, Patg: position-aware temporal graph networks for surgical phase recognition on laparoscopic videos, Int J Comput Assist Radiol Surg, 17, 849, 10.1007/s11548-022-02600-8

Ding, 2022, Exploring segment-level semantics for online phase recognition from surgical videos, IEEE Trans Med Imag, 41, 3309, 10.1109/TMI.2022.3182995

Jin, 2022, Trans-SVNet: hybrid embedding aggregation transformer for surgical workflow analysis, Int J Comput Assist Radiol Surg, 17, 2193, 10.1007/s11548-022-02743-8

Park, 2022, Multi-stage temporal convolutional network with moment loss and positional encoding for surgical phase recognition, Diagnostics, 13, 107, 10.3390/diagnostics13010107

Jalal, 2021, A deep learning framework for recognising surgical phases in laparoscopic videos, IFAC-PapersOnLine, 54, 334, 10.1016/j.ifacol.2021.10.278

Jalal, 2023, Laparoscopic video analysis using temporal, attention, and Multi-Feature fusion Based-Approaches, Sensors (Basel), 23, 1, 10.3390/s23041958

Pan, 2022, Temporal-based swin transformer network for workflow recognition of surgical video, Int J Comput Assist Radiol Surg, 18, 139, 10.1007/s11548-022-02785-y

Zhang, 2022, Surgical workflow recognition with temporal convolution and transformer for action segmentation, Int J Comput Assist Radiol Surg, 1

Deng, 2009, Imagenet: A large-scale hierarchical image database, 248

Howard AG, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. volume abs/1704.04861; 2017. URL: http://arxiv.org/abs/1704.04861.

Wang RJ, Li X, Ling CX. Pelee: A real-time object detection system on mobile devices. In: Bengio S, Wallach H, Larochelle H, et al. (Eds.), Advances in Neural Information Processing Systems, volume 31; 2018.

He, 2016, Deep residual learning for image recognition, 770

Huang, 2017, Densely connected convolutional networks, 2261

Szegedy, 2015, Going deeper with convolutions, 1

Szegedy, 2017, Inception-v4, inception-resnet and the impact of residual connections on learning, 4278

Hu, 2018, Squeeze-and-excitation networks, 7132

Tan, 2021, Efficientnetv2: Smaller models and faster training, 10096

Smith, 2017, Cyclical learning rates for training neural networks, 464

Li, 2022, Application of improved robot-assisted laparoscopic telesurgery with 5G technology in urology, Eur Urol, 83, 41, 10.1016/j.eururo.2022.06.018

Zhuang, 2021, A comprehensive survey on transfer learning, Proc IEEE, 109, 43, 10.1109/JPROC.2020.3004555

Mendes, 2020, Multi-stage transfer learning with an application to selection process, 1295

Zhou, 2016, Learning deep features for discriminative localization, 2921

Selvaraju, 2017, Grad-cam: Visual explanations from deep networks via gradient-based localization, 618

Chattopadhay, 2018, Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks, 839

Wang, 2020, Score-cam: Score-weighted visual explanations for convolutional neural networks, 111

Jiang, 2021, Layercam: Exploring hierarchical class activation maps for localization, IEEE Trans Image Process, 30, 5875, 10.1109/TIP.2021.3089943

Tan, 2019, Rethinking model scaling for convolutional neural networks, 6105

Lam, 2022, Machine learning for technical skill assessment in surgery: a systematic review, npj Digital Med, 5, 24, 10.1038/s41746-022-00566-0

Goodfellow, 2016

Pradeep CS, Sinha N. Fused multilayer layer-cam fine-grained spatial feature supervision for surgical phase classification using cnns. In: Karlinsky L, Michaeli T, Nishino K, editors. Computer Vision – ECCV 2022 Workshops; 2023, pp. 712–726.

Paszke, 2016, Enet: A deep neural network architecture for real-time semantic segmentation, 321

De Ryck, 2021, Change point detection in time series data using autoencoders with a time-invariant representation, IEEE Trans Signal Process, 69, 3513, 10.1109/TSP.2021.3087031

Lee W, Ortiz J, Ko B, Lee RB. Time series segmentation through automatic feature learning. volume abs/1801.05394; 2018. URL: http://arxiv.org/abs/1801.05394.

Naitzat, 2022, Topology of deep neural networks, J Mach Learn Res, 21, 7503

Muller, 2021, Tuning-free yet state-of-the-art data augmentation, 754

Thông tin
Thông tin xuất bản

Biocybernetics and Biomedical Engineering

Tập 43

700-724

Thông tin tác giả