MIcro-surgical anastomose workflow recognition challenge report

Huaulmé，Arnaud1; Sarikaya，Duygu2; Le Mut，Kévin1; Despinoy，Fabien1; Long，Yonghao3,4; Dou，Qi3,4; Chng，Chin Boon5,6; Lin，Wenjun5,6; Kondo，Satoshi7; Bravo-Sánchez，Laura8; Arbeláez，Pablo8; Reiter，Wolfgang9; Mitsuishi，Manoru10; Harada，Kanako10; Jannin，Pierre1

2021-11-01

10.1016/j.cmpb.2021.106452

COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE   影响因子和分区

0169-2607

1872-7565

Background and Objective: Automatic surgical workflow recognition is an essential step in developing context-aware computer-assisted surgical systems. Video recordings of surgeries are becoming widely accessible, as the operational field view is captured during laparoscopic surgeries. Head and ceiling mounted cameras are also increasingly being used to record videos in open surgeries. This makes videos a common choice in surgical workflow recognition. Additional modalities, such as kinematic data captured during robot-assisted surgeries, could also improve workflow recognition. This paper presents the design and results of the MIcro-Surgical Anastomose Workflow recognition on training sessions (MISAW) challenge whose objective was to develop workflow recognition models based on kinematic data and/or videos. Methods: The MISAW challenge provided a data set of 27 sequences of micro-surgical anastomosis on artificial blood vessels. This data set was composed of videos, kinematics, and workflow annotations. The latter described the sequences at three different granularity levels: phase, step, and activity. Four tasks were proposed to the participants: three of them were related to the recognition of surgical workflow at three different granularity levels, while the last one addressed the recognition of all granularity levels in the same model. We used the average application-dependent balanced accuracy (AD-Accuracy) as the evaluation metric. This takes unbalanced classes into account and it is more clinically relevant than a frame-by-frame score. Results: Six teams participated in at least one task. All models employed deep learning models, such as convolutional neural networks (CNN), recurrent neural networks (RNN), or a combination of both. The best models achieved accuracy above 95%, 80%, 60%, and 75% respectively for recognition of phases, steps, activities, and multi-granularity. The RNN-based models outperformed the CNN-based ones as well as the dedicated modality models compared to the multi-granularity except for activity recognition. Conclusion: For high levels of granularity, the best models had a recognition rate that may be sufficient for applications such as prediction of remaining surgical time. However, for activities, the recognition rate was still low for applications that can be employed clinically. The MISAW data set is publicly available at http://www.synapse.org/MISAW to encourage further research in surgical workflow recognition.

Multi-modality OR of the future Surgical process model Workflow recognition

SCI ; EI

英语

其他

Computer Science ; Engineering ; Medical Informatics

Computer Science, Interdisciplinary Applications ; Computer Science, Theory & Methods ; Engineering, Biomedical ; Medical Informatics

WOS:000720354700010

ELSEVIER IRELAND LTD

20214311053714

Blood vessels ; Brain ; Convolutional neural networks ; Kinematics ; Surgery ; Video recording

Biomedical Engineering:461.1 ; Biological Materials and Tissue Engineering:461.2 ; Medicine and Pharmacology:461.6 ; Television Systems and Equipment:716.4 ; Mechanics:931.1

COMPUTER SCIENCE

2-s2.0-85117423187

Scopus

1.Univ Rennes,INSERM,Rennes,LTSI - UMR 1099,F35000,France
2.Gazi University,Faculty of Engineering; Department of Computer Engineering,Ankara,Turkey
3.Department of Computer Science & Engineering,The Chinese University of Hong Kong,China
4.T Stone Robotics Institute,The Chinese University of Hong Kong,China
5.National University of Singapore(NUS),Singapore,Singapore
6.Southern University of Science and Technology (SUSTech),Shenzhen,China
7.Konica Minolta,Inc,Tokyo,Japan
8.Center for Research and Formation in Artificial Intelligence,Department of Biomedical Engineering,Universidad de los Andes,Bogotá,Colombia
9.Wintegral GmbH,München,Germany
10.Department of Mechanical Engineering,the University of Tokyo,Tokyo,113-8656,Japan

Huaulmé，Arnaud,Sarikaya，Duygu,Le Mut，Kévin,et al. MIcro-surgical anastomose workflow recognition challenge report[J]. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE,2021,212.

Huaulmé，Arnaud.,Sarikaya，Duygu.,Le Mut，Kévin.,Despinoy，Fabien.,Long，Yonghao.,...&Jannin，Pierre.(2021).MIcro-surgical anastomose workflow recognition challenge report.COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE,212.

Huaulmé，Arnaud,et al."MIcro-surgical anastomose workflow recognition challenge report".COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 212(2021).