Surgical Hand Gesture Prediction for the Operating Room | JoVE Visualize

Surgical Hand Gesture Prediction for the Operating Room

Inna Skarga-Bandurova¹, Rostislav Siriak², Tetiana Biloborodova²

¹Oxford Brookes University.

Studies in Health Technology and Informatics|October 22, 2020

View abstract on PubMed

Summary

This study introduces a new deep learning model for predicting surgeon gestures using contactless interfaces. The GestureConvLSTM model enhances human-robot interaction and operating room efficiency by recognizing unfinished gestures.

Area of Science:

Medical Robotics
Artificial Intelligence
Human-Computer Interaction

Background:

Smart assistive technology allows contactless control of computer-aided devices in operating rooms.
Understanding surgeon actions is key for natural human-robot interaction and predicting intentions.
This improves robot response times and overall operating room efficiency.

Purpose of the Study:

To develop a deep network for gesture prediction to enable natural human-robot interaction.
To enhance operating room efficiency through improved surgeon-robot collaboration.
To reliably recognize unfinished gestures for real-time assistive applications.

Main Methods:

A novel deep network based on Convolution Long Short-Term Memory (ConvLSTM) was developed.
The network, termed GestureConvLSTM, is specifically configured for gesture prediction in surgical environments.
Performance was evaluated on the LSA64 dataset.

Main Results:

The GestureConvLSTM model demonstrated reliable recognition of unfinished gestures from video data.
Quantitative results confirmed the model's capability in predicting incomplete gestures.
The system showed improved performance compared to baseline systems on the LSA64 dataset.

Conclusions:

The proposed GestureConvLSTM model effectively predicts surgeon gestures, enabling more natural human-robot interaction.
This technology has the potential to significantly improve operating room efficiency and surgeon workflow.
The ability to recognize unfinished gestures is crucial for real-time assistive robotics in surgery.

Keywords: