Haptic Feedback of Flexible Endoscopic Surgical Robots

Robotic flexible endoscopic grasper with a three-axis force sensor

Haptic feedback is absent in flexible endoscopic surgical robots due to the size constraint of installing sensors on the small robotic arms. Besides, inherent hysteresis caused by the nonlinear friction between tendons and sheaths makes it hard to estimate the distal force by modeling. In this work, we addressed this challenge by proposing a new three-axial force sensor. This standalone device can be seamlessly integrated into the endoscopic robotic arm. Three optical fibers with Fiber Bragg Gratings (FBGs) are embedded in the sensing structure, where one is located at the center hole of the structure (1.4 mm), and the other two are eccentrically placed around the structure at 90 apart from each other. This device can measure the pulling force and lateral forces of an articulated surgical instrument. The sensor has a lateral force sensitivity of 838.386 pm/N, with a measurement resolution of 1.19 mN. See a demo here.

Deep learning for haptic force estimation

Accurate haptic feedback is highly challenging for flexible endoscopic surgical robots due to space limitation for sensors on small end-effectors and critical force hysteresis of their tendon-sheath mechanisms (TSMs). This paper proposes a deep learning approach to predicting the distal force of TSMs when manipulating a biological tissue based on only proximal-end measurements. Both Multilayer Perceptron (MLP) and Recurrent Neural Network (RNN) were investigated to study their capabilities of making sequential distal force predictions. The results were compared with those of the conventional modelling approach. It was observed that, when sufficient data was provided for training, RNN achieved the most accurate prediction (RMSE=0.0219 N) in experiments with constant system velocity. The effects of insufficient training data, varying system velocity and irregular motion trajectories on the performance of RNN were further studied. Notably, RNN could precisely identify the current system phase in the force hysteresis profile and can be applied to TSMs with realistic non-periodic movement such as manual manipulation trajectory (RSME=0.2287 N). The proposed approach can be applied to any TSM-driven robotic systems for accurate haptic feedback without requiring sensors at the distal ends of the robots.

© 2021 by LIN CAO