The research focuses on developing innovative solutions for minimally invasive surgery and robotic implantology, eliminating the need for complex hardware sensors through the use of advanced computer vision techniques.
Automatic localization and positioning of the ATHENA robot
- Objective: Real-time 3D localization without physical markers, enabling automatic docking and reducing setup variability. The system receives data from an RGB-D camera.
- Detection architecture: The YOLO11m model was used (selected over YOLO8m/9m/10m versions) for detecting three specific classes: the trocar, the laparoscopic instrument, and the robot’s parallel module (PM).
- 3D Estimation: The system converts pixel coordinates from 2D bounding boxes and depth information into 3D coordinates for key components.
- Performance and latency: Evaluation showed an mAP of approximately 0.9947, a precision of 0.9879, and a recall of 0.9849. Inference latency is approximately 14.7 ms, with an end-to-end latency of approximately 67 ms.
- Impact: The solution achieves positioning with an error margin of ≤ 0.8 mm and ensures a 42% reduction in setup time compared to manual alignment.
Sensorless force estimation in minimally invasive surgery
- Objective: Estimating instrument–tissue interaction forces exclusively from video data, eliminating the need for a distal sensor. The input consists of a single endoscopic RGB frame.
- Model: A lightweight CNN based on EfficientNetV2B0, adapted for force regression to provide a single scalar output in Newtons.
- Dataset: Training was based on 40 video clips (9,691 labeled frames) obtained from in vitro esophagus tests, with force labels in the range of 0–5 N. The ground truth was acquired using a Robotiq FT300 sensor mounted on a KUKA LBR iiwa 7 R800 robot.
- Technical performance: The model reports a mean absolute error (MAE) of 0.017 N and a mean squared error (MSE) of 0.0004 N².
- Deployment: Inference takes approximately 12.34 ms, operating at an update rate of about 6 Hz, with a prediction latency of 15–20 ms.
- Hardware integration: The algorithm operates as a plug-in on the PARA-SILSROB platform and controls a Force Dimension Omega.7 haptic device.
Markerless perception for navigation in dental implantology
- Dynamic navigation and control (objective 1): The system uses a YOLO detector trained on 752 intraoral images to identify anatomical landmarks. This detection achieves a precision of 91.2%, recall of 88.5%, and mAP@0.5 of 91.2%.
- Alignment and architecture: A rigid alignment method based on SVD is used, along with optional ICP refinement, to register the position to STL models generated preoperatively from CBCT scans. A Unity interface centralizes visualization, while a server sends navigation corrections to a KUKA Sunrise robot.
- Tooth-level segmentation (objective 2): To obtain stable tooth geometry in the presence of occlusions and reflections, the single-stage YOLOv8-seg model was implemented. Training was performed on 420 RGB images with manually annotated polygon labels.
- Segmentation performance: The system achieved an IoU of approximately 0.88 and a DSC of approximately 0.92. Instance-level evaluation (Mask mAP@0.5) was 0.907.
- Architectural advantage: The single-stage YOLO-seg pipeline produces significantly more stable masks and considerably lower latency compared to a YOLO+SAM-based system, making it suitable for real-time use.
