Rosota Intelligence Institute

Rosota Intelligence Institute

Pinned

PinnedPolicy learning

From Instrument-Mounted Demonstrations to In-Vivo Execution: Learning Bimanual Laparoscopic Appendectomy Without Robot-Collected Demonstrations

Most minimally invasive surgery is still performed with hand-held instruments, and the surgeon's instrument kinematics are lost when the operation ends. We present an end-to-end pipeline that captures that motion in the operating room and uses it to train a surgical robot policy: a shaft-mounted instrument-state logger, a data pipeline that measures and removes every channel’s latency, a diffusion policy selected by closed-loop rollouts in simulation, and deployment on live animals. Retrained on 849 in-vivo demonstrations, the policy completed the appendectomy in three of four live rabbits with electrosurgery armed.

Dongho Yee et al.

Timeline

Sensing

Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing

Most minimally invasive procedures are performed with hand-held instruments, yet only the endoscopic video is kept and the instrument motion is lost. A clip-on logger fuses a six-axis IMU and a time-of-flight rangefinder with a markerless camera in an error-state Kalman filter under the trocar’s remote centre of motion, with the shaft silhouette supplying heading in place of the magnetometer — whose field changed with tool pose and between sessions in our own measurements. Against a Franka Research 3, and without alignment to it, displacement error was 1.21 mm RMS over 300 trials and relative rotation 0.34° RMS over 180, with 1.22–3.04 mm absolute tip error on continuous trajectories tracked and displayed live.

Jiyul Lee et al.
Robotic ultrasound

Contact-Adaptive Robotic Ultrasound Probe Control for Tissue Exploration and Continuous Task-Relevant Visualization Using Robot-Free Image–Motion Demonstration

While a clinician performs the primary procedure, a robotic assistant can hold the view the clinician chose so that changing tissue stays observable. Sonologger clamps a six-axis IMU to the clinician-operated probe and records synchronized ultrasound and probe rotation with no robot, camera or tracker; rather than imitating trajectories, the recordings measure which probe axis and sign the clinician uses to correct an image offset. Those measurements parameterize an interpretable, non-learned controller, which acquired and held a task-relevant view in 7 of 7 blind poses on a dynamic bladder phantom against 1 of 7 for a static hold.

Seong Jeong et al.
Teleoperation

Surgical Master Console Using General-Purpose Robot Arms and a Separable Articulated Distal Interface: Porcine In-Vivo Evaluation

High-performance surgical master consoles are expensive and hard to reproduce, while commercial haptic devices lack interfaces for wrist articulation and continuous grasp. We build the master from general-purpose 6-DoF robot arms and concentrate everything surgical in a separable distal adapter — a 2-DoF direct-drive wrist, magnetic grasp sensing and clutch input — for about US$5,500 the bimanual pair. One endpoint-independent master state drives either Isaac Sim or an RCM-constrained dual-FR3 platform, with at most 8 ms from publication to target update, and the console was used throughout a porcine cholecystectomy without a single interruption originating from it.

Minsung Kim et al.
Sensing

Drape-Compatible Tool-Tip Localization for Hand-Held Laparoscopic Instruments via UWB Carrier-Phase Ranging and Trocar-Constrained Geometry

A draped instrument hides every optical path on itself, so we estimate the tool tips from distances and an IMU alone. Ultra-wideband carrier phase between nodes on the handle side of the instruments gives three distances; the trocar reduces each instrument to a single insertion depth, leaving one equation to spare that exposes a wrong integer instead of absorbing it. In an 11-hole phantom among metal instruments the three distances hold σ = 0.10–0.33 mm at 35–46 Hz, and a sterile drape pressed onto the antennas leaves the link unchanged.

Jinseok Lee et al.
Simulation

Contact-Stable Deformable Tissue Simulation Using Implicit Integration and Live-Pose Grasp Constraints for Laparoscopic Surgery Robot Policy Evaluation

Closed-loop evaluation of surgical robots needs tissue that deforms, can be grasped and lifted, and reproduces the anatomy the robot will work in. We reconstruct that tissue from a single fixed-view RGB-D recording of the surgical field — composited to remove the instruments, closed into watertight volumes and tetrahedralised — and place it in a bimanual cell where two Franka FR3 arms work through 6 mm trocars. The contribution is the numerical and contact design that keeps that cell stable: implicit integration, simulation meshes separate from collision meshes, numerical guards, and a grasp constraint captured at the live tissue pose.

Juahn Oh et al.