NVIDIA Cosmos3-H-Dreams: Bringing Clinical Surgery to Life Through Generative Simulation
huggingface ·
NVIDIA has introduced Cosmos3-H-Dreams, a generative simulation platform designed to bring clinical surgical procedures to life through interactive, real-time rendering. Building upon previous tabletop robotics models, this new system is trained on video and kinematic data from actual cholecystectomy procedures performed on the CMR Surgical Versius platform. The model leverages the Cosmos 3 foundation and FlashDreams acceleration library to run interactively on a single NVIDIA RTX PRO 6000 GPU. It utilizes a teacher-to-student distillation pipeline, offering both four-step and two-step inference checkpoints to balance visual fidelity with generation speed. By processing complex anatomical responses such as tissue deformation and occlusion, the simulator maintains temporal consistency while responding to live instrument controls. Researchers can adapt the framework to their own datasets using provided fine-tuning recipes and self-forcing distillation guides. While currently positioned as a research tool rather than a validated clinical device, it opens pathways for exploring autonomous surgical policies and hybrid numerical-generative simulations.
huggingface