Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
aws ·
The article describes building a Physical AI model factory using NVIDIA Cosmos 3 on Amazon SageMaker HyperPod, where a continuous pipeline loop generates synthetic data, post-trains perception and policy models, and evaluates them in closed-loop simulation. Cosmos 3 is an open omnimodal world foundation model with Mixture-of-Transformers architecture and a single token stream, letting one model handle video, image, action, and sound in three modes: forward dynamics for video generation, inverse dynamics for action labeling, and deployable policy. This unification allows generation, post-training, and evaluation to run as scheduled workloads on one shared GPU node pool rather than separate per-stage capacity, with cost governed by GPU goodput across the whole loop. The authors map Cosmos 3 design choices to SageMaker HyperPod with Amazon EKS, emphasizing a single persistent cluster, shared FSx for Lustre storage backed by S3, and health-checked auto-recovering nodes. They outline a four-stage flywheel: ingest and curate real data, generate synthetic data with a Cosmos3-Super teacher, post-train a Cosmos3-Nano policy, and evaluate in closed-loop simulation whose failures re-enter the corpus. Model tiers include Nano 16B, Super 64B, and Edge 4B, with post-trained variants such as Cosmos3-Nano-Policy-DROID. The article includes an end-to-end robot-policy walkthrough on the public DROID dataset and links to a GitHub repository with manifests and configuration files.
aws