Uploaded June 2026 | Updated September 2026, 2 weeks ago
In this tutorial, you will learn how to post-train NVIDIA Cosmos 3 into a forward dynamics model — an action-conditioned world foundation model that generates future video states given a robot's action trajectory. Post-training Cosmos 3 into a forward dynamics model lets you simulate what a robot will donext, which is a core building block for robotics policy development. By the end, you will have a post-trained Cosmos 3 model running inference and generating predicted future world states from real robot action data.
Get started with Cosmos 3:
📘 Read the technical blog → nvda.ws/4u5G0x8
🤗 Download models → huggingface.co/collections/nvidia/cosmos3
⚙️ Customize + post-train → github.com/nvidia/Cosmos
🚀 Deploy → build.nvidia.com/models?q=cosmos
Chapters:
00:00 Intro — What is Cosmos 3 action conditioning?
00:40 Architecture diagram — condition frames to future state and actions
01:10 Walk through the YAM dataset class in VS Code
01:35 Review the action joint SFT config (action_joint_sft_nano_yam.py)
02:10 Explain joint mode — training all three action heads simultaneously
02:55 Export the trained checkpoint with EMA weights
03:55 Run inference — predicted vs ground truth side by side
#NVIDIACosmos #WorldFoundationModels #PhysicalAI
NVIDIA Cosmos, World Foundation Models, Physical AI, forward dynamics model, action-conditioned model, Cosmos 3 post-training
Q: What is a forward dynamics model?
A: A forward dynamics model is a model that takes an action trajectory, a conditioning video frame, and a task description as input, and generates a video of the predicted future world state aligned with that trajectory.
Q: What datasets can I use to post-train Cosmos 3 for forward dynamics? A: Any dataset in the LeRobot format works. This tutorial uses a custom dataset collected on the YAM robot. You can convert your own robot data into the LeRobot format following the data preparation instructions in the Cosmos 3 GitHub repo.
Q: How many GPUs do I need to post-train Cosmos 3? A: This tutorial uses four GPUs on a GB200 node. The guide on GitHub has been tested on 8 H100 GPUs. You can adjust the nproc_per_node and data parallelism config settings to match your hardware setup.
Q: Does the same post-training pipeline work for inverse dynamics and policy mode? A: Yes. The post-training pipeline is identical across all three action generation modes — forward dynamics, inverse dynamics, and policy mode. Only the config file and input format differ.
Q: Where do I find the post-training configs and inference docs for Cosmos 3? A: All configs and step-by-step instructions are in the Cosmos 3 GitHub repo under docs/training.md. Inference documentation is also there, along with example input JSON files for the Bridge dataset.
In this tutorial, you will learn how to post-train NVIDIA Cosmos 3 into a forward dynamics model — an action-conditioned world foundation model that generates future video states given a robot's action trajectory. Post-training Cosmos 3 into a forward dynamics model lets you simulate what a robot will donext, which is a core building block for robotics policy development. By the end, you will have a post-trained Cosmos 3 model running inference and generating predicted future world states from real robot action data.
Get started with Cosmos 3:
📘 Read the technical blog → nvda.ws/4u5G0x8
🤗 Download models → huggingface.co/collections/nvidia/cosmos3
⚙️ Customize + post-train → github.com/nvidia/Cosmos
🚀 Deploy → build.nvidia.com/models?q=cosmos
Chapters:
00:00 Intro — What is Cosmos 3 action conditioning?
00:40 Architecture diagram — condition frames to future state and actions
01:10 Walk through the YAM dataset class in VS Code
01:35 Review the action joint SFT config (action_joint_sft_nano_yam.py)
02:10 Explain joint mode — training all three action heads simultaneously
02:55 Export the trained checkpoint with EMA weights
03:55 Run inference — predicted vs ground truth side by side
#NVIDIACosmos #WorldFoundationModels #PhysicalAI
NVIDIA Cosmos, World Foundation Models, Physical AI, forward dynamics model, action-conditioned model, Cosmos 3 post-training
Q: What is a forward dynamics model?
A: A forward dynamics model is a model that takes an action trajectory, a conditioning video frame, and a task description as input, and generates a video of the predicted future world state aligned with that trajectory.
Q: What datasets can I use to post-train Cosmos 3 for forward dynamics? A: Any dataset in the LeRobot format works. This tutorial uses a custom dataset collected on the YAM robot. You can convert your own robot data into the LeRobot format following the data preparation instructions in the Cosmos 3 GitHub repo.
Q: How many GPUs do I need to post-train Cosmos 3? A: This tutorial uses four GPUs on a GB200 node. The guide on GitHub has been tested on 8 H100 GPUs. You can adjust the nproc_per_node and data parallelism config settings to match your hardware setup.
Q: Does the same post-training pipeline work for inverse dynamics and policy mode? A: Yes. The post-training pipeline is identical across all three action generation modes — forward dynamics, inverse dynamics, and policy mode. Only the config file and input format differ.
Q: Where do I find the post-training configs and inference docs for Cosmos 3? A: All configs and step-by-step instructions are in the Cosmos 3 GitHub repo under docs/training.md. Inference documentation is also there, along with example input JSON files for the Bridge dataset.
