Skip to content
AI, TECHNOLOGY AND MEDICINE NEWS
Choose where brand links take you.
YOUR NEXT DISCOVERY
Home / Articles / Technology
Technology · NVIDIA

How NVIDIA uses Cosmos 3 to simulate driving and robotics

In a guide published on Hugging Face on November 5, NVIDIA details four variants of Cosmos 3 Nano for generating driving data and simulating how an environment looks after a robot acts. Its November 2 technical blog already described the weights as available.

Read the analysis
Will Buckner / Wikimedia Commons, CC BY 2.0. Contextual archive photo of the NVIDIA sign in Santa Clara, 2020. Resized image.
Will Buckner / Wikimedia Commons, CC BY 2.0. Contextual archive photo of the NVIDIA sign in Santa Clara, 2020. Resized image. · Image source · CC BY 2.0

In a guide published on Hugging Face on November 5, NVIDIA details four variants of Cosmos 3 Nano for generating driving data and simulating how an environment looks after a robot acts. Its November 2 technical blog already described the weights as available. [1]

What changes

Transfer-Auto varies weather, lighting and time of day in synthetic driving scenes. Sim-Auto generates synchronized camera and lidar observations in successive chunks. With a simulation runtime, it can respond to updated driving controls. [1]

Context

For robotics, Sim-Transfer generates images from spatial inputs such as depth or segmentation maps, while the simulator maintains the physical state. Sim-Bimanual predicts the next visual observation from the actions of a two-armed robot or from camera movement. [1]

Scope and availability

NVIDIA reports 16 billion trainable parameters per variant and distributes the weights under OpenMDW 1.1. The published Sim-Bimanual setup with vLLM-Omni was tested on one GB200. That test describes a specific configuration and leaves minimum memory requirements for other setups undetermined. [1] [2]

What to check in practice

To compare variants for robotics, use the same action sequence and note where the output stops representing the expected movement. Record the hardware, runtime and memory use as well. The published weights make that test possible. The GB200 example does not establish what other hardware can run it.

Sources and documentation

  1. Official source ↗
  2. Technical source 2 ↗
  3. Technical source 3 ↗
  4. Archive photo ↗
  5. CC BY 2.0 ↗

The original material

Read the original source ↗

Was this news useful to you?

Help us improve our coverage.