Question about reproducing DROID `droid_lerobot_example` FD in Cosmos3 cookbook

[WM20009]

Hi Cosmos3 team,

First of all, thank you for releasing Cosmos3, the framework, checkpoints, and the action-generation cookbooks. The unified FD / ID / policy interface is very useful, and the cookbook has been a helpful reference for understanding how to run action-conditioned generation.

I am trying to reproduce the DROID forward-dynamics example in:

cookbooks/cosmos3/generator/action/run_fd_with_cosmos_framework.ipynb

specifically the section using:

cookbooks/cosmos3/generator/action/assets/droid_lerobot_example

While making the example run in my local checkout, I ran into several compatibility issues between the current DROID dataset wrapper, the bundled cookbook asset, and the current LeRobot version.

Local issues encountered with droid_lerobot_example

To get the official asset running, I had to make several local compatibility fixes:

1. droid_lerobot_example is not present in the DROIDLeRobotDataset version config.

   I added a droid_lerobot_example entry in:

   cosmos_framework/data/generator/action/datasets/droid_lerobot_dataset_config.py
   

   and mapped it to the 640x360 DROID v3-style feature names, including:

   observation.image.*
   observation.state.cartesian_position
   action.gripper_position
   

2. Some low-dimensional features in the asset meta/info.json have empty shapes:

   "shape": []

   The current LeRobot loader did not accept empty shapes in my environment, so I changed them to:

   "shape": [1]

3. The asset meta/info.json appears to use older path templates:

   {episode_chunk}
   {episode_file}
   

   while the current LeRobot v3 code expects:

   {chunk_index}
   {file_index}
   

   I updated data_path and video_path accordingly.

4. The asset episode metadata file:

   meta/episodes/chunk-000/file-000.parquet
   

   was missing video index / timestamp range columns required by the current loader.

   I locally added, for the three video keys:

   chunk_index = 0
   file_index = 0
   from_timestamp = 0.0
   to_timestamp = (722 - 1) / 15
   

5. In this asset, sample["task"] returned by LeRobot was an integer task index instead of a string.

   I made DROIDLeRobotDataset.__getitem__ tolerant to non-string task values by falling back to an empty prompt.

   The corresponding tasks.parquet text seems to be empty / placeholder-like, so the prompt remains empty.

These fixes suggest that the current bundled droid_lerobot_example asset may have some metadata / config drift relative to the current DROID dataset wrapper and LeRobot version.

Reproduction metric

After the above fixes, I ran a teacher-forced forward-dynamics evaluation on droid_lerobot_example.

Setup:

model: Cosmos3-Nano
mode: forward_dynamics
dataset: cookbooks/cosmos3/generator/action/assets/droid_lerobot_example
chunk_length: 16
num_chunks: 5
sample starts: [0, 16, 32, 48, 64]
conditioning: GT first frame for each chunk
metric: PSNR between generated FD video and GT video

Result:

DROID droid_lerobot_example FD teacher-forced, 5 chunks:
mean PSNR = 14.1721

I understand this is only a tiny smoke reproduction, not the full DROID benchmark. Still, the result looked lower than I expected for a bundled cookbook asset, so I wanted to check whether my setup is using the intended checkpoint and preprocessing.

Question

Is the public Cosmos3-Nano checkpoint used by the cookbook expected to be the mid-trained checkpoint described in the technical report, or is it a different base / pre-trained / release checkpoint?

In other words, for the DROID FD cookbook example, should users expect the public Cosmos3-Nano checkpoint to reproduce the robotics FD behavior reported, or is additional domain-specific post-training / a different checkpoint required?

Any clarification on the intended checkpoint, expected PSNR range for droid_lerobot_example, or whether the cookbook asset metadata should be updated would be very helpful.

Thanks again for the release and the detailed cookbooks.