Skip to content

Add Cosmos-H-Dreams tabletop distillation recipe - #7

Merged
lukaszbinden merged 2 commits into
mainfrom
feat/cosmos-h-dreams-tabletop-distillation-recipe
Jul 26, 2026
Merged

Add Cosmos-H-Dreams tabletop distillation recipe#7
lukaszbinden merged 2 commits into
mainfrom
feat/cosmos-h-dreams-tabletop-distillation-recipe

Conversation

@lukaszbinden

Copy link
Copy Markdown
Contributor

No description provided.

@greptile-apps

greptile-apps Bot commented Jul 26, 2026

Copy link
Copy Markdown

Greptile Summary

Adds a complete tabletop teacher-training and causal-student distillation recipe.

  • Adds JHU dVRK dataset mixtures, configurable train/test partitioning, and short- and long-horizon dataloaders.
  • Registers teacher fine-tuning, Phase 0 cache generation, causal warmup, and Self Forcing experiments.
  • Adds supporting preprocessing, manifest extraction, launch scripts, and end-to-end tutorial documentation.

Confidence Score: 4/5

The PR should not merge until the prerequisite stops executing an unverified remote installer response directly in the user’s shell.

The documented setup path still pipes mutable content from the installer endpoint directly into sh without pinning or integrity verification, so a compromised response would execute with the user’s privileges.

Files Needing Attention: docs/tutorial_teacher_training_and_self_forcing.md

Important Files Changed

Filename Overview
cosmos_predict2/_src/predict2/action/datasets/gr00t_dreams/data/dataset.py Adds configurable per-dataset split ratios and overrides to the mixed LeRobot dataset pipeline.
cosmos_predict2/_src/predict2/action/configs/action_conditioned/experiment/exp_2B_action_conditioned_rectify_flow_gr00t.py Registers the short-teacher, fine-annealing, and long-horizon tabletop training stages.
cosmos_predict2/_src/predict2/action/inference/inference_jhu_dvrk_warmup.py Adds mixed-dataset Phase 0 cache generation for the long-horizon teacher.
cosmos_predict2/_src/predict2/interactive/configs/experiment/exp_action_self_forcing.py Adds the 73-frame tabletop Self Forcing distillation configuration.
docs/tutorial_teacher_training_and_self_forcing.md Documents the complete tabletop training and deployment workflow, but its prerequisite installation command retains the previously reported unsafe remote-script execution.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    D[Tabletop dataset] --> ST[13-frame teacher]
    ST --> FA[Fine annealing]
    FA --> LT[73-frame teacher]
    LT --> C[Phase 0 trajectory cache]
    C --> W[Causal student warmup]
    LT --> SF[Self Forcing]
    W --> SF
    SF --> DP[Converted deployment checkpoint]
Loading

Reviews (2): Last reviewed commit: "Fix pre-commit formatting and license ch..." | Re-trigger Greptile

cd Cosmos-H-Surgical-Simulator
git lfs pull

curl -LsSf https://astral.sh/uv/install.sh | sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 security Unverified remote installer execution

This prerequisite pipes mutable network content directly into sh without pinning or integrity verification, exposing users to arbitrary command execution if the installer response is compromised. Download a pinned installer artifact and verify its checksum or signature before executing it. How this was verified: The documented command sends the HTTPS response directly to sh without a checksum, signature, or pinned installer version.

@lukaszbinden
lukaszbinden merged commit a230559 into main Jul 26, 2026
2 checks passed
@lukaszbinden
lukaszbinden deleted the feat/cosmos-h-dreams-tabletop-distillation-recipe branch July 26, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant