Skip to content

toml_config: add [job].upload_reproducible_setup, default False for OSS #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
foreverlms merged 4 commits into from
Jul 8, 2026
+36 −10
Merged

toml_config: add [job].upload_reproducible_setup, default False for OSS #92

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c1e3a32
toml_config: add [job].upload_reproducible_setup, default False for OSS
foreverlms Jul 7, 2026
e0e298f
update tests
foreverlms Jul 7, 2026
5feb803
fotmat
foreverlms Jul 7, 2026
ee4f63e
Revert examples/toml/sft_config to main
Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion cosmos_framework/configs/toml_config/sft_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,17 @@ class JobConfig(BaseModel):
"'disabled' (no wandb at all)."
),
)
upload_reproducible_setup: bool = Field(
default=False,
description=(
"Upload the reproducible-setup bundle and wandb save_s3 artifacts "
"to S3. Defaults False (OSS: no S3 access). Set True only when S3 "
"upload is configured. Remapped out of [job] to the top-level "
"config.upload_reproducible_setup, overriding the base config value "
"(the VLM base defaults it True). Always emitted by "
"load_experiment_from_toml, so omitting it forces False."
),
)


# ---------------------------------------------------------------- model
Expand Down Expand Up @@ -721,7 +732,14 @@ def load_experiment_from_toml(

# Validate structure against the pydantic schema (raises ValidationError on
# unknown keys because of ``extra="forbid"``).
SFTExperimentConfig.model_validate(raw)
cfg = SFTExperimentConfig.model_validate(raw)

# ``build_hydra_overrides`` walks the *raw* dict, so an omitted field emits
# no override and the base config's value wins. ``upload_reproducible_setup``
# must instead default to False (OSS: no S3), overriding the VLM base's True.
# Inject the pydantic-resolved value back into raw so the override is always
# emitted — the default lives in exactly one place (JobConfig.Field).
raw.setdefault("job", {})["upload_reproducible_setup"] = cfg.job.upload_reproducible_setup

task = raw.get("job", {}).get("task", "vfm")
try:
Expand Down
6 changes: 6 additions & 0 deletions cosmos_framework/configs/toml_config/toml_config_helper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@
# ``model.config.<X>`` in the Hydra tree. ``attn_implementation`` is a
# VLM-only knob — skip it on VFM. Other sections pass through.
"vfm": {
# [job].upload_reproducible_setup lives at the top-level config field,
# not config.job.* — hoist it out of the job section.
("job", "upload_reproducible_setup"): ("upload_reproducible_setup",),
("model", "attn_implementation"): None,
("model", "backbone"): None, # VLM-only — VFM has no model.config.backbone
# Per-caption token cap lives on the nested SFT dataset, not a top-level
Expand All @@ -63,6 +66,9 @@
# tasks — so the catch-all ``("model",) -> ("model", "config")`` rule
# routes them uniformly. Fields with no VLM analog map to ``None`` (skip).
"vlm": {
# [job].upload_reproducible_setup lives at the top-level config field,
# not config.job.* — hoist it out of the job section.
("job", "upload_reproducible_setup"): ("upload_reproducible_setup",),
# No VLM analog — skip these leaves
("model", "max_num_tokens_after_packing"): None,
("model", "joint_attn_implementation"): None,
Expand Down
20 changes: 11 additions & 9 deletions docs/sft_config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,14 +70,15 @@ The full pipeline (dataloader class, dataset wiring, model_instance LazyCall, et

Run identity + meta-fields that pick the Hydra config tree to load.

| field | default | description |
| ------------ | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `task` | `"vfm"` | **META** — chooses which `make_config()` to call: `"vfm"` → `cosmos_framework/configs/base/config.py`, `"vlm"` → `cosmos_framework/configs/base/reasoner/config.py`. Also picks the path-remap rules in `toml_config_helper.PATH_REMAPS`. |
| `experiment` | `""` | **META** — names the Hydra experiment LazyDict registered in `ConfigStore` under `experiment/<name>`. Resolved at load time via `experiment=<name>` (e.g. `vision_sft_nano`). |
| `project` | `""` | W&B project (team-level bucket). Flows to `config.job.project`. |
| `group` | `""` | W&B sub-label for clustering related runs (e.g. `"sft"`). Flows to `config.job.group`. |
| `name` | `""` | W&B run name; forms part of the output dir `$IMAGINAIRE_OUTPUT_ROOT/<project>/<group>/<name>/`. Leave empty (or use `${now:%Y-%m-%d}_${now:%H-%M-%S}`) for auto-timestamped subdir. |
| `wandb_mode` | `"disabled"` | `"online"` (real-time, needs `WANDB_API_KEY`), `"offline"` (log locally, sync later via `wandb sync`), or `"disabled"`. |
| field | default | description |
| --------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `task` | `"vfm"` | **META** — chooses which `make_config()` to call: `"vfm"` → `cosmos_framework/configs/base/config.py`, `"vlm"` → `cosmos_framework/configs/base/reasoner/config.py`. Also picks the path-remap rules in `toml_config_helper.PATH_REMAPS`. |
| `experiment` | `""` | **META** — names the Hydra experiment LazyDict registered in `ConfigStore` under `experiment/<name>`. Resolved at load time via `experiment=<name>` (e.g. `vision_sft_nano`). |
| `project` | `""` | W&B project (team-level bucket). Flows to `config.job.project`. |
| `group` | `""` | W&B sub-label for clustering related runs (e.g. `"sft"`). Flows to `config.job.group`. |
| `name` | `""` | W&B run name; forms part of the output dir `$IMAGINAIRE_OUTPUT_ROOT/<project>/<group>/<name>/`. Leave empty (or use `${now:%Y-%m-%d}_${now:%H-%M-%S}`) for auto-timestamped subdir. |
| `wandb_mode` | `"disabled"` | `"online"` (real-time, needs `WANDB_API_KEY`), `"offline"` (log locally, sync later via `wandb sync`), or `"disabled"`. |
| `upload_reproducible_setup` | `false` | Upload the reproducible-setup bundle + wandb `save_s3` artifacts to S3. **Defaults `false`** (OSS: no S3 access); set `true` only when S3 upload is configured. Remapped out of `[job]` to the top-level `config.upload_reproducible_setup`, overriding the base config value (the VLM base defaults it `true`). Always emitted by `load_experiment_from_toml`, so omitting it forces `false`. |

## `[model]`

Expand Down Expand Up @@ -280,6 +281,7 @@ The same TOML key lands at different Hydra paths depending on `[job].task`:

| TOML path | VFM (`task="vfm"`) Hydra path | VLM (`task="vlm"`) Hydra path |
| ---------------------------------------------------------------------------------------- | ----------------------------------------- | ----------------------------------------- |
| `job.upload_reproducible_setup` | `upload_reproducible_setup` | `upload_reproducible_setup` |
| `model.<X>` | `model.config.<X>` | `model.config.<X>` |
| `model.parallelism.*` | `model.config.parallelism.*` | `model.config.parallelism.*` |
| `model.compile.*` | `model.config.compile.*` | `model.config.compile.*` |
Expand Down Expand Up @@ -310,7 +312,7 @@ A few useful knobs aren't currently modeled by `SFTExperimentConfig` because the
`load_experiment_from_toml(toml_path, extra_overrides)` (in `sft_config.py`) is the end-to-end loader. It:

1. Reads the TOML with `tomllib`.
2. Validates the parsed dict against `SFTExperimentConfig` (raises `ValidationError` on unknown keys).
2. Validates the parsed dict against `SFTExperimentConfig` (raises `ValidationError` on unknown keys), then injects the pydantic-resolved `job.upload_reproducible_setup` back into the raw dict so it is always emitted (defaulting to `false`) even when the TOML omits it.
3. Picks the base config from `[job].task`: `TASK_TO_BASE_CONFIG["vfm"|"vlm"]`.
4. Calls `build_hydra_overrides(raw)` to produce a `["--", "experiment=<name>", "k.p=v", …]` list with per-task remaps applied and MISSING values filtered. `[custom]` is skipped here (it is injected verbatim in step 7, not per-leaf-remapped).
5. Appends `extra_overrides` (CLI tail) so they take precedence over the TOML.
Expand Down