Skip to content

toml_config: add [job].upload_reproducible_setup, default False for OSS#92

Merged
foreverlms merged 4 commits into
mainfrom
oss-toml-upload-reproducible-setup
Jul 8, 2026
Merged

toml_config: add [job].upload_reproducible_setup, default False for OSS#92
foreverlms merged 4 commits into
mainfrom
oss-toml-upload-reproducible-setup

Conversation

@foreverlms

Copy link
Copy Markdown
Collaborator

The VLM base config defaults upload_reproducible_setup=True, so a run launched through the structured-TOML flow attempts the reproducible-setup S3 upload (and wandb save_s3, which interpolate from ${upload_reproducible_setup}) by default. OSS users mostly have no S3 access and no way to turn it off from the TOML.

Add a [job].upload_reproducible_setup knob (default False) that maps to the top-level config.upload_reproducible_setup:

  • JobConfig gains the field (default False).
  • PATH_REMAPS (vfm + vlm) hoist ("job","upload_reproducible_setup") to the top-level ("upload_reproducible_setup",).
  • load_experiment_from_toml keeps the validated model and always injects the resolved value into raw before building overrides, so the override is emitted as false even when the TOML omits it (build_hydra_overrides walks the raw dict, so an omitted field would otherwise emit nothing and the base True would win).

Users opt back into S3 upload with upload_reproducible_setup = true.

Docs (sft_config.md) + all example [job] blocks updated; adds tests/toml_config_test.py.

@foreverlms foreverlms requested review from lfengad and yy-code-nv July 7, 2026 09:31
@foreverlms foreverlms enabled auto-merge (squash) July 7, 2026 10:38
foreverlms and others added 4 commits July 7, 2026 20:07
The VLM base config defaults upload_reproducible_setup=True, so a run
launched through the structured-TOML flow attempts the reproducible-setup
S3 upload (and wandb save_s3, which interpolate from
${upload_reproducible_setup}) by default. OSS users mostly have no S3
access and no way to turn it off from the TOML.

Add a [job].upload_reproducible_setup knob (default False) that maps to the
top-level config.upload_reproducible_setup:

- JobConfig gains the field (default False).
- PATH_REMAPS (vfm + vlm) hoist ("job","upload_reproducible_setup") to the
  top-level ("upload_reproducible_setup",).
- load_experiment_from_toml keeps the validated model and always injects the
  resolved value into raw before building overrides, so the override is
  emitted as false even when the TOML omits it (build_hydra_overrides walks
  the raw dict, so an omitted field would otherwise emit nothing and the base
  True would win).

Users opt back into S3 upload with `upload_reproducible_setup = true`.

Docs (sft_config.md) + all example [job] blocks updated; adds
tests/toml_config_test.py.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yy-code-nv yy-code-nv force-pushed the oss-toml-upload-reproducible-setup branch from 2920306 to ee4f63e Compare July 8, 2026 03:08
@foreverlms foreverlms merged commit 3017efb into main Jul 8, 2026
8 checks passed
@foreverlms foreverlms deleted the oss-toml-upload-reproducible-setup branch July 8, 2026 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants