Skip to content

finetune: make training seed explicit - #108

Merged
HenryNdubuaku merged 1 commit into
cactus-compute:mainfrom
TeaShaman-cyber:fix/finetune-seed
Sep 7, 2026
Merged

finetune: make training seed explicit#108
HenryNdubuaku merged 1 commit into
cactus-compute:mainfrom
TeaShaman-cyber:fix/finetune-seed

Conversation

@TeaShaman-cyber

Copy link
Copy Markdown
Contributor

Summary

Expose a --seed option on needle finetune and use it consistently for the three stochastic parts of a training run:

  • LoRA initialization;
  • validation-set selection;
  • epoch shuffling.

The realized seed is also stored in the adapter metadata, so downstream evaluation and provenance tooling can identify which training run produced an adapter.

Motivation

On current main, LoRA initialization and validation splitting are fixed at seed 0, while epoch shuffling uses the process-global NumPy RNG. Two otherwise identical fresh processes can therefore train on different epoch orders, and the CLI provides no way to reproduce or intentionally vary that randomness.

This patch keeps the existing behavior backward-compatible by defaulting --seed to 0, but makes the full training RNG explicit and reproducible.

Verification

  • TDD: both new regression tests failed before the production change and pass after it.
  • tests/test_finetune.py: 5 passed on a real JAX/Flax/Optax runtime.
  • needle finetune --help exposes --seed SEED.
  • python -m py_compile needle/cli.py needle/model/finetune.py passes.
  • git diff --check passes.

This is intentionally scoped to reproducibility only; it does not change the loss, optimizer, QAT behavior, or build/export semantics.

— Semyon Poklad & Шут (Jester)

Signed-off-by: TeaShaman-cyber <pokladsemen@gmail.com>
@HenryNdubuaku
HenryNdubuaku merged commit cd2eae3 into cactus-compute:main Sep 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants