Code

Multinode support in torchtune#2301

Code

Merged

joecummings merged 36 commits intopytorch:mainpytorch/torchtune:mainfrom joecummings:multi-node-supportjoecummings/torchtune:multi-node-supportCopy head branch name to clipboardFeb 5, 2025

+331-37

Commits

Commits on Jan 27, 2025

• Remove last references to from training

  

  Joe Cummings

  committed
  bbd81fd

  Copy full SHA for bbd81fd

• Deprecate and use new function

  

  Joe Cummings

  committed
  c04ebaf

  Copy full SHA for c04ebaf

• Expose

  

  Joe Cummings

  committed
  e02d39b

  Copy full SHA for e02d39b

• Update API docs

  

  Joe Cummings

  committed
  c558f27

  Copy full SHA for c558f27

• Add tests

  

  Joe Cummings

  committed
  454536c

  Copy full SHA for 454536c

• Merge remote-tracking branch 'upstream/main' into multi-node-support

  

  Joe Cummings

  committed
  78bb2ae

  Copy full SHA for 78bb2ae

• Lint

  

  Joe Cummings

  committed
  66b06e1

  Copy full SHA for 66b06e1

• Add multinode recipe and sbatch script

  

  Joe Cummings

  committed
  0d5aeb4

  Copy full SHA for 0d5aeb4

• Update launch commands

  

  Joe Cummings

  committed
  afc9c2e

  Copy full SHA for afc9c2e

• Move env variables around

  

  Joe Cummings

  committed
  c4748a5

  Copy full SHA for c4748a5

• Multi-node tutorial

  

  Joe Cummings

  committed
  94440f9

  Copy full SHA for 94440f9

Commits on Jan 28, 2025

• Updates

  

  Joe Cummings

  committed
  deffeca

  Copy full SHA for deffeca

• Update code block

  

  Joe Cummings

  committed
  f441721

  Copy full SHA for f441721

• asdf

  

  Joe Cummings

  committed
  9ba9e24

  Copy full SHA for 9ba9e24

• Fix linting errors

  

  Joe Cummings

  committed
  b36325a

  Copy full SHA for b36325a

Commits on Jan 29, 2025

• Updates

  

  Joe Cummings

  committed
  fc9afbd

  Copy full SHA for fc9afbd

• Lint

  

  Joe Cummings

  committed
  373e0c0

  Copy full SHA for 373e0c0

• Pass test

  

  Joe Cummings

  committed
  4659938

  Copy full SHA for 4659938

• Updates to tutorial

  

  Joe Cummings

  committed
  693b8cb

  Copy full SHA for 693b8cb

• Remove full_finetune_multinode from recipes registry

  

  Joe Cummings

  committed
  3d8d73d

  Copy full SHA for 3d8d73d

• Lint

  

  Joe Cummings

  committed
  c0345a5

  Copy full SHA for c0345a5

• Last link

  

  Joe Cummings

  committed
  a3aaeb4

  Copy full SHA for a3aaeb4

• Merge remote-tracking branch 'upstream/main' into multi-node-support

  

  Joe Cummings

  committed
  8e20394

  Copy full SHA for 8e20394

Commits on Jan 30, 2025

• Merge remote-tracking branch 'upstream/main' into multi-node-support

  

  Joe Cummings

  committed
  427a290

  Copy full SHA for 427a290

Commits on Jan 31, 2025

• Evan updates

  

  Joe Cummings

  committed
  b56b6be

  Copy full SHA for b56b6be

• Merge remote-tracking branch 'upstream/main' into multi-node-support

  

  Joe Cummings

  committed
  63205da

  Copy full SHA for 63205da

• Merge remote-tracking branch 'upstream/main' into multi-node-support

  

  Joe Cummings

  committed
  76ea872

  Copy full SHA for 76ea872

• Update comment

  

  Joe Cummings

  committed
  63eb274

  Copy full SHA for 63eb274

• Move process initialization

  joecummings

  committed
  4d027b0

  Copy full SHA for 4d027b0

Commits on Feb 1, 2025

• Move init process group to above checkpoint instantiation

  joecummings

  committed
  34aa18b

  Copy full SHA for 34aa18b

• Update intro

  joecummings

  committed
  30b7366

  Copy full SHA for 30b7366

• Docs r dumb

  joecummings

  committed
  c7fdc21

  Copy full SHA for c7fdc21

• Wow

  joecummings

  committed
  900d643

  Copy full SHA for 900d643

Commits on Feb 3, 2025

• Rework intro

  joecummings

  committed
  9e230ca

  Copy full SHA for 9e230ca

Commits on Feb 5, 2025

• Merge remote-tracking branch 'upstream/main' into multi-node-support

  joecummings

  committed
  fa23015

  Copy full SHA for fa23015

• We have PyTorch 2.6 now!

  joecummings

  committed
  fa64d05

  Copy full SHA for fa64d05