Skip to content

Commit

Permalink
update readme with 4 gpus instruction
Browse files Browse the repository at this point in the history
  • Loading branch information
samsja committed Sep 26, 2024
1 parent b624c43 commit 0dc5638
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,22 @@ ZERO_BAND_LOG_LEVEL=DEBUG torchrun --nproc_per_node=2 src/zeroband/train.py @co

## run diloco

To run diloco locally you can use the helper script `scripts/simulatsimulate_multi_nodee_mutl.sh`
To run diloco locally you can use the helper script `scripts/simulatsimulate_multi_nodee_mutl.sh`

:note: you need 4 gpus to run the following command

```bash
ZERO_BAND_LOG_LEVEL=DEBUG ./scripts/simulate_multi_node.sh 2 2 src/zeroband/train.py @configs/debug/diloco.toml
```

if you have only two gpus

```bash
ZERO_BAND_LOG_LEVEL=DEBUG ./scripts/simulate_multi_node.sh 2 1 src/zeroband/train.py @configs/debug/diloco.toml
```

One gpu is not supported at the moment because of a fsdp bug in our implementation.

## run test

You need a machine with a least two gpus to run the full test suite.
Expand Down

0 comments on commit 0dc5638

Please sign in to comment.