[fix] fail when diloco is used with global pg size of 1

[Jackmin801]

Without this, running the wrong simulate_multi_node shell script doesnt lead to errors but instead creates diloco workers that dont communicate.

This PR causes the training to error in this case as you would never want to do this.

[samsja]

I don't understand why this pr is needed. I might want to do diloco with only one worker, especially for ablation

[Jackmin801]

hrmm makes sense. what if we set the default global size to 0 and error in diloco for that? the diloco launch script will set 1 and pass but the non-diloco launch script sets nothing and gets 0 and will thus fail if it tries to create the diloco class.

[samsja]

hrmm makes sense. what if we set the default global size to 0 and error in diloco for that? the diloco launch script will set 1 and pass but the non-diloco launch script sets nothing and gets 0 and will thus fail if it tries to create the diloco class.

This feel hacky IMO. And I still want to be able to start diloco one worker purely using torchrun

[Jackmin801]

hrmm makes sense. I guess we just have to be careful with the scripts