[WIP] Async checkpointing#3701

S1ro1 wants to merge 3 commits intomainhuggingface/accelerate:mainfrom

feat/async-checkpointinghuggingface/accelerate:feat/async-checkpointingCopy head branch name to clipboard