Apply dualpipe from deepseek-v3 to a trainer or model

[jp1924]

Feature request

Applying deepseek's dual-pipe to transformers or accelrate

Motivation

Recently, deepseek released the proposed dual-pipe code in the DeepSeek-V3 Technical Report.
Looking at the code structure, it is 100% python, and it seems easy to apply.

Your contribution

We can modify the code of dual-pipe and apply it to transformer.