require_grad

[1764758458]

[YingHuTsing]

Hi, this warning does not have impact on training process and model performance. Our training also has this warning printed.

[1764758458]

Thank you very much for your answer.！But he this loss stays at 0 it is assumed that the optimization has been reached and then the training is interrupted.

[ggcr]

@1764758458

Make sure you are using the correct conv_version flag.

• --conv_version phi for Phi-2, StableLM, Qwen-1.5
• --conv_version llama for TinyLlama, OpenELM
• --conv_version gemma for Gemma

[Daming-W]

Thank you very much for your answer.！But he this loss stays at 0 it is assumed that the optimization has been reached and then the training is interrupted.

I am facing the same error. Have you resolved this?
My recipe is clip-vit&dinov2-vit mof with Vicuna-7b as LLM.

[1764758458]

Thank you very much for your answer.！But he this loss stays at 0 it is assumed that the optimization has been reached and then the training is interrupted.

I am facing the same error. Have you resolved this? My recipe is clip-vit&dinov2-vit mof with Vicuna-7b as LLM.

我之前用的phi跑出来是这个情况，然后我换成tinyllama之后就正常了

[Daming-W]

Thank you very much for your answer.！But he this loss stays at 0 it is assumed that the optimization has been reached and then the training is interrupted.

I am facing the same error. Have you resolved this? My recipe is clip-vit&dinov2-vit mof with Vicuna-7b as LLM.

我之前用的phi跑出来是这个情况，然后我换成tinyllama之后就正常了

刚刚解决啦, 我这边是将脚本中的 --fp16 True改为 --bf16 True就可以了，在deepspeed的repo中有类似的问题和解法
ref: microsoft/DeepSpeed#4017