Your question

Is there any explanation about this? Seems the difference between LinearWithFrozenWeight and LinearWithGradAccumulationAndAsyncCommunication is that LinearWithFrozenWeight does not calculate the gradient of weight. This looks like a performance optimization, but turns out this could cause different results.
Your question

Is there any explanation about this? Seems the difference between LinearWithFrozenWeight and LinearWithGradAccumulationAndAsyncCommunication is that LinearWithFrozenWeight does not calculate the gradient of weight. This looks like a performance optimization, but turns out this could cause different results.