Progressive learning of EfficientNetV2

[pawopawo]

Do you will try to reproduce Progressive Learning in EfficientNetv2 ?

[liu-zhenhua]

+1

[fffffgggg54]

I'm interested in writing a PR for this, since I use it in my own training scripts. I have it implemented by modifying the dataset transforms every epoch.

IME the main issue is that that the start of training uses far less vram compared to the end of training. Additional throughput can be had by adjusting batch size/gradient accumulation to maximize vram usage, but implementing this adjustment is nightmarish. I was trying to do it by halving/doubling the values, respectively, but the vram would not deallocate. Might be better with the timm script, since its set up differently.