SCST on Diffusion Model

[vinevix]

Hi, I'm trying to use SCST on a diffusion model for image captioning. It uses Cross Entropy Loss but it's trained on global features (and not local ones), is there any way reinforcement learning is going to work on a model like this?

[ruotianluo]

It does not matter if you use global or local. The FC model is using global feature and can be applied reinforcement learning.