Training speed for transformer using SCST

[liuaohanjsj]

Hi,
I'm wondering the training speed of transformer using new-self-critical or SCST. Because during training the model should be inferenced and the inference speed of transformers are much slower than training. In RNN this should not be a problem, but I think that using the transformer the training would be much slower (I implemented a version my self and the training using RL was about 20x slower).
I'm curious about the training speed in your experiment. Do you have any suggestions?

[ruotianluo]

yes, it is much slower. I did add some optimization to speed it up a little, but didn't do any quantitative comparisons.