Is it possible to apply PPLM to encoder-decoder models (such as BERT2GPT2)

[nooralahzadeh]

No description provided.

[andreamad8]

Hi @nooralahzadeh,

yes, it is possible. Just need to get the activation, attentions matrix, of both enc-dec.

Andrea

[monkdou0]

IF it is possible, what's the input of the discriminator?
The encoder hidden state or the decoder hidden state or both?

[andreamad8]

I think the decoder hidden state should work, but the perturbation can be for both the encoder and the decoder.