Losses

[redhat12345]

In ADDA, classifier loss and advarsarial loss are used. In which file you are using these two losses ?

[aasharma90]

The author has used nn.CrossEntropyLoss() for both the classification and adversarial loss functions, in the pretrain.py and adapt.py files respectively. Look for "criterion" word in the two files.

[taylover-pei]

The author has used nn.CrossEntropyLoss() for both the classification and adversarial loss functions, in the pretrain.py and adapt.py files respectively. Look for "criterion" word in the two files.

I still have no idea where the advarsarial loss in the code？
The author used the nn.LogSoftmax() in the adapt.py, and then used the nn.CrossEntropyLoss() in the main.py to train the discriminator. As we know, the nn.CrossEntropyLoss() combines nn.LogSoftmax() and nn.NLLLoss() in one single class, so is this use not repeated?
And other question is the nn.CrossEntropyLoss() and the nn.LogSoftmax() is equivalent to the advarsarial loss?
Thank you very much! Looking forward to your reply.

[aasharma90]

Hi @taylover-pei

I am assuming that you meant nn.LogSoftMax() in discriminator.py and nn.CrossEntropyLoss() in adapt.py to train the discriminator. Yes, on another look, I think that there is redundancy there. Perhaps, it should have been nn.NLLLoss() in adapt.py then (provided we do not wish to change anything in discriminator.py). The current implementation could be buggy.

For your other question, it's my own understanding that the author has imagined the discriminator as some sort of a binary classifier, since it only needs to discriminate between the source and target features (or classify them into binary outputs, 1 and 0 respectively). So, perhaps he used the standard classification loss for this task. As I said, it's my own understanding and a better answer can only be provided by the author.

Hope it helps!

[taylover-pei]

Hi @taylover-pei

I am assuming that you meant nn.LogSoftMax() in discriminator.py and nn.CrossEntropyLoss() in adapt.py to train the discriminator. Yes, on another look, I think that there is redundancy there. Perhaps, it should have been nn.NLLLoss() in adapt.py then (provided we do not wish to change anything in discriminator.py). The current implementation could be buggy.

For your other question, it's my own understanding that the author has imagined the discriminator as some sort of a binary classifier, since it only needs to discriminate between the source and target features (or classify them into binary outputs, 1 and 0 respectively). So, perhaps he used the standard classification loss for this task. As I said, it's my own understanding and a better answer can only be provided by the author.

Hope it helps!

Thank you very much. I think you are right. The current implementation could be buggy. It should have been nn.NLLLoss() in adapt.py. And also, when training, using the nn.BCELoss() is OK.
I am a new one in domain adaptation of GAN，there is so many I do not know. Thank you so much to explain there to me.