Loss cardinality does not make sense in the setting when using focal loss

[RomStriker]

Hi,

I see that starting from deformable DETR, it doesn't make sense to have cardinality loss. In the original DETR paper, the class_embed head outputs num_classes + 1, to also output a value for the no object class, and if we keep the same structure then the Cardinality loss specifically the line would work as commented in the code. However, in all the versions of DETR from IDEA-Research that uses focal loss, uses class_embed head that only outputs num_classes, so no output for the no object class, and then the Cardinality loss instead considers the last class the no object class. Can you explain its use?

Also, why did you make the choice to not use the no-object class for focal loss, is there some explanation that I am missing?

[Chrazqee]

I have the same question like that! I guess if add no-object classes to class-embed, it will cause class unbalance question, as I met the question. It causes the network more and more focusing on no-object class, since DINO has lots of queries than DETR, the situation will be more serious I guess! Owning to their code is quite complicated to me, so I do not modified their code to validate my guess. Also, I do not know how the authors calculates the AP in their project! Have you understanding the question? If so, wish you can share it for me, thanks a lot!

best!