where you ignore labels of pixels belonging to unseen classes during training

[ghost]

Thank you for sharing the useful code.

@subhc , as you discussed in article you consider all images during training, but pixels which relate to unseen classes will be ignored in calculating loss.

I completely explore your codes, but all classes are considered ( 0 to 15 for seen and 16 to 20 for unseen) for calculating loss.

I checked target tensor in this part which calculate loss. Some times in target we have pixels relate to unseen classes ( index from 16 to 20), but they are not ignored for calculating loss.

        for output in outputs:
            # Resize target for {100%, 75%, 50%, Max} outputs
            target_ = resize_target(target, output.size(2))
            target_ = torch.tensor(target_).to(device)
            loss += criterion.forward(output, target_)

Thank you for your answer

[qiang92]

Their code ignored labels which is equal -1.
criterion = nn.CrossEntropyLoss(ignore_index=-1))

And they processed the target labels in their loader, which is "LoaderZLS".
It changed labels of unseen classes and background to -1 in file cocostuff.py.
tmp_label = label
if(self.visibility_mask is not None):
#2 ("novel {}".format(with_novel))
;label = self.visibility_mask[with_novel][tmp_label]

[ghost]

Thanks @qiang92 ,

I am using the code for fewshot.

I saw this part in cocostuff. But again when I print target unique indexes, there are some labels for unseen classes.

When i change inputmix input variable to 'seen' option (there are three options seen, novel, both), it is correct and there are not any unseen indexes in target tensor.

But in this situation, when fine tuning for few shots of novel classes happen (as discussed in paper )?

[ghost]

I noticed that when input mix is in 'both' option, code considers few shots of unseen (same shot for all iterations) classes during training. So, some times during training we have labels of unseen classes in target tensor.

As discussed in article, training few shots of unseen classes during training of seen classes is not best one. Fine tuning for unseen classes after training seen classes works better. But this code trains fewshots of unseen classes during training of seen classes.

I hope authors of article @subhc , write a better read me file for using this useful code.