Reproducing the result of the paper

[naeunhub]

Hi, thanks for sharing the great work!

I’ve been trying to reproduce the results from your paper using the official repo. However, I’m running into some inconsistencies — especially with “the performance on the VisA 'cashew', 'pcb1', 'pipe_fryum' categories seems much lower than reported”.
When I use your code and set the config file to VisA, result of 'pipe_fryum' is "(66.2, 93.4), 74.2".

I wanted to ask:

Is there anything not mentioned in the paper or repo that could affect the results?
If possible, could you share the domain-adapted features (DA feat) and the pretrained checkpoint you used for your final experiments?

These would really help me verify that I'm on the right track.
Thanks in advance!

[arimousa]

Hi, It is a bit hard for me to say why you get such a low number. Did the model work for MVTec and other categories and being only problematic for pip_fryum?

[naeunhub]

Thanks for your response.

MVTec and VisA datasets work well in most categories, but there are some categories that have very low results, like pipe_fryum, pcb1 in VisA.

Paper's results are not exactly consistent in all categories except for MVTec's grid.
It's getting a little bit different, and the categories with the biggest differences are pipe_fryum and pcb1 in VisA.