Downstream Fine-tuning for Segmentation Task

[KOO-DE]

Hello.

I looked into the train.py file inside the TransUNet folder to fine-tune the segmentation task using the endo_fm.pth file. However, I couldn't find any code that declares the pretrained weights. Could you please guide me on how to utilize endo_fm.pth?

Thank you.

[Kyfafyd]

Hi, @KOO-DE thanks for your interest in our work!
The pre-trained weight is loaded here:

Endo-FM/TransUNet/networks/vit_seg_modeling.py

Line 260 in 3374b57

weight = '../checkpoints/endo_fm.pth'

[KOO-DE]

I have reviewed the code you provided. Thank you so much for your response.

To obtain segmentation results using Endo-FM as described in the paper, should I use the ViT model (R50-ViT-B_16) pretrained on imagenet21k?

[Kyfafyd]

Hi, @KOO-DE you may load our pre-trained weights endo_fm.pth for segmentation fine-tuning. The original TransUNet used the pre-trained weights on ImageNet, which may be not fit in endoscopy scenario due to domain gap.