Error When Using Fine-Tuned SAM2 Model with SAMURAI

[Mahdi-Koochali]

Hi,
Thanks for your great work!

I'm encountering an issue when trying to use my fine-tuned version of the latest SAM2 model with SAMURAI. I have the config file for my fine-tuned model, but when I run SAMURAI, I receive the following error:

model = instantiate(cfg.model, _recursive_=True) File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/instantiate/_instantiate2.py", line 226, in instantiate return instantiate_node( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/instantiate/_instantiate2.py", line 347, in instantiate_node return _call_target(_target_, partial, args, kwargs, full_key) File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target raise InstantiationException(msg) from e hydra.errors.InstantiationException: Error in call to target 'sam2.sam2_video_predictor.SAM2VideoPredictor': TypeError("SAM2Base.__init__() missing 3 required positional arguments: 'image_encoder', 'memory_attention', and 'memory_encoder'") full_key: model

I followed the instructions and suggestions from this open issue in the SAM2 repo, but the solution did not work when applied to SAMURAI.

Details:
I’m using a fine-tuned SAM2 model with its corresponding config file. The config file works fine when tested in isolation with SAM2, but I encounter the issue when integrating it into SAMURAI.
The specific error relates to the missing required positional arguments in the SAM2Base.init() method.
My questions:
Is there a recommended way to ensure compatibility between SAMURAI and a fine-tuned SAM2 model?
Are there specific changes needed to the instantiate() or Hydra config handling to avoid this issue?
How should the image_encoder, memory_attention, and memory_encoder arguments be provided to SAMURAI?
I’d appreciate any guidance on how to resolve this issue.

Thank you for your time and help!

Best regards,

[yangchris11]

Do the issue happened when you try to load your own fine-tuned ckpt/weight?

I don't think we touch any part of the hydra related component. Can you share your ckpt and the script you ran in order for reproducing the issue?