Auto3dseg - algorithm bundle vs monai bundle confusion - easy way to convert between the two?

[mea02jct]

Hi,

I'm trying to standardise our processes for creating and deploying segmentation tools. One potential choice is to use auto3dseg for everything. According to some of the documentation I've read this has the added advantage that trained models should already be in MONAI bundle format (another good standard to work to).

However, although auto3dseg uses algorithm bundles (templates) it seems that these are quite different to 'standard' MONAI bundles. I personally find this quite confusing!

Firstly, I wondered whether different terminology should be used to avoid this mis-understanding? Secondly, is there an easy way to convert a trained auto3dseg algorithm into MONAI bundle format? From what I can see it currently requires manual creation of new metadata.json file etc.

Thanks

[diazandr3s]

Hi @mea02jct,

This is great a question. I've also tried to use trained Auto3DSeg models on MONAI Label but it was not a straightforward thing to do.

The bundles created by Auto3DSeg seem to be different from the ones created in the MONAI Model Zoo: https://github.com/Project-MONAI/model-zoo/tree/dev/models - So they can't be consumed by MONAI Label directly.

I had to manually update the configs, trainers and infers files from the Segmentation model in the Radiology app using the hyperparameters created by Auto3DSeg.

I'd suggest opening a ticket or moving this one to the main MONAI repo and see if there is interest from the community in making this easier.

Which models are you training using Auto3DSeg?

Hope this helps,

[mea02jct]

Hi Andres,

Thanks for the reply. Good to know that someone else has struggled with this. I shall move my question to the main repo as suggested.

I've currently only trained a cardiac segmentation model, which worked well but hesitating about training more models until I get clarity on which 'standard' MONAI will be using for trained models in future.

Thanks again

[diazandr3s]

Hi @mea02jct,

If you already trained the Auto3Seg on cardiac segmentation, I'd suggest you add the model to this Slicer module: https://github.com/lassoan/SlicerMONAIAuto3DSeg

The community will benefit from the cardiac model you've trained.

I am happy to provide more details if you need them.

[che85]

Just checking on this issue. Has there been any progress ever since? I am interested in deploying Auto3DSeg, but in a client-server-like way similar to MONAILabel. Auto3DSeg is great, but I wouldn't want the users to download models and run on their local machine which might not be as fast and more cumbersome to maintain.

[che85]

I could imagine refactoring MONAIAuto3DSegLogic and extracting the parts responsible for running inference. Similarly to the monailabel server, we could create an auto3dseg server that can run independently of Slicer with FastAPI.

Let me know what you think. I don't want to duplicate work, but MONAILabel cannot use multiple input volumes (to the best of my knowledge).

[che85]

The Slicer Webserver is also an option and I started experimenting with it.

[diazandr3s]

I could imagine refactoring MONAIAuto3DSegLogic and extracting the parts responsible for running inference. Similarly to the monailabel server, we could create an auto3dseg server that can run independently of Slicer with FastAPI.

Let me know what you think. I don't want to duplicate work, but MONAILabel cannot use multiple input volumes (to the best of my knowledge).

Hi @che85,

The app I mentioned here is an example for single sequence. But with a simple change it should be possible to multisequence inference.

Here I explain how adding a transform that read the other sequence will make this app to run inference on multisequence:

#1443

and here

#241 (comment)

MONAI Label (server itself) does support training and inference with multisequence. The difficulty comes with the MONAI Label Slicer plugin.

Hope this helps,

[che85]

I am working on an implementation adopting Slicer's WebServer. You can host a server from MONAIAuto3DSeg and connect to a remote server. It's a very basic implementation (not session based) but gets the job done.

[diazandr3s]

Great! If possible, please share your experience. Thanks in advance!

[pieper]

@che85 if there are no other solutions in the works we could investigate using Slicer's WebServer to provide an Auto3DSeg server. It's already possible to post volumes to Slicer and trigger computation so it wouldn't be too hard to add this feature as part of the extension (or a new extension).