Allow model ensemble?

[che85]

@diazandr3s Do you think there is a way to bring model ensemble into this module? I just trained on some data which provided great results during the testing phase using model ensemble that usually runs in the end after finishing 5-fold training with Auto3DSeg.

When running on the same data, but choosing to run with the seemingly best model (when looking at the Tensorboard graphs), it turns out to not be as good as the same data segmented with model ensemble.

for reference: https://github.com/Project-MONAI/tutorials/blob/main/auto3dseg/docs/ensemble.md

I am unsure how much of the original training information it would need since it's requesting the work_dir to make all these ranking decisions.

Is there a better way of combining all models into the best one to just use the current module as is?

[che85]

For reference, I started a discussion asking for combining all best ranked models into one: Project-MONAI/MONAI#8074

[che85]

Comment by @diazandr3s : Project-MONAI/tutorials#1713 (reply in thread)

[diazandr3s]

Hi @che85,

Thanks for your question! You raise a valid point.
We initially used ensemble models, but they significantly increased inference time. That's why we switched to a single model.
However, if users are okay with longer inference times, I don't see any blockers to reintroducing this feature.