Active learning guidance needed

[pritesh-mehta]

Will more guidance be provided on the different strategies for active learning and how they can be implemented with monailabel?

Currently, on clicking "Update Model" in 3DSlicer, model.pt will be updated using labels with prefix "label_final_" in folder Y/labels (monailabel start_server --app X --studies Y). If folder Y/label contains only a single label with prefix "label_final_", model.pt will be updated using this single image and label pair only, which could lead the model to overfit on this example. Therefore care would be needed to select a low number of epochs, to reduce the potential for overfitting.

An alternative approach would be to populate folders Y and Y/label with all images and manual labels used to trained pretrained.pt. Each manual label would need to be renamed to have prefix "label_final_". On clicking "Update Model", model.pt would be updated using manually labelled examples and new examples created by DeepEdit/DeepGrow. Naturally, this strategy is less likely to result in overfitting, but requires more execution time.

Other active learning strategies may exist in the literature.