Fix (maskrcnn-example): improvements on maskrcnn-example (#89)

* feat: make training log optional

* fix: skip non-positive width/height

* Update examples/maskrcnn-example/utils/encord_dataset.py

Co-authored-by: Eloy Pérez Torres <[email protected]>
Signed-off-by: Gorkem Polat <[email protected]>

* feat: use logging to give message to the user

* fix: formatting

* Feat (maskrcnn-example): improving labels workflow (#92)

* feat: add label improvement workflow

* fix: typo

* fix: add more explanation

* fix: check if there answers before accessing the first one (#96)

* fix: check if there answers before accessing the first one

* fix: generate embedding only if needed and ignore checkbox classifications

* fix: don't run irrelevant object metrics

Signed-off-by: Gorkem Polat <[email protected]>
Co-authored-by: David Sapiro <[email protected]>

* Feat (maskrcnn-example): Save last epoch (#93)

* feat: add label improvement workflow

* fix: typo

* fix: add more explanation

* feat: Add saving last epoch

* feat: maskrcnn nms argument

* fix: black formattiing

Signed-off-by: Gorkem Polat <[email protected]>
Co-authored-by: Eloy Pérez Torres <[email protected]>
Co-authored-by: David Sapiro <[email protected]>

Author: 3 people

examples/maskrcnn-example/readme.md

@@ -25,11 +25,12 @@ conda activate encord-maskrcnn
 ```
 
 ## Training
-1. Create a config.ini file by looking at the example_config.ini
-2. You can use resize the imagens and corresponding annotations via utils/downscle_dataset.py.
-2. For the training, the only required fields are [DATA], [LOGGING], and [TRAIN] sections
-3. Activate the environment and run `python train.py`
-4. You can track the progress of the training on wandb platformdd
+1. You can use Encord-Active's Actions tab to create COCO annotations.
+2. Create a config.ini file by looking at the example_config.ini
+3. You can resize the images and corresponding annotations via utils/downscale_dataset.py.
+4. For the training, the only required fields are [DATA], [LOGGING], and [TRAIN] sections
+5. Activate the environment and run `python train.py`
+6. You can track the progress of the training on wandb platform.
 
 
 ## Importing Encord-Active predictions
@@ -42,4 +43,4 @@ conda activate encord-maskrcnn
 encord-active import predictions /path/to/predictions.pkl -t /path/to/project
 ```
 
-5. Now you can see the model performance on the __Model Quality__ tab.
+5. Open the app. Now you can see the model performance on the __Model Quality__ tab.

examples/maskrcnn-example/train.py

@@ -58,6 +58,7 @@ def main(params):
     setup_reproducibility(35)
 
     best_map = 0
+    last_epoch = 0
     early_stop_counter = 0
 
     device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
@@ -123,6 +124,7 @@ def main(params):
     val_map_metric = MeanAveragePrecision(iou_type="segm").to(device)
 
     for epoch in range(params.train.max_epoch):
+        last_epoch = epoch
         print(f"Epoch: {epoch}")
         train_one_epoch(model, device, data_loader, optimizer, log_freq=10)
 
@@ -170,6 +172,14 @@ def main(params):
                 print("Early stopping at: " + str(epoch))
                 break
 
+    if params.logging.wandb_enabled:
+        torch.save(
+            model.state_dict(),
+            os.path.join(wandb.run.dir, f"epoch_{last_epoch}_maskrcnn.ckpt"),
+        )
+    else:
+        torch.save(model.state_dict(), f"weights/epoch_{last_epoch}_maskrcnn.ckpt")
+
     print("Training finished")
 
 

examples/maskrcnn-example/utils/encord_dataset.py

@@ -2,6 +2,8 @@
 from pathlib import Path
 
 sys.path.append(Path(__file__).parent.as_posix())
+import logging
+
 import torch
 import torchvision
 from provider import convert_coco_poly_to_mask
@@ -21,6 +23,16 @@ def __getitem__(self, idx):
 
         boxes, labels, area, iscrowd = [], [], [], []
         for target_item in target:
+            if target_item["bbox"][2] <= 0 or target_item["bbox"][3] <= 0:
+                logging.warning(
+                    f"ERROR: Target bbox for the following image \n"
+                    f'title:      {img_metadata[0]["image_title"]} \n'
+                    f'label_hash: {img_metadata[0]["label_hash"]} \n'
+                    f'data_hash:  {img_metadata[0]["data_hash"]} \n'
+                    f'has non-positive width/height => [x,y,w,h]: {target_item["bbox"]}. \n'
+                    f"Therefore, skipping this annotation."
+                )
+                continue
             boxes.append(
                 [
                     target_item["bbox"][0],

examples/maskrcnn-example/utils/model_libs.py

@@ -4,9 +4,11 @@
 from torchvision.models.detection.mask_rcnn import MaskRCNNPredictor
 
 
-def get_model_instance_segmentation(num_classes, fine_tuning=False):
+def get_model_instance_segmentation(num_classes, fine_tuning=False, box_nms_thresh=0.5):
     # load an instance segmentation model pre-trained on COCO
-    model = torchvision.models.detection.maskrcnn_resnet50_fpn(weights=MaskRCNN_ResNet50_FPN_Weights.DEFAULT)
+    model = torchvision.models.detection.maskrcnn_resnet50_fpn(
+        weights=MaskRCNN_ResNet50_FPN_Weights.DEFAULT, box_nms_thresh=box_nms_thresh
+    )
 
     # get number of input features for the classifier
     in_features = model.roi_heads.box_predictor.cls_score.in_features

examples/maskrcnn-example/workflows/improve_dataset_by_fixing_labels.md

@@ -0,0 +1,51 @@
+# Improving the labels using Encord-Active
+
+1. Copy the project on the Encord-Annotate platform (we do not want to lose the original labels).
+
+On Encord-Annotate, open the project, go to Settings => Copy project. Select everything (labels, images etc.) when copying the project.
+
+2. Import the new project to Encord-Active.
+
+See documentation: https://encord-active-docs.web.app/cli/import-encord-project 
+
+3. Open the Encord-Active app.
+
+```shell
+(encord-active-environment) > encord-active visualise
+```
+
+4. Go to **Label Quality => Explore**, and choose **Object annotation quality** metric.
+
+5. Fix labels via Encord Annotate by clicking the editor button.
+
+6. Once the fixing operation is finished, the local project should be updated again because labels have changed 
+on the Encord Annotate side. So delete the local project and import it again.
+
+7. Open the App. Go to **Actions => Filter & Export**, click **Generate COCO file** and when activated 
+click **Download filtered data** to download COCO file.
+
+8. Fill in **config.ini** file.
+
+9. Train the model
+
+```shell
+# inside project root folder
+(encord-maskrcnn) > python train.py
+```
+
+10. Import predictions. Check the readme.md file for detailed instructions of importing Encord predictions.
+
+```shell
+(encord-maskrcnn) > python generate_ea_predictions.py
+```
+
+```shell
+(encord-active-environment) > encord-active import predictions /path/to/pickle/file -t /path/to/project
+```
+
+11. Open the app and check the **Model quality** tab for the details.
+
+
+
+
+