Complete Instances Mining for Weakly Supervised Instance Segmentation

This project hosts the code for implementing the CIM algorithm for weakly supervised instance segmentation.

Quick View

Since running the code requires preparing a lot of data, if you just want to understand how we implement CIM, you can directly choose to read the paper and the following code.

• Paper [Paper]
• Pipeline [code]
• CIM Strategy [code]

Installation

Please follow the instructions in INSTALL.md.

Preparation

Please follow the instructions in DATASET.md.

Experiments

Before starting the experiment. You need to update these four values cfg_file, output_file, dataset, iter_time in *.sh files.

Training

bash ./scripts/train_CIM.sh

Evaluation

bash ./scripts/eval_CIM.sh

Mask R-CNN Refinement

# generate pseudo labels from CIM for training Mask R-CNN
bash ./scripts/generate_msrcnn_label.sh

Then, we use mmdetection for Mask R-CNN Refinement.

Visualization

# install mmcv-full==1.*, instead of 2.* to avoid conflict
bash ./scripts/visual_result_mmcv.sh

Results

CIM uses image-level labels to generate pseudo labels. CIM-p uses point-level labels to generate pseudo labels. CIM+ means CIM with Mask R-CNN refinement. CIM-p+ means CIM-p with Mask R-CNN refinement.

VOC2012

Method	Backbone	mAP25	mAP50	mAP70	mAP75
CIM	ResNet-50	64.9	51.1	32.4	26.1
CIM-p	ResNet-50	65.2	51.6	33.3	27.2
CIM	VGG-16	65.6	50.8	31.0	25.2
CIM	HRNet-W48	68.3	52.6	33.7	28.4
CIM+	ResNet-50	68.7	55.9	37.1	30.9
CIM-p+	ResNet-50	67.8	55.5	36.6	31.1

COCO val2017

Method	Backbone	AP	mAP50	mAP75
CIM	ResNet-50	11.9	22.8	11.1
CIM+	ResNet-50	17.0	29.4	17.0

COCO test-dev

Method	Backbone	AP	mAP50	mAP75
CIM	ResNet-50	12.0	23.0	11.3
CIM+	ResNet-50	17.2	29.7	17.3

Download

Results of instance segmentation on the VOC2012 and COCO datasets can be downloaded from OneDrive | Google Drive.

Contact

If you have any questions, please feel free to contact Zecheng Li ([email protected]). Thank you.

Acknowledgement

Our implementation is based on these repositories:

• (PRM) https://github.com/ZhouYanzhao/PRM
• (PCL) https://github.com/ppengtang/pcl.pytorch
• (MIST) https://github.com/NVlabs/wetectron
• (HRNet) https://github.com/HRNet/HRNet-Image-Classification
• (mmdetection) https://github.com/open-mmlab/mmdetection

Citation

If you find this work useful, please consider giving it a star ⭐ and citing our paper in your work:

@inproceedings{zecheng2023CIM,
  title={Complete Instances Mining for Weakly Supervised Instance Segmentation},
  author={Li, Zecheng and Zeng, Zening and Liang, Yuqi and Yu, Jin-Gang},
  booktitle={International Joint Conference on Artificial Intelligence},
  year={2023},
}