First please make sure the modified OpenCLIP has been installed as follows
cd CLIM
pip install -e . -v
Then please refer to this README to install the detector.
Please refer to this README.
We provide checkpoints of models that were trained by CLIM in
Google Drive. Put them under
CLIM/ovdet/checkpoints.
Take ViT-B/16 on OV-COCO as example, run the following to train the detector
cd CLIM/ovdet
bash tools/dist_train.sh \
configs/clip_based/openai_vitb16/faster_rcnn_fpn_openai_vitb16_clim_bs64_ov_coco_3e.py 8 \
--work-dir your/output/directory/ovdet_openai_vitb16_ov_coco_clim
We also provide the following checkpoints of the trained detectors in
Google Drive. Download and
put them under CLIM/ovdet/checkpoints.
Note: the released code for the ViT-based detector achieves better results than that we have initially reported in the paper.
| OV-COCO | Backbone | Novel AP50 | Config | Download |
|---|---|---|---|---|
| Paper | ViT-B/16 | 25.7 | - | - |
| This Repo | ViT-B/16 | 29.7 | config | model |
| OV-LVIS | Backbone | Mask APr | Config | Download |
|---|---|---|---|---|
| Paper | ViT-B/16 | 20.8 | - | - |
| This Repo | ViT-B/16 | 24.3 | config | model |
| Paper | RN50x64 | 32.3 | - | - |
| This Repo | RN50x64 | 32.4 | config | model |
Take ViT-B/16 on OV-COCO as example, run the following script to test the detector
cd CLIM/ovdet
bash tools/dist_test.sh \
configs/clip_based/openai_vitb16/faster_rcnn_fpn_openai_vitb16_clim_bs64_ov_coco_3e.py \
checkpoints/ovdet_openai_vitb16_ov_coco_clim.pth \
8 --work-dir your/output/directory/ovdet_openai_vitb16_ov_coco_clim