-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
a337ff7
commit 05c0064
Showing
5 changed files
with
245 additions
and
433 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,215 +1,48 @@ | ||
| <div align="center"> | ||
| <img src="resources/mmrotate-logo.png" width="450"/> | ||
| <div> </div> | ||
| <div align="center"> | ||
| <b><font size="5">OpenMMLab website</font></b> | ||
| <sup> | ||
| <a href="https://openmmlab.com"> | ||
| <i><font size="4">HOT</font></i> | ||
| </a> | ||
| </sup> | ||
| | ||
| <b><font size="5">OpenMMLab platform</font></b> | ||
| <sup> | ||
| <a href="https://platform.openmmlab.com"> | ||
| <i><font size="4">TRY IT OUT</font></i> | ||
| </a> | ||
| </sup> | ||
| </div> | ||
| <div> </div> | ||
| # Scene Graph Generation in Large-Size VHR Satellite Imagery: A Large-Scale Dataset and A Context-Aware Approach | ||
|
|
||
| [](https://pypi.org/project/mmrotate) | ||
| [](https://mmrotate.readthedocs.io/en/latest/) | ||
| [](https://github.com/open-mmlab/mmrotate/actions) | ||
| [](https://codecov.io/gh/open-mmlab/mmrotate) | ||
| [](https://github.com/open-mmlab/mmrotate/blob/main/LICENSE) | ||
| [](https://github.com/open-mmlab/mmrotate/issues) | ||
| [](https://github.com/open-mmlab/mmrotate/issues) | ||
| The official implementation of the oriented object detection part of the paper "[Scene Graph Generation in Large-Size VHR Satellite Imagery: A Large-Scale Dataset and A Context-Aware Approach](https://arxiv.org/abs/)". | ||
|
|
||
| [📘Documentation](https://mmrotate.readthedocs.io/en/latest/) | | ||
| [🛠️Installation](https://mmrotate.readthedocs.io/en/latest/install.html) | | ||
| [👀Model Zoo](https://mmrotate.readthedocs.io/en/latest/model_zoo.html) | | ||
| [🆕Update News](https://mmrotate.readthedocs.io/en/latest/changelog.html) | | ||
| [🚀Ongoing Projects](https://github.com/open-mmlab/mmrotate/projects) | | ||
| [🤔Reporting Issues](https://github.com/open-mmlab/mmrotate/issues/new/choose) | ||
| ## ⭐️ Highlights | ||
|
|
||
| </div> | ||
| **TL;DR:** We propose RSG, the first large-scale dataset for scene graph generation in large-size VHR SAI. Containing more than `210,000` objects and over `400,000` triplets across `1,273` complex scenarios globally. | ||
|
|
||
| <!--中/英 文档切换--> | ||
| <p align="center"> | ||
| <img src="demo/rsg.jpg" alt="scatter" width="98%"/> | ||
| </p> | ||
|
|
||
| <div align="center"> | ||
| ## 📌 Abstract | ||
|
|
||
| English | [简体中文](README_zh-CN.md) | ||
| Scene graph generation (SGG) in satellite imagery (SAI) benefits promoting intelligent understanding of geospatial scenarios from perception to cognition. In SAI, objects exhibit great variations in scales and aspect ratios, and there exist rich relationships between objects (even between spatially disjoint objects), which makes it necessary to holistically conduct SGG in large-size very-high-resolution (VHR) SAI. However, the lack of SGG datasets with large-size VHR SAI has constrained the advancement of SGG in SAI. Due to the complexity of large-size VHR SAI, mining triplets $<$subject, relationship, object$>$ in large-size VHR SAI heavily relies on long-range contextual reasoning. Consequently, SGG models designed for small-size natural imagery are not directly applicable to large-size VHR SAI. To address the scarcity of datasets, this paper constructs a large-scale dataset for SGG in large-size VHR SAI with image sizes ranging from 512 × 768 to 27,860 × 31,096 pixels, named RSG, encompassing over 210,000 objects and more than 400,000 triplets. To realize SGG in large-size VHR SAI, we propose a context-aware cascade cognition (CAC) framework to understand SAI at three levels: object detection (OBD), pair pruning and relationship prediction. As a fundamental prerequisite for SGG in large-size SAI, a holistic multi-class object detection network (HOD-Net) that can flexibly integrate multi-scale contexts is proposed. With the consideration that there exist a huge amount of object pairs in large-size SAI but only a minority of object pairs contain meaningful relationships, we design a pair proposal generation (PPG) network via adversarial reconstruction to select high-value pairs. Furthermore, a relationship prediction network with context-aware messaging (RPCM) is proposed to predict the relationship types of these pairs. To promote the development of SGG in large-size VHR SAI, this paper releases a SAI-oriented SGG toolkit with 3 OBD methods and 5 SGG methods, and develops a benchmark based on RSG where our RPCM outperforms the SOTA methods with a large margin of 3.65\%/5.17\%/3.80\% at HMR@1500 on PredCls/SGCls/SGDet. The RSG dataset and SAI-oriented toolkit will be made publicly available at https://linlin-dev.github.io/project/RSG.html. | ||
|
|
||
| </div> | ||
| ## 🛠️ Usage | ||
|
|
||
| ## Introduction | ||
| For instructions on installation, pretrained models, training and evaluation, please refer to [mmrotate](README_en.md). | ||
|
|
||
| MMRotate is an open-source toolbox for rotated object detection based on PyTorch. | ||
| It is a part of the [OpenMMLab project](https://github.com/open-mmlab). | ||
| ## 🚀 Released Models | ||
|
|
||
| The master branch works with **PyTorch 1.6+**. | ||
| ### Oriented Object Detection | ||
|
|
||
| https://user-images.githubusercontent.com/10410257/154433305-416d129b-60c8-44c7-9ebb-5ba106d3e9d5.MP4 | ||
| | Detector | mAP | Configs | Download | | ||
| | :--------: |:---:|:-------:|:--------:| | ||
| | KLD | 25.0 | [rotated_retinanet_hbb_kld_r50_fpn_1x_rsg_oc](configs/kld/rotated_retinanet_hbb_kld_r50_fpn_1x_rsg_oc.py) | [log](https://huggingface.co/yangxue/rsg-mmrotate/raw/main/rotated_retinanet_hbb_kld_r50_fpn_1x_rsg_oc.log) \| [ckpt](https://huggingface.co/yangxue/rsg-mmrotate/resolve/main/rotated_retinanet_hbb_kld_r50_fpn_1x_rsg_oc-343a0b83.pth?download=true) | | ||
| | GWD | 25.3 | [rotated_retinanet_hbb_gwd_r50_fpn_1x_rsg_oc](configs/gwd/rotated_retinanet_hbb_gwd_r50_fpn_1x_rsg_oc.py) | [log](https://huggingface.co/yangxue/rsg-mmrotate/raw/main/rotated_retinanet_hbb_gwd_r50_fpn_1x_rsg_oc.log) \| [ckpt](https://huggingface.co/yangxue/rsg-mmrotate/resolve/main/rotated_retinanet_hbb_gwd_r50_fpn_1x_rsg_oc-566d2398.pth?download=true) | | ||
| | KFIoU | 25.5 | [rotated_retinanet_hbb_kfiou_r50_fpn_1x_rsg_oc](configs/kfiou/rotated_retinanet_hbb_kfiou_r50_fpn_1x_rsg_oc.py) | [log](https://huggingface.co/yangxue/rsg-mmrotate/raw/main/rotated_retinanet_hbb_kfiou_r50_fpn_1x_rsg_oc.log) \| [ckpt](https://huggingface.co/yangxue/rsg-mmrotate/resolve/main/rotated_retinanet_hbb_kfiou_r50_fpn_1x_rsg_oc-198081a6.pth?download=true) | | ||
| | FCOS | 28.1 | [rotated_fcos_r50_fpn_1x_rsg_le90](configs/rotated_fcos/rotated_fcos_r50_fpn_1x_rsg_le90.py) | [log](https://huggingface.co/yangxue/rsg-mmrotate/raw/main/rotated_fcos_r50_fpn_1x_rsg_le90.log) \| [ckpt](https://huggingface.co/yangxue/rsg-mmrotate/resolve/main/rotated_fcos_r50_fpn_1x_rsg_le90-a579fbf7.pth?download=true) | | ||
| | KLD | 25.0 | [](configs//.py) | [log](https://huggingface.co/yangxue/rsg-mmrotate/raw/main/.log) \| [ckpt]() | | ||
|
|
||
| <details open> | ||
| <summary><b>Major Features</b></summary> | ||
| ## 🖊️ Citation | ||
|
|
||
| - **Support multiple angle representations** | ||
|
|
||
| MMRotate provides three mainstream angle representations to meet different paper settings. | ||
|
|
||
| - **Modular Design** | ||
|
|
||
| We decompose the rotated object detection framework into different components, | ||
| which makes it much easy and flexible to build a new model by combining different modules. | ||
|
|
||
| - **Strong baseline and State of the art** | ||
|
|
||
| The toolbox provides strong baselines and state-of-the-art methods in rotated object detection. | ||
|
|
||
| </details> | ||
|
|
||
| ## What's New | ||
|
|
||
| ### Highlight | ||
|
|
||
| We are excited to announce our latest work on real-time object recognition tasks, **RTMDet**, a family of fully convolutional single-stage detectors. RTMDet not only achieves the best parameter-accuracy trade-off on object detection from tiny to extra-large model sizes but also obtains new state-of-the-art performance on instance segmentation and rotated object detection tasks. Details can be found in the [technical report](https://arxiv.org/abs/2212.07784). Pre-trained models are [here](https://github.com/open-mmlab/mmrotate/tree/1.x/configs/rotated_rtmdet). | ||
|
|
||
| [](https://paperswithcode.com/sota/real-time-instance-segmentation-on-mscoco?p=rtmdet-an-empirical-study-of-designing-real) | ||
| [](https://paperswithcode.com/sota/object-detection-in-aerial-images-on-dota-1?p=rtmdet-an-empirical-study-of-designing-real) | ||
| [](https://paperswithcode.com/sota/object-detection-in-aerial-images-on-hrsc2016?p=rtmdet-an-empirical-study-of-designing-real) | ||
|
|
||
| | Task | Dataset | AP | FPS(TRT FP16 BS1 3090) | | ||
| | ------------------------ | ------- | ------------------------------------ | ---------------------- | | ||
| | Object Detection | COCO | 52.8 | 322 | | ||
| | Instance Segmentation | COCO | 44.6 | 188 | | ||
| | Rotated Object Detection | DOTA | 78.9(single-scale)/81.3(multi-scale) | 121 | | ||
|
|
||
| <div align=center> | ||
| <img src="https://user-images.githubusercontent.com/12907710/208044554-1e8de6b5-48d8-44e4-a7b5-75076c7ebb71.png"/> | ||
| </div> | ||
|
|
||
| **0.3.4** was released in 01/02/2023: | ||
|
|
||
| - Fix compatibility with numpy, scikit-learn, and e2cnn. | ||
| - Support empty patch in Rotate Transform | ||
| - use iof for RRandomCrop validation | ||
|
|
||
| Please refer to [changelog.md](docs/en/changelog.md) for details and release history. | ||
|
|
||
| ## Installation | ||
|
|
||
| MMRotate depends on [PyTorch](https://pytorch.org/), [MMCV](https://github.com/open-mmlab/mmcv) and [MMDetection](https://github.com/open-mmlab/mmdetection). | ||
| Below are quick steps for installation. | ||
| Please refer to [Install Guide](https://mmrotate.readthedocs.io/en/latest/install.html) for more detailed instruction. | ||
|
|
||
| ```shell | ||
| conda create -n open-mmlab python=3.7 pytorch==1.7.0 cudatoolkit=10.1 torchvision -c pytorch -y | ||
| conda activate open-mmlab | ||
| pip install openmim | ||
| mim install mmcv-full | ||
| mim install mmdet | ||
| git clone https://github.com/open-mmlab/mmrotate.git | ||
| cd mmrotate | ||
| pip install -r requirements/build.txt | ||
| pip install -v -e . | ||
| ``` | ||
|
|
||
| ## Get Started | ||
|
|
||
| Please see [get_started.md](docs/en/get_started.md) for the basic usage of MMRotate. | ||
| We provide [colab tutorial](demo/MMRotate_Tutorial.ipynb), and other tutorials for: | ||
|
|
||
| - [learn the basics](docs/en/intro.md) | ||
| - [learn the config](docs/en/tutorials/customize_config.md) | ||
| - [customize dataset](docs/en/tutorials/customize_dataset.md) | ||
| - [customize model](docs/en/tutorials/customize_models.md) | ||
| - [useful tools](docs/en/tutorials/useful_tools.md) | ||
|
|
||
| ## Model Zoo | ||
|
|
||
| Results and models are available in the *README.md* of each method's config directory. | ||
| A summary can be found in the [Model Zoo](docs/en/model_zoo.md) page. | ||
|
|
||
| <details open> | ||
| <summary><b>Supported algorithms:</b></summary> | ||
|
|
||
| - [x] [Rotated RetinaNet-OBB/HBB](configs/rotated_retinanet/README.md) (ICCV'2017) | ||
| - [x] [Rotated FasterRCNN-OBB](configs/rotated_faster_rcnn/README.md) (TPAMI'2017) | ||
| - [x] [Rotated RepPoints-OBB](configs/rotated_reppoints/README.md) (ICCV'2019) | ||
| - [x] [Rotated FCOS](configs/rotated_fcos/README.md) (ICCV'2019) | ||
| - [x] [RoI Transformer](configs/roi_trans/README.md) (CVPR'2019) | ||
| - [x] [Gliding Vertex](configs/gliding_vertex/README.md) (TPAMI'2020) | ||
| - [x] [Rotated ATSS-OBB](configs/rotated_atss/README.md) (CVPR'2020) | ||
| - [x] [CSL](configs/csl/README.md) (ECCV'2020) | ||
| - [x] [R<sup>3</sup>Det](configs/r3det/README.md) (AAAI'2021) | ||
| - [x] [S<sup>2</sup>A-Net](configs/s2anet/README.md) (TGRS'2021) | ||
| - [x] [ReDet](configs/redet/README.md) (CVPR'2021) | ||
| - [x] [Beyond Bounding-Box](configs/cfa/README.md) (CVPR'2021) | ||
| - [x] [Oriented R-CNN](configs/oriented_rcnn/README.md) (ICCV'2021) | ||
| - [x] [GWD](configs/gwd/README.md) (ICML'2021) | ||
| - [x] [KLD](configs/kld/README.md) (NeurIPS'2021) | ||
| - [x] [SASM](configs/sasm_reppoints/README.md) (AAAI'2022) | ||
| - [x] [Oriented RepPoints](configs/oriented_reppoints/README.md) (CVPR'2022) | ||
| - [x] [KFIoU](configs/kfiou/README.md) (arXiv) | ||
| - [x] [G-Rep](configs/g_reppoints/README.md) (stay tuned) | ||
|
|
||
| </details> | ||
|
|
||
| ## Data Preparation | ||
|
|
||
| Please refer to [data_preparation.md](tools/data/README.md) to prepare the data. | ||
|
|
||
| ## FAQ | ||
|
|
||
| Please refer to [FAQ](docs/en/faq.md) for frequently asked questions. | ||
|
|
||
| ## Contributing | ||
|
|
||
| We appreciate all contributions to improve MMRotate. Please refer to [CONTRIBUTING.md](.github/CONTRIBUTING.md) for the contributing guideline. | ||
|
|
||
| ## Acknowledgement | ||
|
|
||
| MMRotate is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new methods. | ||
|
|
||
| ## Citation | ||
|
|
||
| If you use this toolbox or benchmark in your research, please cite this project. | ||
| If you find this work helpful for your research, please consider giving this repo a star ⭐ and citing our paper: | ||
|
|
||
| ```bibtex | ||
| @inproceedings{zhou2022mmrotate, | ||
| title = {MMRotate: A Rotated Object Detection Benchmark using PyTorch}, | ||
| author = {Zhou, Yue and Yang, Xue and Zhang, Gefan and Wang, Jiabao and Liu, Yanyi and | ||
| Hou, Liping and Jiang, Xue and Liu, Xingzhao and Yan, Junchi and Lyu, Chengqi and | ||
| Zhang, Wenwei and Chen, Kai}, | ||
| booktitle={Proceedings of the 30th ACM International Conference on Multimedia}, | ||
| year={2022} | ||
| @article{li2024scene, | ||
| title={Scene Graph Generation in Large-Size VHR Satellite Imagery: A Large-Scale Dataset and A Context-Aware Approach}, | ||
| author={L1, Yansheng and Wang, Linlin and Wang, Tingzhu and Wang, Qi and Sun, Xian and Yang, Xue and Wang, Wenbin and Luo, Junwei and Deng, Youming and Li, Haifeng and Dang, Bo and Zhang, Yongjun and Yan Junchi}, | ||
| journal={arXiv preprint arXiv:}, | ||
| year={2024} | ||
| } | ||
| ``` | ||
|
|
||
| ## License | ||
|
|
||
| This project is released under the [Apache 2.0 license](LICENSE). | ||
|
|
||
| ## Projects in OpenMMLab | ||
|
|
||
| - [MMCV](https://github.com/open-mmlab/mmcv): OpenMMLab foundational library for computer vision. | ||
| - [MIM](https://github.com/open-mmlab/mim): MIM installs OpenMMLab packages. | ||
| - [MMClassification](https://github.com/open-mmlab/mmclassification): OpenMMLab image classification toolbox and benchmark. | ||
| - [MMDetection](https://github.com/open-mmlab/mmdetection): OpenMMLab detection toolbox and benchmark. | ||
| - [MMDetection3D](https://github.com/open-mmlab/mmdetection3d): OpenMMLab's next-generation platform for general 3D object detection. | ||
| - [MMRotate](https://github.com/open-mmlab/mmrotate): OpenMMLab rotated object detection toolbox and benchmark. | ||
| - [MMSegmentation](https://github.com/open-mmlab/mmsegmentation): OpenMMLab semantic segmentation toolbox and benchmark. | ||
| - [MMOCR](https://github.com/open-mmlab/mmocr): OpenMMLab text detection, recognition, and understanding toolbox. | ||
| - [MMPose](https://github.com/open-mmlab/mmpose): OpenMMLab pose estimation toolbox and benchmark. | ||
| - [MMHuman3D](https://github.com/open-mmlab/mmhuman3d): OpenMMLab 3D human parametric model toolbox and benchmark. | ||
| - [MMSelfSup](https://github.com/open-mmlab/mmselfsup): OpenMMLab self-supervised learning toolbox and benchmark. | ||
| - [MMRazor](https://github.com/open-mmlab/mmrazor): OpenMMLab model compression toolbox and benchmark. | ||
| - [MMFewShot](https://github.com/open-mmlab/mmfewshot): OpenMMLab fewshot learning toolbox and benchmark. | ||
| - [MMAction2](https://github.com/open-mmlab/mmaction2): OpenMMLab's next-generation action understanding toolbox and benchmark. | ||
| - [MMTracking](https://github.com/open-mmlab/mmtracking): OpenMMLab video perception toolbox and benchmark. | ||
| - [MMFlow](https://github.com/open-mmlab/mmflow): OpenMMLab optical flow toolbox and benchmark. | ||
| - [MMEditing](https://github.com/open-mmlab/mmediting): OpenMMLab image and video editing toolbox. | ||
| - [MMGeneration](https://github.com/open-mmlab/mmgeneration): OpenMMLab image and video generative models toolbox. | ||
| - [MMDeploy](https://github.com/open-mmlab/mmdeploy): OpenMMLab model deployment framework. | ||
| ## 📃 License | ||
|
|
||
| This project is released under the [Apache license](LICENSE). Parts of this project contain code and models from other sources, which are subject to their respective licenses. |
Oops, something went wrong.