BachGAN: High-Resolution Image Synthesis from Salient Object Layout

BachGAN: High-Resolution Image Synthesis from Salient Object Layout
Yandong Li, Yu Cheng, Zhe Gan, Licheng Yu, Liqiang Wang, and Jingjing Liu
In CVPR 2020.

Qualitive Results

Examples of image synthesis results from different models on the Cityscapes dataset

Examples of image synthesis results from different models on the ADE20K dataset

Examples of generated images by adding bounding boxes sequentially on Cityscapes.

Examples of generated images by adding bounding boxes sequentially on ADE20K.

Usage

Follow the instructions in SPADE to prepare the Cityscapes and ADE20K datasets, and install the dependencies (PyTorch 1.0, python 3+, Synchronized-BatchNorm-PyTorch, etc.)

Download the pretrained models and retrieval results and unzip the downloaded file (BachGan.zip).

mv BachGan/ade_retrieval/validation.zip datasets/ADEChallengeData2016/retrival_halfset/
mv BachGan/city_retrieval/retrival_img_pairs_halfset_val_all.pkl scripts/retrival_ious_city_halfset/retrival_img_pairs_halfset_val_all.pkl
cd datasets/ADEChallengeData2016/retrival_halfset/
unzip validation.zip

If you want to get the retrieval results by yourself, please refer to scripts/retrival_ious_city_halfset/retrival_seg_halfset.py

Evaluate the pretrained model on Cityscapes.

mv BachGan/city_model/60_net_G.pth ./checkpoints/city_box_retrival_hallucinate/
python test.py --name city_box_retrival_hallucinate/ --dataset_mode cityscapes --dataroot datasets/cityscapes --retrival_memory --batchSize=10 --gpu_id=0,1 --no_instance --which_epoch=60

Download ADE20K meta data. And put the downloaded folder under ../

git clone https://github.com/CSAILVision/sceneparsing
mv sceneparsing ../

Evaluate the pretrained model on ADE20K.

mv BachGan/ade_model/150_net_G.pth ./checkpoints/ade_box_retrival_hallucinate/
python test.py --name ./ade_box_retrival_hallucinate/ --dataset_mode ade20k --dataroot ./datasets/ADEChallengeData2016/ --retrival_memory --batchSize=14 --gpu_id=0,1 --which_epoch=150

Citation

If you use this code for your research, please cite our papers.

@inproceedings{li2020BachGAN,
  title={BachGAN: High-Resolution Image Synthesis from Salient Object Layout},
  author={Li, Yandong and Cheng, Yu and Gan, Zhe and Yu, Licheng and Wang, Liqiang and Liu, Jingjing},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2020}
}

Acknowledgments

Pytorh-fid is from @mseitzer's implementation.
This code borrows heavily from pix2pixHD and SPADE. We thank Jiayuan Mao for his Synchronized Batch Normalization code.