AI-Generated Image Quality Refiner

Chunyi Li¹ Haoning Wu², Hongkun Hao¹, Zicheng Zhang¹, Tengchuan Kou¹, Chaofeng Chen², Xiaohong Liu¹, Lei Bai³, Weisi Lin², Guangtao Zhai¹

¹Shanghai Jiao Tong University University, ²Nanyang Technological University, ³Shanghai AI Lab

The official repo of AIGC image quality refiners:

[ICME 2024] Q-Refine: Single optimization in perceptual quality.

[ACMMM 2024] G-Refine: General optimization in perceptual/alignment quality.

🔎Quality Map

Perceptual Quality Map

python PQ-Map.py -p /orignal/image/path

Change draw to enable a quality map output. multi for using onr/multiple embedings. (Condisering overall quality, or quality in different aspect)

Alignment Quality Map

python AQ-Map.py -p /orignal/image/path -q "your prompt"

Change draw to enable a quality map output.

🚀 Full Refining Pipeline

Q-Refine

G-Refine

For the backbone of refining steps 1 and 2. First, please ensure you have the SDXL-Inpainting model. Then run https://github.com/yangxy/PASD.git to install PASD in this dictionary.

Then please use refine.ipynb to refine the alignment-example.png image. You may also use your own image and its corresponding prompt.

🌈Training

📌 TODO

• ✅ Release the PQ-Map code (Q-Refine and G-Refine)
• ✅ Release the AQ-Map code (G-Refine only)
• Release the Q-Refine code
• ✅ Release the G-Refine code
• Release the training script

📧 Contact

If you have any inquiries, please don't hesitate to reach out via email at [email protected]

🎓Citations

If you find G-Refine is helpful, please cite:

@misc{G-Refine,
      title={G-Refine: A General Quality Refiner for Text-to-Image Generation}, 
      author={Chunyi Li and Haoning Wu and Hongkun Hao and Zicheng Zhang and Tengchaun Kou and Chaofeng Chen and Lei Bai and Xiaohong Liu and Weisi Lin and Guangtao Zhai},
      year={2024},
      eprint={2404.18343},
      archivePrefix={arXiv},
      primaryClass={cs.MM}
}

If you find Q-Refine is helpful, please cite:

@misc{Q-Refine,
      title={Q-Refine: A Perceptual Quality Refiner for AI-Generated Image}, 
      author={Chunyi Li and Haoning Wu and Zicheng Zhang and Hongkun Hao and Kaiwei Zhang and Lei Bai and Xiaohong Liu and Xiongkuo Min and Weisi Lin and Guangtao Zhai},
      year={2024},
      eprint={2401.01117},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}