v2.0.0

.gitignore

@@ -1,5 +1,4 @@
-data/*
-model/*
-out/*
 .vscode/
-__pycache__/
+.idea/
+
+__pycache__/

cal_QualityScore.m → Matlab_based_IQAM/main.m

@@ -1,4 +1,6 @@
-function [score_quality] = cal_QualityScore(img, type)
+function [score_quality] = main(img, type)
+% img: (H W) 0-255 grayscale image.
+% type: HEVC or JPEG.
 
 %% Settings
 if type == "HEVC"

README.md

@@ -2,15 +2,9 @@
 
 - [*Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images* (ECCV 2020)](#early-exit-or-not-resource-efficient-blind-quality-enhancement-for-compressed-images-eccv-2020)
   - [0. Background](#0-background)
-  - [1. Pre-request](#1-pre-request)
-    - [1.1. Environment](#11-environment)
-    - [1.2. Data and pre-trained models](#12-data-and-pre-trained-models)
-    - [1.3. Compress images](#13-compress-images)
-  - [2. Test](#2-test)
-  - [3. Training](#3-training)
-  - [4. License & Citation](#4-license--citation)
-
-:rocket: **Update** (21/4/27): We release the improved RBQE, training code and Python-based IQA module at [PowerQE](https://github.com/RyanXingQL/PowerQE). Codes of all compared approaches are also released.
+  - [1. Codes and Pre-trained Models](#1-codes-and-pre-trained-models)
+  - [2. Difference from the Paper](#2-difference-from-the-paper)
+  - [3. License & Citation](#3-license--citation)
 
 ## 0. Background
 
@@ -22,90 +16,23 @@ Official repository of [*Early Exit or Not: Resource-Efficient Blind Quality Enh
 
 ![network](https://user-images.githubusercontent.com/34084019/105739729-637dd200-5f73-11eb-923a-bb67ee9959eb.png)
 
-Feel free to contact: <[email protected]>.
-
-## 1. Pre-request
-
-### 1.1. Environment
-
-PYTHON 3.7, PYTORCH > 1.0, PILLOW, IMAGEIO:
-
-```bash
-conda create -n rbqe python=3.7 pillow=7.1.2 libtiff=4.1.0 imageio=2.9.0
-conda activate rbqe
-conda install -c pytorch pytorch=1.5
-```
-
-MATLAB R2019b.
-
-### 1.2. Data and pre-trained models
-
-All files below are prepared in GitHub Releases or [[百度网盘 (rbqe)]](https://pan.baidu.com/s/1U9BtmZVxno_ZAON17XRBjg). For demo, we prepare only 5 raw TIFF images.
-
-### 1.3. Compress images
-
-We use [RAISE](http://loki.disi.unitn.it/RAISE/) as raw image dataset. Download the TIFF images in RAISE, or prepare your own raw images.
-
-<details>
-<summary><b>Overview</b></summary>
-<p>
-
-- To generate HEVC-MSP-compressed test set:
-  - center-crop raw images into `512x512`, considering that some compared approaches can not process larger images with prevalent GPUs.
-  - stack images into a YUV video, which is convenient for compression by HM16.5.
-  - compress this raw YUV video into 5 compressed YUV videos with 5 different QPs by HM16.5 (mode: main still picture, MSP). Therefore, each YUV video is a batch of images with the same QP.
-- To generate JPEG-compressed test set:
-  - compress each raw image into 5 compressed images with 5 different QFs by Python Pillow.
-  - center-crop raw images into `512x512`.
-  - stack images with the same QF into one YUV video. Therefore, 5 QFs correspond to 5 compressed YUV videos. Besides, raw images are also stacked into a raw YUV video (this video may be different from the raw video for HEVC experiment because the image order may be different).
-
-</p>
-</details>
-
-To generate HEVC-MSP-compressed test set:
-
-1. `main_tiff2yuv420p.m`: Center-crop these images into `512x512` images, and stack them into a single YUV video.
-2. `main_compress.bat` (Windows system): Compress this yuv with 5 different QPs: 22, 27, 32, 37 and 42. Then we get 5 YUV videos: `RAISE_qp22_512x512_test.yuv`, `RAISE_qp27_512x512_test.yuv`, `RAISE_qp32_512x512_test.yuv`, `RAISE_qp37_512x512_test.yuv`, and `RAISE_qp42_512x512_test.yuv`.
-
-Note: you can also compress the YUV video on Ubuntu system using `main_compress.sh`.
-
-To generate JPEG-compressed test set:
-
-1. `python main_JPEG_compression.py`: Compress these images with 5 different QFs: 10, 20, 30, 40 and 50. Then we get 5 JPEG images for each raw image.
-2. `main_jpeg2yuv420p.m`: Center-crop these images into `512x512` images, and stack them into 5 YUV videos: `RAISE_raw_512x512_test_jpeg.yuv`, `RAISE_qf10_512x512_test_jpeg.yuv`, `RAISE_qf20_512x512_test_jpeg.yuv`, `RAISE_qf30_512x512_test_jpeg.yuv`, `RAISE_qf40_512x512_test_jpeg.yuv`, and `RAISE_qf50_512x512_test_jpeg.yuv`.
-
-## 2. Test
-
-<details>
-<summary><b>Overview</b></summary>
-<p>
-
-- Test all compressed YUV videos (actually compressed images in batches). For each compressed image, we obtain 5 enhanced images corresponding to 5 outputs of the network. Note that we do not use early-exit in this step, because we want to observe the PSNR vs. FLOPs performance under different threshold `T` in the next step. Therefore, the ave result in this step is not the final result.
-- Evaluate quality score of each enhanced images by our Tchebichef-moments based IQA model.
-- Generate the final PSNR vs. FLOPs result under one chosen threshold `T`.
-
-</p>
-</details>
-
-1. `python main_test.py -t HEVC -g 0`: test HEVC-compressed images, using gpu 0.
-   Or: `python main_test.py -t JPEG -g 0`: test JPEG-compressed images, using gpu 0.
-2. `main_cal_QualityScore.m`: change `type_test` into `HEVC` or `JPEG` in line 3, then run it by MATLAB.
-3. `python main_tradeoff.py -t HEVC` or `python main_tradeoff.py -t JPEG`
-   - EXP 1: Ablation. We force all QP=i (i=22,27,32,37,42) images to output at output=k (k=1,2,3,4,5), and observe the PSNR vs. FLOPs performance.
-   - EXP 2: Tradeoff curve. We draw the PSNR vs. FLOPs under different threshold `T`.
-   - EXP 3: Optimal result. As stated in our paper, we choose the turning point of the curve as the optimal `T`. Based on curves in EXP 2, we choose `T=0.84` for HEVC dataset and `T=0.67` for JPEG dataset.
+Feel free to contact: `[email protected]`.
 
-**Note**: we use `T=0.89` for 1000-image HEVC dataset in our paper and `T=0.79` for 1000-image JPEG dataset in our paper. Here we have only 5 images, and the threshold `T` is re-chosen according to the curve.
+## 1. Codes and Pre-trained Models
 
-**Note**: the curve may not be smooth, since we have only 5 images here.
+To unify most of the quality enhancement approaches, we have released the improved RBQE at [PowerQE](https://github.com/RyanXingQL/PowerQE). Codes of all compared approaches are also presented there.
 
-![result](https://user-images.githubusercontent.com/34084019/105739748-68428600-5f73-11eb-9195-959682b67981.png)
+A Python-based image quality assessment module (IQAM) is provided at [PowerQE](https://github.com/RyanXingQL/PowerQE). The MATLAB-based IQAM is provided at this repository, which is much, much faster.
 
-## 3. Training
+## 2. Difference from the Paper
 
-See [PowerQE](https://github.com/RyanXingQL/PowerQE).
+1. **Dataset**. In the paper, we use the high-resolution RAISE dataset. In [PowerQE](https://github.com/RyanXingQL/PowerQE), the commonly-used DIV2K dataset is adopted for all approaches.
+2. **Image compression**. In the paper, we use HM software to obtain HEVC-compressed images. In [PowerQE](https://github.com/RyanXingQL/PowerQE), BPG is adopted to obtain compressed images, which is simpler.
+3. **YCbCr or RGB**. In the paper, we only enhance the Y channel and report the Y-PSNR result. In [PowerQE](https://github.com/RyanXingQL/PowerQE), since the input images are with PNG format, we enhance all R, G and B channels.
+4. **Image quality assessment (IQA)**. In the paper, the IQA is conducted on the Y channel, and the threshold of IQA module (IQAM) is determined according to the Y performance. In [PowerQE](https://github.com/RyanXingQL/PowerQE), we conduct IQA on R channel for simplicity and then set the threshold.
+5. **IQA implementation**. In [PowerQE](https://github.com/RyanXingQL/PowerQE), a Python-based IQAM is provided for an end-to-end Python experience. In the paper, we use a MATLAB-based IQAM, which is much faster but independent of the Python-based enhancement model. The MATLAB-based IQAM is provided at this repository. Note that the thresholds are different between two versions.
 
-## 4. License & Citation
+## 3. License & Citation
 
 You can **use, redistribute, and adapt** the material for **non-commercial purposes**, as long as you give appropriate credit by **citing our paper** and **indicating any changes** that you've made.