Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GFP-GAN: new lab added #21

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
432 changes: 432 additions & 0 deletions week13/labs/GFPGAN_demo_w251.ipynb

Large diffs are not rendered by default.

10 changes: 10 additions & 0 deletions week13/labs/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,14 @@ DALI stands for Data Loading Library. This Open Source project is another tool
* Start the pytorch ngc container, e.g. ```docker run --rm --gpus all -p 8888:8888 --ipc=host -ti nvcr.io/nvidia/pytorch:22.03-py3``` and then start jupyter lab , e.g. ```jupyter lab --ip=0.0.0.0 --allow-root```
* Upload the [Getting Started](getting_started_dali.ipynb) example to your jupyter environment and run through it. What acceleration did you get in the last cell? Can you suggest what contributes to that number?

### GFP-GAN
GFP-GAN stands for "Generative Facial Prior Generative Adversarial Network". It is used for blind face restoration. GFP is incorporated into the face restoration process via channel-split spatial feature transform layers, which allow for a good balance between realness and fidelity. The goal of this lab is to experiment with the model(s) and evaluate the GAN performance against a baseline of your own photo, taken with the camera on your device and artificially converted into a retro image.

Try the following:
* minimal setup: deploy and run CPU-based model on your desktop or laptop, levaraging the provided [Jupyter notebook](https://github.com/alsavelv/v3/blob/main/week13/labs/GFPGAN_demo_w251.ipynb) for generating a photo collage
* more advanced: do the same on your Jetson
* extra bonus: deploy and run GPU-based model with colorization (either on Jetson or on your device, if it has GPU)

Detailed instructions for model installation and scripts for running inferences are provided both in [the repo](https://github.com/TencentARC/GFPGAN). Please copy the Jupyter notebook to the root of the cloned repo and run it from there, as it executes some commands that depend on this code. The notebook was created for a Windows-based setup. Minor changes to scripts will be required to make it compatible with Mac and Linux. Also, make sure to update directory paths to match your setup.

Once you get the inference result, you can experiment with different types of filters, increasing or decreasing the level of photo distortion, image resolution, using colored or monochrome images, etc. Once you get sufficient impression, think critically about the quality of the output produced by the model. Keep in mind that the photo impression is created by a combination of many factors, including facial features, emotions, colors, etc. Imagine, for the sake of an argument, that in a few hundred years the "vintage" version of the photo is the only digital trace that left of you for your ancestors; would you be comfortable to let them rely on the result of processing it with GFP-GAN, or prefer to use the low-quality image as is? This perspective is important, because people whose images were taken in late 19th/20th century are in this position, having their old photographs restored with AI. Think about possible impacts of inaccurate photo restoration in other scenarios. Also, consider whether overreliance on AI in general and for photo restoration specifically can be mitigated, and in which ways.
Binary file added week13/labs/photo template landscape mids.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.