Warning: the repo is under construction so the code may not be runnable out of the box!
This is a repository containing torch7 code implementing
DeepWarp. Be sure to check out the
online demo. Due to licensing restrictions
I'm not able to release the original gaze manipulation code and corresponding dataset, but the model presented here
is very similar and you are free to use it as a reference point.
To showcase general applicability of the approach, I'm using an ever so slightly modified network to handle smile addition and removal in the images containg human faces.
The model (shown above) contains three networks:
- Transformer T. Warps the image given δ which is either +1 (add smile) or -1 (remove smile)
- Discriminator D. Makes sure that the warped output looks plausible.
- Attribute branch A. Infers attribute (1 for "smile", -1 for "no smile") value given an image.
The training objective is a weighted sum of two components:
where the first term is called the attribute loss and is a standard GAN cost.
Please cite the DeepWarp paper in your publications if this repo helps your research:
@inproceedings{ganin2016deepwarp,
title={DeepWarp: Photorealistic image resynthesis for gaze manipulation},
author={Ganin, Yaroslav and Kononenko, Daniil and Sungatullina, Diana and Lempitsky, Victor},
booktitle={European Conference on Computer Vision},
pages={311--326},
year={2016},
organization={Springer}
}