We present a method for cross-modal style transfer between an English sentence to an image, to produce a new image that imbibes the essential theme of the sentence. We do this by modifying the style transfer mechanism used in image style transfer to incorporate a style component derived from the given sentence.
This repo provides PyTorch Implementation of the Cross-Modal-Style transfer paper from:-
Paper:- https://ieeexplore.ieee.org/document/8451734
We demonstrate promising results using the YFCC100m dataset - link
The code benefits from outstanding prior work and their implementations including:
- Texture Networks: Feed-forward Synthesis of Textures and Stylized Images by Ulyanov et al. ICML 2016. (code)
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution by Johnson et al. ECCV 2016 (code) and its pytorch implementation code by Abhishek.
- Image Style Transfer Using Convolutional Neural Networks by Gatys et al. CVPR 2016 and its torch implementation code by Johnson.
- Skip-Thought Vectors. by Kiros et al.
- Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books. by Zhu et al.