Input live webcam video for live streaming?

[vallamost]

Has anyone tried or thought about the possibility to use VToonify with live image inputs from a webcam or virtual camera? Or utilizing it for livestreams?

Someone mentioned being able to use this on Linux - https://github.com/umlaeute/v4l2loopback

[joshdance]

Is VTonnify fast enough to support live processing? I haven't used it yet, researching, but I thought you had to process videos.

[williamyang1991]

Since VToonify is based on a big model of StyleGAN/DualStyleGAN, it is currently not real time.
We test on generating 1600×1280 videos, the ideal running time excluding video reading/writing on a NVIDIA Tesla V100
GPU is about 0.2s per frame.
With video reading/writing and other processes for live stream, the time for each frame will be even greater.

[ileocho]

In your opinion, could it potentially run on lower-resolution videos ? Or is this model specific for high-res ?

[williamyang1991]

For low-resolution, I think it is possible to use stylegan-256 rather than stylegan-1024,
and train the corresponding dualstylegan and vtoonify, which needs some code modifications I suppose.