How to make blink effect?

[pango99]

below is the video I create with 5sec duration audio

ztest_Res512Case_retarget.mp4

and this is the video I create with 30sec duration audio

ztest4_no_retarget_30sec.mp4

the 5sec dura video has eye blink effect, but 30sec video haven't, why 30sec video without blink effect? How can I add blink effect to video?

[tiankuan93]

This vivid blinking action comes from the change in the v-kps sequence. You're at 30s audio, but the v-kps is only 5s, so our current code interpolates the 5s v-kps to 30s by default, which attenuates the change in v-kps corresponding to blinking, which results in some of the blinking disappearing. To fix this, I suggest you use a longer natural v-kps sequence, preferably around 30s to accompany the audio, which will better preserve the facial detail of the original v-kps.

Here I provide a longer v-kps sequence corresponding to the raw video, you can replace the input v-kps file with the longer kps file I provided and do the inference again. I hope this is helpful to you.

https://github.com/tencent-ailab/V-Express/blob/c9f3d7d02eb3181e79a693eada75e12b2a3114b0/test_samples/short_case/AOC/AOC_long_kps.pth