How to produce video with lower resolution

[pango99]

the time for V-Express to produce video is too long, it takes 800sec to produce 10sec video on my RTX4070 card,I notice the output video's resolution is 512x512, I think if I reduce the resolution, the time should be shorten, so I use below command to do the test:

python inference.py ^
--reference_image_path "./test_samples/short_case/AOC/ref.jpg" ^
--audio_path ".\test_samples\short_case\10\aud.mp3" ^
--kps_path ".\test_samples\short_case\AOC\kps.pth" ^
--output_path ".\output\ztest4_no_retarget.mp4" ^
--retarget_strategy "no_retarget" ^
--num_inference_steps 25 ^
--image_width 128 ^
--image_height 128

the test result shows the time of video generation is reduced greatly, but the output video is wrong,it always display the ref image,so what's wrong with my command? Can I produce video with lower resolution?

https://github.com/tencent-ailab/V-Express/assets/6065571/24f3b8bd-a4de-47d1-bb02-a860878e4bb8

[pango99]

I had generated the kps sequence for 256x256 resolution,and use it to produce 256x256 res video use below command,

python inference.py ^
--reference_image_path "./test_samples/Res256_case/AOC/ref.jpg" ^
--audio_path "./test_samples/Res256_case/AOC/aud.mp3" ^
--kps_path ".\test_samples\Res256_case\AOC\kps.pth" ^
--output_path ".\output\ztest_Res256Case_retarget.mp4" ^
--retarget_strategy "no_retarget" ^
--num_inference_steps 25 ^
--image_height 256 ^
--image_width 256

now in the produced video, the face and mouse is moved, but the effect is not good as Res512 video, is it because 512x512 is the smallest resolution?

ztest_Res256Case_retarget_1.mp4

[tiankuan93]

We are using portrait videos of 512x512 size for our training, so the results may not be perfect for small sized faces. Maybe you can try cropping a 384x384 reference image with less background, but keeping the face size the same, as follows.