Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Anybody else getting washed out colors #42

Open
oisilener1982 opened this issue Jun 17, 2024 · 13 comments
Open

Anybody else getting washed out colors #42

oisilener1982 opened this issue Jun 17, 2024 · 13 comments

Comments

@oisilener1982
Copy link

oisilener1982 commented Jun 17, 2024

The output quality of v-express is not that great. Sadtalker does not have this problem. I hope tencent would be able to fix this

@tiankuan93
Copy link
Collaborator

What kind of script, data did you use? And what kind of results?

@oisilener1982
Copy link
Author

oisilener1982 commented Jun 17, 2024

I used this tutorial to install locally: https://www.youtube.com/watch?v=ttEOIg9j2B4&t=327s

These are the sample results and the output does not have the same quality as the reference.
Capture2
Capture

The video below was created using sadtalker and it retained the skin color and texture.
https://youtube.com/shorts/qJR0m4jGeGw?feature=share

This is the Video output from V-express
https://youtube.com/shorts/LCOiMrVLwaE?feature=share

@tiankuan93
Copy link
Collaborator

Since we only use portrait data for training, we currently only support a large and clear face image. Instead of a bust photo, you should enter an input like the one below.

image

@oisilener1982
Copy link
Author

oisilener1982 commented Jun 18, 2024

Since we only use portrait data for training, we currently only support a large and clear face image. Instead of a bust photo, you should enter an input like the one below.

image

Thanks for the clarification but in the image below the reference image is high quality with clear skin texture while the output video has washed out color and too bright light. The animation is actually good and much better than Sadtalker but v-express has some output quality issues. IF this can be fixed then i will definitely switch to v-express

1111

@zhangjun001
Copy link
Collaborator

zhangjun001 commented Jun 18, 2024

I did not test the gradio version. The ComfyUI version works well for me.
My settings are shown bellow:
image

test3.53.mp4

@oisilener1982
Copy link
Author

oisilener1982 commented Jun 18, 2024

I just installed comfyui but i dont have any idea :)

Hope someone could post a tutorial about how to use v-express in comfyui

@oisilener1982
Copy link
Author

oisilener1982 commented Jun 18, 2024

I did not test the gradio version. The ComfyUI version works well for me. My settings are shown bellow: image

test3.53.mp4

Would it be ok if you try this exact image?
1234

This is my output

NEw.mp4

@zhangjun001
Copy link
Collaborator

zhangjun001 commented Jun 19, 2024

Here are some results from my test

test3.61.mp4
test3.63.mp4
test3.59.mp4

@zhangjun001
Copy link
Collaborator

zhangjun001 commented Jun 19, 2024

Take a look at this one. I adjusted the parameter of cfg=1.5.

test3.68.mp4
test3.69.mp4

@zhangjun001
Copy link
Collaborator

It's our fault for not giving good instructions and guidance. Maybe after some time, I can provide a simple tutorial to tell you how to select kps videos and how to set parameters.

@oisilener1982
Copy link
Author

oisilener1982 commented Jun 19, 2024

Thanks for this.
Test3.61 and test3.63 both have washed up color. Test3.59 and Test3.68 are fine but a lot can still be improved in terms of quality. What settings do you have for 59 and 68? Is it just the cfg 1.5 what was adjusted? Thanks in advance

I hope you improve the skin color and texture to match better the Reference image.

@oisilener1982
Copy link
Author

oisilener1982 commented Jun 19, 2024

This is what i mean with better skin texture and output that closely resemble the Reference image. I made this from sadtalker but sadly the emotional expression is missing and v-express is way better in this aspect.

xxxxxx.mp4

@zhangjun001
Copy link
Collaborator

zhangjun001 commented Jun 20, 2024

Thanks for this. Test3.61 and test3.63 both have washed up color. Test3.59 and Test3.68 are fine but a lot can still be improved in terms of quality. What settings do you have for 59 and 68? Is it just the cfg 1.5 what was adjusted? Thanks in advance

I hope you improve the skin color and texture to match better the Reference image.

Since I want to do it using a retargeting strategy, the results will be more natural. And because naive-retarget cannot be effective for any facial posture, choosing a kps video with a roughly similar facial posture to drive can often achieve better results.

image
test3.74.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants