Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The image quality in VLMEvalKit is compressed and lower than that provided by Benchmark itself. #665

Open
Sync-yxh opened this issue Dec 13, 2024 · 1 comment

Comments

@Sync-yxh
Copy link

The image quality in VLMEvalKit is compressed and lower than that provided by Benchmark itself.

@SYuan03
Copy link
Contributor

SYuan03 commented Dec 18, 2024

Hi, @Sync-yxh
During our pre-processing, we have converted images to JPEG format. We will try to build another version that do not perform the JPEG conversion and do the evaluation again to see the gap.
We have noticed this in issues/578 and are working on building a new version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants