We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
用llama3.2-11b-vision 测试MTVQA的指标,您发布的为Overall:15.3,我实测为Overall:22.3812, 测试MME,您发布的指标为Perception:1380.9、Cognition:439.6,我实测为perception: 1319.5、Cognition:283 请问是什么原因?感谢,辛苦您解答或核实下榜单
The text was updated successfully, but these errors were encountered:
@FangXinyu-0913
Sorry, something went wrong.
您好,想问一下您是用我们的codebase进行评测吗,以及评测时的参数设定是怎么设定的呢,我们这边按您的设定再评测试一下
没改参数,只是config内设置了llama3.2_11b_vision的路径,.env中设置了openai的密钥。运行命令为 python run.py --data MME MTVQA_TEST --model Llama-3.2-11B-Vision-Instruct --nproc 4 --work-dir /workspace/mydir --verbose
感谢!
我们这边重新测试了一遍,和榜单上的指标没有过大差别 这是我们的环境 想问一下您的环境配置是怎样的呢
FangXinyu-0913
No branches or pull requests
用llama3.2-11b-vision
测试MTVQA的指标,您发布的为Overall:15.3,我实测为Overall:22.3812,
测试MME,您发布的指标为Perception:1380.9、Cognition:439.6,我实测为perception: 1319.5、Cognition:283
请问是什么原因?感谢,辛苦您解答或核实下榜单
The text was updated successfully, but these errors were encountered: