New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OlympiadBench数量跟论文不一致 #667

Open

wdp-007 opened this issue Dec 14, 2024 · 1 comment

wdp-007 commented Dec 14, 2024

代码里的https://opencompass.openxlab.space/utils/VLMEval/OlympiadBench.tsv这个文件里question只有6765道题，但我看原论文里数学有6142，物理有2334，加起来有8476道题才对，请问这个数据是做了筛减吗？

Member

kennymckormick commented Dec 17, 2024

Hi, @wdp-007 ,
We don't keep those Theorem Proving questions in the original dataset since those questions need to be verified by human.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment