Add 3DSRBench

[wufeim]

Add support for 3DSRBench. Data files are hosted on huggingface.co.

[PhoenixZ810]

Hi,
Thank you for submitting 3DRSBench to our attention. We have reviewed your contribution and noticed that while the link to 3DRSBench has been provided, the specific evaluation metrics have not yet been included.

Please complete the detailed evaluation method for your benchmark. Once we have this information, we will be able to proceed with adding 3DRSBench to the VLMEvalkit.

Best regards

[wufeim]

Hi, Thank you for submitting 3DRSBench to our attention. We have reviewed your contribution and noticed that while the link to 3DRSBench has been provided, the specific evaluation metrics have not yet been included.

Please complete the detailed evaluation method for your benchmark. Once we have this information, we will be able to proceed with adding 3DRSBench to the VLMEvalkit.

Best regards

Thanks for the feedback. I see that from the documentation certain functions may need to be implemented. However, 3DSRBench follows the standard multiple choice VQA datasets, and can be evaluated with the following command:

python3 run.py --data 3DSRBenchv1 --model GPT4o_20240806

I assume no other functions need to be implemented? Otherwise could you point me to the relevant functions/metrics that need to be provided? Thanks!

[PhoenixZ810]

Hi, Thank you for submitting 3DRSBench to our attention. We have reviewed your contribution and noticed that while the link to 3DRSBench has been provided, the specific evaluation metrics have not yet been included.
Please complete the detailed evaluation method for your benchmark. Once we have this information, we will be able to proceed with adding 3DRSBench to the VLMEvalkit.
Best regards

Thanks for the feedback. I see that from the documentation certain functions may need to be implemented. However, 3DSRBench follows the standard multiple choice VQA datasets, and can be evaluated with the following command:

python3 run.py --data 3DSRBenchv1 --model GPT4o_20240806

I assume no other functions need to be implemented? Otherwise could you point me to the relevant functions/metrics that need to be provided? Thanks!

Understood. We will proceed to validate this pr shortly. Thank you for your patience.