[Benchmark] Support Video MCQ with TaskMeAnything-v1-video-random as an example

[weikaih04]

Hi,

For this PR

1. I added TaskMeAnything-v1-video-random video benchmark which includes 2700 video mcq questions.
2. Along with the benchmark, I found that unlike image mcq, video datasets don’t have a video_mcq.py file, which might hard for adding other mcq video benchmark. Therefore, I implemented video_mcq.py following the logic of image_mcq.py.

The usage of video_mcq.py is the same as image_mcq.py:
Just convert the benchmark to a TSV file, and encode the MP4 video to base64. I have provided the function named mp4_to_base64 in vlmeval/dataset/utils/video_mcq_utils.py.

I added the TaskMeAnything-v1-video-random video benchmark as an example for video_mcq.py and tested it on Paligemma (ImageQA model) and Video-LLaVA (VideoQA model), and it works well.

[junming-yang]

Hi, @weikaih04. We found that TaskMeAnything_v1_imageqa_random has a minor typo. The field of tsv file spelled category incorrectly, resulting in the final calculation result can only output the overall score, but not the category.

[weikaih04]

@junming-yang Hi, thank you for pointing this out! I’ve uploaded new versions of ImageQA and VideoQA and changed the md5, it should work now.

[JieyuZ2]

hey @junming-yang do you think it's ready to merge?

[kennymckormick]

Hi, @JieyuZ2 ,
I may need to take a look at this PR and run the evaluation. I will try to complete this in this week.

[JieyuZ2]

Hi, @JieyuZ2 , I may need to take a look at this PR and run the evaluation. I will try to complete this in this week.

Thanks!