Skip to content

Latest commit

 

History

History
25 lines (13 loc) · 479 Bytes

EVALUATION.md

File metadata and controls

25 lines (13 loc) · 479 Bytes
Raw

2. Non-training Few-shot Evaluation on downstream tasks

  • Image-text VQA

  • Image-text Captioning

  • Image-text Retrieval

  • Image Classification

  • Video-text VQA

  • Video-text Captioning

  • Video-text MC

  • 38 image-text datasets over datacomp

Simply, can test all tasks with one run

python src/main_eval_multiprocess.py  --base_config src/config/eval/eval_base_multiprocess.yaml --variant_config src/config/eval/mistral/7b_base_local.yaml --world_size 8