Refactor GPT4V model and MMMU template YAML

Refactor the GPT4V model in lmms_eval/models/gpt4v.py to include a check for the "num_beams" key in the gen_kwargs dictionary. Also, update the MMMU template YAML file in lmms_eval/tasks/mmmu/_default_template_yaml to increase the value of "max_new_tokens" from 16 to 1024.
EvolvingLMMs-Lab · Jan 16, 2025 · ffe7ba1 · ffe7ba1
1 parent 34853d0
commit ffe7ba1
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/lmms_eval/tasks/mmmu/_default_template_yaml b/lmms_eval/tasks/mmmu/_default_template_yaml
@@ -1,5 +1,5 @@
 generation_kwargs:
-  max_new_tokens: 16
+  max_new_tokens: 1024
 
 metadata:
   version: 0.0