File tree Expand file tree Collapse file tree 2 files changed +54
-0
lines changed
configs/recipes/vision/smolvlm/inference Expand file tree Collapse file tree 2 files changed +54
-0
lines changed Original file line number Diff line number Diff line change
1
+ # SmolVLM Instruct inference config.
2
+ #
3
+ # Requirements:
4
+ # - Run `pip install oumi[gpu]`
5
+ #
6
+ # Usage:
7
+ # oumi infer -i -c configs/recipes/vision/smolvlm/inference/infer.yaml \
8
+ # --image "tests/testdata/images/the_great_wave_off_kanagawa.jpg"
9
+ #
10
+ # See Also:
11
+ # - Documentation: https://oumi.ai/docs/en/latest/user_guides/infer/infer.html
12
+ # - Config class: oumi.core.configs.InferenceConfig
13
+ # - Config source: https://github.com/oumi-ai/oumi/blob/main/src/oumi/core/configs/inference_config.py
14
+ # - Other inference configs: configs/**/inference/
15
+
16
+ model :
17
+ model_name : " HuggingFaceTB/SmolVLM-Instruct"
18
+ torch_dtype_str : " bfloat16"
19
+ model_max_length : 3072
20
+ chat_template : " llava"
21
+ trust_remote_code : True
22
+
23
+ generation :
24
+ max_new_tokens : 32
25
+ batch_size : 1
26
+
27
+ engine : NATIVE
Original file line number Diff line number Diff line change
1
+ # vLLM inference config for SmolVLM Instruct.
2
+ #
3
+ # Requirements:
4
+ # - Run `pip install oumi[gpu]`
5
+ #
6
+ # Usage:
7
+ # oumi infer -i -c configs/recipes/vision/smolvlm/inference/vllm_infer.yaml \
8
+ # --image "tests/testdata/images/the_great_wave_off_kanagawa.jpg"
9
+ #
10
+ # See Also:
11
+ # - Documentation: https://oumi.ai/docs/en/latest/user_guides/infer/infer.html
12
+ # - Config class: oumi.core.configs.InferenceConfig
13
+ # - Config source: https://github.com/oumi-ai/oumi/blob/main/src/oumi/core/configs/inference_config.py
14
+ # - Other inference configs: configs/**/inference/
15
+
16
+ model :
17
+ model_name : " HuggingFaceTB/SmolVLM-Instruct"
18
+ torch_dtype_str : " bfloat16"
19
+ model_max_length : 3072
20
+ chat_template : " llava"
21
+ trust_remote_code : True
22
+
23
+ generation :
24
+ max_new_tokens : 32
25
+ batch_size : 1
26
+
27
+ engine : VLLM
You can’t perform that action at this time.
0 commit comments