Skip to content

Commit b206d6c

Browse files
authored
Add sample inference configs for HuggingFaceTB/SmolVLM-Instruct (#1703)
1 parent edc471f commit b206d6c

File tree

2 files changed

+54
-0
lines changed

2 files changed

+54
-0
lines changed
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# SmolVLM Instruct inference config.
2+
#
3+
# Requirements:
4+
# - Run `pip install oumi[gpu]`
5+
#
6+
# Usage:
7+
# oumi infer -i -c configs/recipes/vision/smolvlm/inference/infer.yaml \
8+
# --image "tests/testdata/images/the_great_wave_off_kanagawa.jpg"
9+
#
10+
# See Also:
11+
# - Documentation: https://oumi.ai/docs/en/latest/user_guides/infer/infer.html
12+
# - Config class: oumi.core.configs.InferenceConfig
13+
# - Config source: https://github.com/oumi-ai/oumi/blob/main/src/oumi/core/configs/inference_config.py
14+
# - Other inference configs: configs/**/inference/
15+
16+
model:
17+
model_name: "HuggingFaceTB/SmolVLM-Instruct"
18+
torch_dtype_str: "bfloat16"
19+
model_max_length: 3072
20+
chat_template: "llava"
21+
trust_remote_code: True
22+
23+
generation:
24+
max_new_tokens: 32
25+
batch_size: 1
26+
27+
engine: NATIVE
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# vLLM inference config for SmolVLM Instruct.
2+
#
3+
# Requirements:
4+
# - Run `pip install oumi[gpu]`
5+
#
6+
# Usage:
7+
# oumi infer -i -c configs/recipes/vision/smolvlm/inference/vllm_infer.yaml \
8+
# --image "tests/testdata/images/the_great_wave_off_kanagawa.jpg"
9+
#
10+
# See Also:
11+
# - Documentation: https://oumi.ai/docs/en/latest/user_guides/infer/infer.html
12+
# - Config class: oumi.core.configs.InferenceConfig
13+
# - Config source: https://github.com/oumi-ai/oumi/blob/main/src/oumi/core/configs/inference_config.py
14+
# - Other inference configs: configs/**/inference/
15+
16+
model:
17+
model_name: "HuggingFaceTB/SmolVLM-Instruct"
18+
torch_dtype_str: "bfloat16"
19+
model_max_length: 3072
20+
chat_template: "llava"
21+
trust_remote_code: True
22+
23+
generation:
24+
max_new_tokens: 32
25+
batch_size: 1
26+
27+
engine: VLLM

0 commit comments

Comments
 (0)