Skip to content

Commit 6f125af

Browse files
committed
feat(decoder): add support for granite models
Using straight Llama modeling for now -> generated content does not look so good, so maybe differences in rope ?
1 parent cb1a4d0 commit 6f125af

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

optimum/exporters/neuron/model_configs/decoder_configs.py

+6
Original file line numberDiff line numberDiff line change
@@ -63,3 +63,9 @@ class Qwen2NeuronConfig(TextNeuronDecoderConfig):
6363
NEURONX_CLASS = Qwen2ForSampling
6464
CONTINUOUS_BATCHING = True
6565
FUSE_QKV = False
66+
67+
68+
@register_in_tasks_manager("granite", "text-generation")
69+
class GraniteNeuronConfig(TextNeuronDecoderConfig):
70+
NEURONX_CLASS = LlamaForSampling
71+
CONTINUOUS_BATCHING = True

0 commit comments

Comments
 (0)