Skip to content

Commit 9ec7076

Browse files
committed
Update CodeTrans default model align with docker compose
Signed-off-by: chensuyue <[email protected]>
1 parent d68c5ad commit 9ec7076

File tree

2 files changed

+6
-6
lines changed

2 files changed

+6
-6
lines changed

helm-charts/codetrans/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ scripts/update_dependency.sh
1414
helm dependency update codetrans
1515
export HFTOKEN="insert-your-huggingface-token-here"
1616
export MODELDIR="/mnt/opea-models"
17-
export MODELNAME="mistralai/Mistral-7B-Instruct-v0.3"
17+
export MODELNAME="Qwen/Qwen2.5-Coder-7B-Instruct"
1818
# To use CPU with vLLM
1919
helm install codetrans codetrans --set global.HF_TOKEN=${HFTOKEN} --set global.modelUseHostPath=${MODELDIR} --set llm-uservcie.LLM_MODEL_ID=${MODELNAME} --set vllm.LLM_MODEL_ID=${MODELNAME} -f codetrans/cpu-values.yaml
2020
# To use CPU with TGI
@@ -31,7 +31,7 @@ helm install codetrans codetrans --set global.HF_TOKEN=${HFTOKEN} --set global.m
3131

3232
### IMPORTANT NOTE
3333

34-
1. To use model `mistralai/Mistral-7B-Instruct-v0.3`, you should first goto the [huggingface model card](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) to apply for the model access first. You need to make sure your huggingface token has at least read access to that model.
34+
1. To use model `Qwen/Qwen2.5-Coder-7B-Instruct`, you should first goto the [huggingface model card](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) to apply for the model access first. You need to make sure your huggingface token has at least read access to that model.
3535

3636
2. Make sure your `MODELDIR` exists on the node where your workload is schedueled so you can cache the downloaded model for next time use. Otherwise, set `global.modelUseHostPath` to 'null' if you don't want to cache the model.
3737

@@ -70,5 +70,5 @@ Open a browser to access `http://<k8s-node-ip-address>:${port}` to play with the
7070
| ----------------- | ------ | -------------------------------------- | -------------------------------------------------------------------------------------- |
7171
| image.repository | string | `"opea/codetrans"` | |
7272
| service.port | string | `"7777"` | |
73-
| tgi.LLM_MODEL_ID | string | `"mistralai/Mistral-7B-Instruct-v0.3"` | Models id from https://huggingface.co/, or predownloaded model directory |
73+
| tgi.LLM_MODEL_ID | string | `"Qwen/Qwen2.5-Coder-7B-Instruct"` | Models id from https://huggingface.co/, or predownloaded model directory |
7474
| global.monitoring | bool | `false` | Enable usage metrics for the service components. See ../monitoring.md before enabling! |

helm-charts/codetrans/values.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,15 +60,15 @@ affinity: {}
6060
# To override values in subchart tgi
6161
tgi:
6262
enabled: false
63-
LLM_MODEL_ID: mistralai/Mistral-7B-Instruct-v0.3
63+
LLM_MODEL_ID: Qwen/Qwen2.5-Coder-7B-Instruct
6464

6565
vllm:
6666
enabled: true
67-
LLM_MODEL_ID: mistralai/Mistral-7B-Instruct-v0.3
67+
LLM_MODEL_ID: Qwen/Qwen2.5-Coder-7B-Instruct
6868

6969
llm-uservice:
7070
TEXTGEN_BACKEND: vLLM
71-
LLM_MODEL_ID: mistralai/Mistral-7B-Instruct-v0.3
71+
LLM_MODEL_ID: Qwen/Qwen2.5-Coder-7B-Instruct
7272

7373
nginx:
7474
service:

0 commit comments

Comments
 (0)