diff --git a/helm-charts/common/gpt-sovits/README.md b/helm-charts/common/gpt-sovits/README.md index a59568404..d9651d184 100644 --- a/helm-charts/common/gpt-sovits/README.md +++ b/helm-charts/common/gpt-sovits/README.md @@ -12,6 +12,34 @@ helm install gpt-sovits gpt-sovits --set global.modelUseHostPath=${MODELDIR} The gpt-sovits service will download model `lj1995/GPT-SoVITS` which is about 2.8GB. +### Install the microservice in air gapped (offline) mode + +To run `gpt-sovits` microservice in an air gapped environment, users are required to pre-download the model `lj1995/GPT-SoVITS` to a shared storage. + +Below is an example for using node level local directory to download the model data: + +Assuming the model data is shared using node-local directory `/mnt/opea-models`. + +``` +# On every K8s node, run the following command: +export MODEL_DIR=/mnt/opea-models +# Download model, assumes Python huggingface_hub[cli] module is already installed +huggingface-cli download --local-dir-use-symlinks False --local-dir "${MODEL_DIR}/lj1995/GPT-SoVITS" lj1995/GPT-SoVITS +# On K8s master node, run the following command: +# Install using Helm with the following additional parameters: +helm install ... ... --set global.offline=true,global.modelUseHostPath=${MODEL_DIR} +``` + +Assuming we share the offline data on cluster level using a persistent volume (PV), first we need to create the persistent volume claim (PVC) with name `opea-model-pvc` to store the model data. + +``` +# Download model openai/whisper-small at the root directory of the corresponding PV +# ... ... +# Install using Helm with the following additional parameters: +# export MODEL_PVC=opea-model-pvc +# helm install ... ... --set global.offline=true,global.modelUsePVC=${MODEL_PVC} +``` + ## Verify To verify the installation, run the command `kubectl get pod` to make sure all pods are running. @@ -40,9 +68,11 @@ curl localhost:9880/ -XPOST -d '{ ## Values -| Key | Type | Default | Description | -| ------------------------------- | ------ | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| image.repository | string | `"opea/gpt-sovits"` | | -| service.port | string | `"9880"` | | -| global.HUGGINGFACEHUB_API_TOKEN | string | `insert-your-huggingface-token-here` | Hugging Face API token | -| global.modelUseHostPath | string | `""` | Cached models directory, service will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to the container and the downloaded model will be saved to directory `lj1995/GPT-SoVITS`. Set this to null/empty will force it to download model. | +| Key | Type | Default | Description | +| ------------------------------- | ------ | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| image.repository | string | `"opea/gpt-sovits"` | | +| service.port | string | `"9880"` | | +| global.HUGGINGFACEHUB_API_TOKEN | string | `insert-your-huggingface-token-here` | Hugging Face API token | +| global.offline | bool | `false` | Whether to run the microservice in air gapped environment | +| global.modelUseHostPath | string | `""` | Cached models directory on Kubernetes node, service will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if `global.modelUsePVC` is also set. | +| global.modelUsePVC | string | `""` | Name of Persistent Volume Claim to use for model cache. The Persistent Volume will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if `global.modelUseHostPath` is also set. | diff --git a/helm-charts/common/gpt-sovits/templates/deployment.yaml b/helm-charts/common/gpt-sovits/templates/deployment.yaml index 32db46dea..e9e5c9ad7 100644 --- a/helm-charts/common/gpt-sovits/templates/deployment.yaml +++ b/helm-charts/common/gpt-sovits/templates/deployment.yaml @@ -28,6 +28,7 @@ spec: serviceAccountName: {{ include "gpt-sovits.serviceAccountName" . }} securityContext: {{- toYaml .Values.podSecurityContext | nindent 8 }} + {{- if not .Values.global.offline }} initContainers: - name: model-downloader envFrom: @@ -66,6 +67,7 @@ spec: name: model-volume - mountPath: /tmp name: tmp + {{- end }} containers: - name: gpt-sovits envFrom: @@ -115,8 +117,10 @@ spec: hostPath: path: {{ .Values.global.modelUseHostPath }} type: Directory - {{- else }} + {{- else if not .Values.global.offline }} emptyDir: {} + {{- else }} + {{- fail "'global.modelUsePVC' or 'global.modelUseHostPath' must be set in offline mode" }} {{- end }} - name: tmp emptyDir: {} diff --git a/helm-charts/common/gpt-sovits/values.yaml b/helm-charts/common/gpt-sovits/values.yaml index 57e289e2c..316697c16 100644 --- a/helm-charts/common/gpt-sovits/values.yaml +++ b/helm-charts/common/gpt-sovits/values.yaml @@ -85,6 +85,9 @@ global: # If set, and serviceAccount.create is false, it will assume this service account is already created by others. sharedSAName: "" + # Running microservice in air gapped (offline) mode + # If offline is enabled, user must set either modelUseHostPath or modelUsePVC and download model `lj1995/GPT-SoVITS`, i.e. + # huggingface-cli download --local-dir-use-symlinks False --local-dir /lj1995/GPT-SoVITS lj1995/GPT-SoVITS # Choose where to save your downloaded models # Set modelUseHostPath for local directory, this is good for one node test. Example: # modelUseHostPath: /mnt/opea-models