Skip to content

gpt-sovits: Support air gapped environment #1130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
poussa merged 1 commit into from
Jun 24, 2025
Merged

gpt-sovits: Support air gapped environment #1130

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 36 additions & 6 deletions helm-charts/common/gpt-sovits/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,34 @@ helm install gpt-sovits gpt-sovits --set global.modelUseHostPath=${MODELDIR}

The gpt-sovits service will download model `lj1995/GPT-SoVITS` which is about 2.8GB.

### Install the microservice in air gapped (offline) mode

To run `gpt-sovits` microservice in an air gapped environment, users are required to pre-download the model `lj1995/GPT-SoVITS` to a shared storage.

Below is an example for using node level local directory to download the model data:

Assuming the model data is shared using node-local directory `/mnt/opea-models`.

```
# On every K8s node, run the following command:
export MODEL_DIR=/mnt/opea-models
# Download model, assumes Python huggingface_hub[cli] module is already installed
huggingface-cli download --local-dir-use-symlinks False --local-dir "${MODEL_DIR}/lj1995/GPT-SoVITS" lj1995/GPT-SoVITS
# On K8s master node, run the following command:
# Install using Helm with the following additional parameters:
helm install ... ... --set global.offline=true,global.modelUseHostPath=${MODEL_DIR}
Copy link
Collaborator

@eero-t eero-t Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
helm install ... ... --set global.offline=true,global.modelUseHostPath=${MODEL_DIR}
helm install ... --set global.offline=true,global.modelUseHostPath=${MODEL_DIR}

```

Assuming we share the offline data on cluster level using a persistent volume (PV), first we need to create the persistent volume claim (PVC) with name `opea-model-pvc` to store the model data.

```
# Download model openai/whisper-small at the root directory of the corresponding PV
# ... ...
# Install using Helm with the following additional parameters:
# export MODEL_PVC=opea-model-pvc
# helm install ... ... --set global.offline=true,global.modelUsePVC=${MODEL_PVC}
Comment on lines +37 to +40
Copy link
Collaborator

@eero-t eero-t Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO more readable with single ... continuation indicator:

Suggested change
# ... ...
# Install using Helm with the following additional parameters:
# export MODEL_PVC=opea-model-pvc
# helm install ... ... --set global.offline=true,global.modelUsePVC=${MODEL_PVC}
# ...
# Install using Helm with the following additional parameters:
# export MODEL_PVC=opea-model-pvc
# helm install ... --set global.offline=true,global.modelUsePVC=${MODEL_PVC}

```

## Verify

To verify the installation, run the command `kubectl get pod` to make sure all pods are running.
Expand Down Expand Up @@ -40,9 +68,11 @@ curl localhost:9880/ -XPOST -d '{

## Values

| Key | Type | Default | Description |
| ------------------------------- | ------ | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| image.repository | string | `"opea/gpt-sovits"` | |
| service.port | string | `"9880"` | |
| global.HUGGINGFACEHUB_API_TOKEN | string | `insert-your-huggingface-token-here` | Hugging Face API token |
| global.modelUseHostPath | string | `""` | Cached models directory, service will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to the container and the downloaded model will be saved to directory `lj1995/GPT-SoVITS`. Set this to null/empty will force it to download model. |
| Key | Type | Default | Description |
| ------------------------------- | ------ | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| image.repository | string | `"opea/gpt-sovits"` | |
| service.port | string | `"9880"` | |
| global.HUGGINGFACEHUB_API_TOKEN | string | `insert-your-huggingface-token-here` | Hugging Face API token |
| global.offline | bool | `false` | Whether to run the microservice in air gapped environment |
| global.modelUseHostPath | string | `""` | Cached models directory on Kubernetes node, service will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if `global.modelUsePVC` is also set. |
| global.modelUsePVC | string | `""` | Name of Persistent Volume Claim to use for model cache. The Persistent Volume will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if `global.modelUseHostPath` is also set. |
6 changes: 5 additions & 1 deletion helm-charts/common/gpt-sovits/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ spec:
serviceAccountName: {{ include "gpt-sovits.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
{{- if not .Values.global.offline }}
initContainers:
- name: model-downloader
envFrom:
Expand Down Expand Up @@ -66,6 +67,7 @@ spec:
name: model-volume
- mountPath: /tmp
name: tmp
{{- end }}
containers:
- name: gpt-sovits
envFrom:
Expand Down Expand Up @@ -115,8 +117,10 @@ spec:
hostPath:
path: {{ .Values.global.modelUseHostPath }}
type: Directory
{{- else }}
{{- else if not .Values.global.offline }}
emptyDir: {}
{{- else }}
{{- fail "'global.modelUsePVC' or 'global.modelUseHostPath' must be set in offline mode" }}
{{- end }}
- name: tmp
emptyDir: {}
Expand Down
3 changes: 3 additions & 0 deletions helm-charts/common/gpt-sovits/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ global:
# If set, and serviceAccount.create is false, it will assume this service account is already created by others.
sharedSAName: ""

# Running microservice in air gapped (offline) mode
# If offline is enabled, user must set either modelUseHostPath or modelUsePVC and download model `lj1995/GPT-SoVITS`, i.e.
# huggingface-cli download --local-dir-use-symlinks False --local-dir <modelUseHostPath>/lj1995/GPT-SoVITS lj1995/GPT-SoVITS
# Choose where to save your downloaded models
# Set modelUseHostPath for local directory, this is good for one node test. Example:
# modelUseHostPath: /mnt/opea-models
Expand Down