Skip to content

retriever-usvc: Adapt to support air gapped mode #1071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lianhao merged 1 commit into from
Jun 10, 2025
Merged

retriever-usvc: Adapt to support air gapped mode #1071

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 0 additions & 25 deletions helm-charts/common/retriever-usvc/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,31 +42,6 @@ helm install retriever-usvc . --set TEI_EMBEDDING_ENDPOINT=${TEI_EMBEDDING_ENDPO
# helm install retriever-usvc . --set TEI_EMBEDDING_ENDPOINT=${TEI_EMBEDDING_ENDPOINT} --set global.HUGGINGFACEHUB_API_TOKEN=${HF_TOKEN} --set RETRIEVER_BACKEND=${RETRIEVER_BACKEND} --set QDRANT_HOST=${DB_HOST}
```

### Install the microservice in air gapped(offline) mode

To support running this microservice in an air gapped environment, users are required to download the `nltk` data to a shared storage. Below is an example for using node level local directory to download the offline data:

Assuming the `nltk` data is ahred using node-local directory `/mnt/nltk_data`.

```
# On every K8s node, run the following command:
export NLTKDATA=/mnt/nltk_data
# Download nltk data, assumes Python nltk module(s) are already installed
python -m nltk.downloader -d $NLTKDATA all && chmod -R a+r $NLTKDATA

# Install using Helm with the following additional parameters:
# helm install ... ... --set global.offline=true,global.nltkDataUseHostPath=${NLTKDATA}
```

Assuming we share the `nltk` data on cluster level using a persistent volume(PV), first we need to create a persistent volume claim(PVC) with name `opea-nltk-pvc`:

```
# Download nltk data at the root of the corresponding PV
# Install using Helm with the following additional parameters:
# export NLTKPVC=opea-nltk-pvc
# helm install ... ... --set global.offline=true,global.nltkDataUsePVC=${NLTKPVC}
```

## Verify

To verify the installation, run the command `kubectl get pod` to make sure all pods are running.
Expand Down
2 changes: 1 addition & 1 deletion helm-charts/common/retriever-usvc/templates/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ data:
http_proxy: {{ .Values.global.http_proxy | quote }}
https_proxy: {{ .Values.global.https_proxy | quote }}
{{- if and (and (not .Values.MILVUS_HOST ) (not .Values.REDIS_HOST)) (and (not .Values.TEI_EMBEDDING_ENDPOINT) (or .Values.global.http_proxy .Values.global.https_proxy)) }}
no_proxy: "{{ .Release.Name }}-tei,{{ .Release.Name }}-redis-vector-db,{{ .Release.Name }}-milvus,{{ .Values.global.no_proxy }}"
no_proxy: "{{ .Release.Name }}-tei,{{ .Release.Name }}-redis-vector-db,{{ .Release.Name }}-milvus,{{ .Release.Name }}-qdrant,{{ .Values.global.no_proxy }}"
{{- else }}
no_proxy: {{ tpl .Values.global.no_proxy . | quote }}
{{- end }}
Expand Down
17 changes: 0 additions & 17 deletions helm-charts/common/retriever-usvc/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,6 @@ spec:
volumeMounts:
- mountPath: /tmp
name: tmp
- mountPath: /home/user/nltk_data
name: nltk-data
{{- if .Values.livenessProbe }}
livenessProbe:
{{- toYaml .Values.livenessProbe | nindent 12 }}
Expand All @@ -101,21 +99,6 @@ spec:
volumes:
- name: tmp
emptyDir: {}
- name: nltk-data
{{- if .Values.global.offline }}
{{- if .Values.global.nltkDataUsePVC }}
persistentVolumeClaim:
claimName: {{ .Values.global.nltkDataUsePVC }}
{{- else if .Values.global.nltkDataUseHostPath }}
hostPath:
path: {{ .Values.global.nltkDataUseHostPath }}
type: Directory
{{- else }}
{{- fail "'global.nltkDataUsePVC' or 'global.nltkDataUseHostPath' must be set in offline mode" }}
{{- end }}
{{- else }}
emptyDir: {}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
Expand Down
10 changes: 0 additions & 10 deletions helm-charts/common/retriever-usvc/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,16 +124,6 @@ global:
# If set, and serviceAccount.create is false, it will assume this service account is already created by others.
sharedSAName: ""

# Running service in air gapped(offline) mode
# If offline is enabled, user must set either nltkDataUseHostPath or nltkDataUsePVC and download nltk data out of band.
offline: false
# To store offline nltk data in local directory for one node K8s environment, set nltkDataUseHostPath
# Download nltk data: python -m nltk.downloader -d <nltkDataUseHostPath> all && chmod a+r <nltkDataUseHostPath>
nltkDataUseHostPath: ""
# To store offline nltk data in persistent volume(PV) to be shared by multinode K8s environment, set nltkDataUsePVC
# and follow the similar above steps to download the nltk data to the root directory of that PV
nltkDataUsePVC: ""

# Install Prometheus serviceMonitor for service
monitoring: false

Expand Down