Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions Chart.lock

This file was deleted.

14 changes: 1 addition & 13 deletions Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,10 @@ apiVersion: v2
name: private-ai
description: A Helm chart that deploys the Private AI deidentification solution

version: 1.5.0
version: 1.6.0

# This is the default version deployed by the chart.
# However, this is simply a label - if you wish to change the version, update your values.yaml
appVersion: "4.2.2-cpu"

icon: https://cdn.prod.website-files.com/6682257d9fc729d59a7f121b/66cba74e2ef9cbe1c20e261c_favicon.png

dependencies:
- name: cert-manager
repository: oci://quay.io/jetstack/charts
version: v1.18.2
alias: certmanager
condition: certmanager.enabled
- name: ingress-nginx
repository: https://kubernetes.github.io/ingress-nginx
version: 4.14.1
alias: ingressnginx
condition: ingressnginx.enabled
193 changes: 164 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,19 +29,18 @@ kubectl -n private-ai create secret docker-registry crprivateaiprod-creds \
helm registry login crprivateaiprod.azurecr.io

# Create a custom values file for your specific installation
helm show values oci://crprivateaiprod.azurecr.io/helm/private-ai:1.5.0 > values.custom.yaml
helm show values oci://crprivateaiprod.azurecr.io/helm/private-ai:1.6.0 > values.custom.yaml

# Copy your license.json file contents and paste them into the license.data section of the values.custom.yaml file with single quotes surrounding, as per below
license:
data: '{"id":"..."}'

# Upgrade or install the Private AI chart with a name and namespace of private-ai
helm upgrade --install \
--namespace private-ai \
private-ai \
-f values.custom.yaml \
oci://crprivateaiprod.azurecr.io/helm/private-ai \
--version 1.5.0
private-ai oci://crprivateaiprod.azurecr.io/helm/private-ai \
--namespace private-ai \
-f values.custom.yaml \
--version 1.6.0
```

## Testing
Expand All @@ -64,14 +63,59 @@ helm uninstall --namespace private-ai private-ai
To customize your deployment, enable different sections of your values.yaml file as per the documentation below.

### Ingress Controller
If you would like to set up an external ingress to enable external traffic to reach your the Private AI deployment, you must enable the ingress-nginx and cert-manager helm charts in the values.yaml file. Additionally, a sample ingress deployment file is included, and can be deployed with self-signed certificates for testing.
If you would like to set up an external ingress to enable external traffic to reach your the Private AI deployment, you must enable the haproxy-ingress and cert-manager helm charts in the values.yaml file. Additionally, a sample ingress deployment file is included, and can be deployed with self-signed certificates for testing.

If you would like to deploy your own certificate issuer and certificates, please see the [cert-manager docs](https://cert-manager.io/docs/)
If you would like to deploy your own certificate issuer and certificates, please see the [cert-manager docs](https://cert-manager.io/docs/).

The ingress-nginx configuration required to host a certificate and manage incoming traffic from the ingress to the Private AI deployment is included with the chart. For more advanced configuration, please see the [ingress-nginx docs](https://github.com/kubernetes/ingress-nginx)
The haproxy-ingress configuration required to host a certificate and manage incoming traffic from the ingress to the Private AI deployment is included with the chart. For more advanced configuration, please see the [haproxy-ingress docs](https://haproxy-ingress.github.io/docs/getting-started/).

Note: The haproxy-ingress and cert-manager helm charts are not listed as dependencies in this chart, and must be installed separately prior to installation.

```console
# Create a namespaces in your cluster for the cert-manager chart
kubectl create namespace cert-manager

# Upgrade or install cert-manager into the cluster with custom resource definitions
helm upgrade --install \
cert-manager oci://quay.io/jetstack/charts/cert-manager \
--namespace cert-manager \
--set crds.enabled=true \
--set clusterResourceNamespace=private-ai

# Required webhook settings if deploying to AWS with the VPC-CNI plugin
# --set webhook.hostNetwork=true \
# --set webhook.securePort=10255

# Create a namespaces in your cluster for the haproxy-ingress chart
kubectl create namespace haproxy-controller

# Upgrade or install haproxy-ingress into the cluster
helm upgrade --install \
haproxy-kubernetes-ingress oci://ghcr.io/haproxytech/helm-charts/kubernetes-ingress \
--namespace haproxy-controller \
--set controller.service.type=LoadBalancer

# Required setting for AWS, see https://github.com/haproxytech/helm-charts/tree/main/kubernetes-ingress#installing-on-amazon-elastic-kubernetes-service-eks
# --set controller.service.enablePorts.quic=false

# Required setting for Azure, see https://github.com/haproxytech/helm-charts/tree/main/kubernetes-ingress#installing-on-azure-managed-kubernetes-service-aks
# --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz


# Update your values.custom.yaml file with the appropriate values under ingress

# Proceed with installing / upgrading private-ai via helm into the private-ai namespace
helm upgrade --install \
private-ai oci://crprivateaiprod.azurecr.io/helm/private-ai \
--namespace private-ai \
-f values.custom.yaml \
--version 1.6.0
```

### External Secrets Operator
If you would like to store you license file and docker credentials in an external secret store, you can use the External Secrets Operator helm chart. Note: The External Secrets chart is not listed as a dependency in this chart, and must be installed separately prior to installation.
If you would like to store you license file and docker credentials in an external secret store, you can use the External Secrets Operator helm chart. Please see the [external secrets operator docs](https://external-secrets.io/latest/).

Note: The External Secrets chart is not listed as a dependency in this chart, and must be installed separately prior to installation.

```console
# Create a namespace in your cluster for the private-ai deployment
Expand All @@ -82,24 +126,28 @@ helm repo add external-secrets https://charts.external-secrets.io
helm repo update

# Upgrade or install the External Secrets operator into the private-ai namespace
helm -n private-ai upgrade --install external-secrets external-secrets/external-secrets
helm upgrade --install \
external-secrets external-secrets/external-secrets \
--namespace private-ai

# Create two secrets, one for the license file and one for the docker credentials, in your external secret store of choice
# Create two secrets, one for the license file and one for the docker credentials, in your external secret store of choice. You can optionally create a secret for environment variables to configure the Private AI container.
```
#### Example AWS secret for Private AI license file
![license-type](./images/license-type.png)
![license-name](./images/license-name.png)
#### Example AWS secret for Private AI docker credentials
![docker-type](./images/docker-type.png)
![docker-name](./images/docker-name.png)
#### Example AWS secret for Private AI environment variables

#### AWS Secrets Manager Steps
Example AWS secret for Private AI license file
![license-type](./images/aws-license-type.png)
![license-name](./images/aws-license-name.png)
Example AWS secret for Private AI docker credentials
![docker-type](./images/aws-docker-type.png)
![docker-name](./images/aws-docker-name.png)
Example AWS secret for Private AI environment variables
This is optional, and can be enabled or disabled in the values file.
![env-type](./images/env-type.png)
![env-name](./images/env-name.png)
![env-type](./images/aws-env-type.png)
![env-name](./images/aws-env-name.png)

Next, configure the AWS Secret Store. See [the AWS Secrets Manager docs](https://external-secrets.io/latest/provider/aws-secrets-manager/) for detailed instructions.
```console
# Create a secret-store within the private-ai namespace
# See https://external-secrets.io/latest/api/secretstore/ for detailed instructions
# Example AWS secret store based on access key:
kubectl create secret -n private-ai generic awssm-secret --from-file=./access-key --from-file=./secret-access-key
kubectl apply -f aws-secret-store.yaml
Expand Down Expand Up @@ -127,18 +175,105 @@ spec:
name: awssm-secret
key: secret-access-key
```
#### Azure Key Vault Steps
Note: Azure does not allow for multiple key-value pairs per secret entry. To match the data structure of the External Secrets Operator, each secret must be uploaded as a JSON object, with the key matching the property from the values file.

Example Azure secret for Private AI license file
![license-name](./images/azure-license-name.png)
```json
{
"license.json": { "id": 1, "tier": "..." }
}
```
Example Azure secret for Private AI docker credentials
![docker-name](./images/azure-docker-name.png)
```json
{
"server": "crprivateaiprod.azurecr.io",
"username": "docker-username",
"password": "docker-password"
}
```
Example Azure secret for Private AI environment variables
This is optional, and can be enabled or disabled in the values file.
![env-name](./images/azure-env-name.png)
```json
{
"PAI_AZ_COMPUTER_VISION_KEY": "secretvalue"
}
```

Next, configure the Azure Secret Store. See the [Azure Key Vault docs](https://external-secrets.io/latest/provider/azure-key-vault/) for more detailed instructions.

You must configure access from the Secret Store object in kubernetes to the Azure Key Vault. In this example, we are using a workload identity federated with an OIDC provider in the kubernetes cluster, and a simple access policy on the key vault. See the [Azure Workload Identity docs](https://azure.github.io/azure-workload-identity/docs/quick-start.html) for more options.

```console
# Set up your AKS cluster and resource group variables
export AKS_CLUSTER_NAME="yourcluster"
export RESOURCE_GROUP="yourresourcegroup"
export USER_ASSIGNED_IDENTITY_NAME="youridentityname"

# Get the Key Vault URL
export KEY_VAULT_URL="$(az keyvault show --name $AKS_CLUSTER_NAME --query 'properties.vaultUri' -o tsv)"

# Create a user-assigned managed identity if using user-assigned managed identity for this tutorial
az identity create --name $USER_ASSIGNED_IDENTITY_NAME --resource-group $RESOURCE_GROUP

# Create key vault policy for UAI
az keyvault set-policy --name $AKS_CLUSTER_NAME \
--secret-permissions get \
--object-id $(az identity show --name $USER_ASSIGNED_IDENTITY_NAME --resource-group $RESOURCE_GROUP --query 'principalId' -o tsv)

# Create kubernetes service account bound to UAI
export USER_ASSIGNED_IDENTITY_CLIENT_ID="$(az identity show --name $USER_ASSIGNED_IDENTITY_NAME --resource-group $RESOURCE_GROUP --query 'clientId' -o tsv)"
export USER_ASSIGNED_IDENTITY_TENANT_ID="$(az identity show --name $USER_ASSIGNED_IDENTITY_NAME --resource-group $RESOURCE_GROUP --query 'tenantId' -o tsv)"
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: ${USER_ASSIGNED_IDENTITY_CLIENT_ID}
azure.workload.identity/tenant-id: ${USER_ASSIGNED_IDENTITY_TENANT_ID}
name: azure-keyvault
namespace: private-ai
EOF

# Create federated credential
az identity federated-credential create \
--name "kubernetes-federated-credential" \
--identity-name $USER_ASSIGNED_IDENTITY_NAME \
--resource-group $RESOURCE_GROUP \
--issuer $(az aks show --resource-group $RESOURCE_GROUP --name $AKS_CLUSTER_NAME --query "oidcIssuerProfile.issuerUrl" -o tsv) \
--subject "system:serviceaccount:private-ai:azure-keyvault"

# Create the secret store
cat <<EOF | kubectl apply -f -
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
name: azure-backend
namespace: private-ai
spec:
provider:
azurekv:
authType: WorkloadIdentity
vaultUrl: ${KEY_VAULT_URL}
serviceAccountRef:
name: azure-keyvault
EOF
```

Update your values.custom.yaml file to enable the external secrets operator, and disable the default secret creation. Ensure to update the docker credentials and license remoteRefKey and properties as per the secret names and properties, respectively.

```console
# Update your values.custom.yaml file to enable the external secrets operator, and disable the default secret creation
# Ensure to update the docker credentials and license remoteRefKey and properties as per the secret names and properties, respectively
externalsecrets:
enabled: true
...

# Proceed with installing / upgrading private-ai via helm into the private-ai namespace
helm upgrade --install \
--namespace private-ai \
private-ai \
-f values.custom.yaml \
oci://crprivateaiprod.azurecr.io/helm/private-ai \
--version 1.5.0
private-ai oci://crprivateaiprod.azurecr.io/helm/private-ai \
--namespace private-ai \
-f values.custom.yaml \
--version 1.6.0
```
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
Binary file added images/azure-docker-name.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/azure-env-name.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/azure-license-name.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 5 additions & 4 deletions templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
Successfully installed {{ .Release.Name }} version {{ .Chart.AppVersion }}
{{- if .Values.ingress.enabled }}
To test the ingress connectivity, run the following
curl {{ if .Values.ingress.selfsigned }}-k {{ end }}https://{{ .Values.ingress.host }}/
{{- else if eq "LoadBalancer" .Values.service.type }}

NOTE: It can take a few minutes to download and start the container.
To get the status of the container, run the following
kubectl --namespace {{ .Release.Namespace }} get pods

{{- if .Values.ingress.enabled }}
To test the ingress connectivity, run the following
curl {{ if .Values.ingress.selfsigned }}-k {{ end }}https://{{ .Values.ingress.host }}/
{{- else if eq "LoadBalancer" .Values.service.type }}

NOTE: It may take a few minutes for the LoadBalancer IP to be available.
To get the status of the load balancer, run the following
kubectl --namespace {{ .Release.Namespace }} get service {{ include "private-ai.fullname" . }}
Expand Down
6 changes: 0 additions & 6 deletions templates/cert-manager/cert-manager-namespace.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion templates/cert-manager/self-signed-certificate.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if and .Values.certmanager.enabled .Values.ingress.selfsigned }}
{{- if and .Values.ingress.enabled .Values.ingress.selfsigned }}
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
Expand Down
2 changes: 1 addition & 1 deletion templates/cert-manager/self-signed-cluster-issuer.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if and .Values.certmanager.enabled .Values.ingress.selfsigned }}
{{- if and .Values.ingress.enabled .Values.ingress.selfsigned }}
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
kind: SecretStore
refreshInterval: {{ .Values.externalsecrets.secrets.refreshInterval }}
target:
name: {{ .Values.secrets.name }}
name: {{ .Values.externalsecrets.secrets.name }}
dataFrom:
- extract:
key: {{ .Values.externalsecrets.secrets.remoteRefKey }}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,5 @@ spec:
- secretKey: license.json
remoteRef:
key: {{ .Values.externalsecrets.license.remoteRefKey }}
property: license.json
property: {{ .Values.externalsecrets.license.licenseProperty }}
{{- end }}
Loading