diff --git a/authN-authZ/auth-apisix/README.md b/authN-authZ/auth-apisix/README.md index bf46a0db9..2296f1dbd 100644 --- a/authN-authZ/auth-apisix/README.md +++ b/authN-authZ/auth-apisix/README.md @@ -2,16 +2,19 @@ Follow the steps to enable authentication and authorization of OPEA services using APISIX api gateway and Identity provider like keycloak -## Prerequisites +## Start ChatQnA service -1. Make sure ChatQnA service is running and accessible locally +Please refer to [GenAIExamples ChatQnA](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/kubernetes) to start `chatqna` megaservice. -2. Run keycloak, setup a realm with OIDC based authentication and add users with passwords for each user. +## Starting and Configuring Keycloak -Steps to start keycloak from official keycloak helm chart +In this stage we run keycloak and setup a realm with OIDC based authentication. For demonstration, we will add a realm called `apisix` and add a user called `mary` with password. In the authentication step, only the user from `apisix` realm can access the chatQnA pipeline. + +### Step 1: Prerequisite. + +Create a PersistentVolume of 9Gi for keycloak-postgress with RWO access (to persist updated keycloak configuration) ```sh -# Prerequisite: Create a PersistentVolume of 9Gi for keycloak-postgress with RWO access (to persist updated keycloak configuration) # Below is a reference to create PersistentVolume which will be used by keycloak-postgress kubectl apply -f - < +export KEYCLOAK_REALM='apisix' # change if needed +export KEYCLOAK_CLIENT_ID='apisix' # change if needed +export KEYCLOAK_CLIENT_SECRET= ``` -Once the keycloak pod is up and running, access the UI through the Keycloak's NodePort service and set the necessary configuration +**Trouble Shooting: https required** + +If you meet "https required" issue when you open the console, you can fix with the following steps: + +```bash +kubectl exec -it ${keycloak_pods_id} -- /bin/bash +cd /opt/keycloak/bin/ +./kcadm.sh config credentials --server ${KEYCLOAK_ADDR} --realm master --user admin ## need to type in password set before +./kcadm.sh update realms/master -s sslRequired=NONE --server ${KEYCLOAK_ADDR} +``` + +Then go to the Keycloak console and find the "Realm setting" for `apisix` realm, set "Require SSL" to "None". ## Update values -Update the following values in values.yaml +Update the following values in values.yaml, you can take values_megaservice.yaml as a reference. 1. Update all the entries in oidc config pertaining to your identity provider 2. Update all the entries in API specific configs -## Install +## Install APISIX ```sh -# Install apisix api gateway and ingress controller [reference: https://apisix.apache.org/docs/apisix/installation-guide/] +# Install APISIX api gateway and ingress controller [reference: https://apisix.apache.org/docs/apisix/installation-guide/] helm repo add apisix https://charts.apiseven.com helm repo update helm install auth-apisix apisix/apisix -f values_apisix_gw.yaml --create-namespace --namespace auth-apisix -# WAIT UNTIL apisix-ingress-controller POD IS READY by checking status with 'kubectl get -n auth-apisix pods' +# WAIT UNTIL the apisix-ingress-controller POD IS READY by checking it's status with 'kubectl get -n auth-apisix pods' # The pod is ready when READY status shows 1/1 # Publish authenticated APIs in APISIX gateway helm install auth-apisix-crds . --namespace auth-apisix ``` +
+APISIX helm chart provides configs to change the service type to other options like LoadBalancer (apisix.service.type) and externalTrafficPolicy to 'local'(apisix.service.externalTrafficPolicy). These can be added in values_apisix_gw.yaml

+ ## Usage -The published APIs in apisix gateway are accessible through auth-apisix-gateway kubernetes service. By default, it is a NodePort service and can be accessed as: +The published APIs in APISIX gateway are accessible through auth-apisix-gateway kubernetes service. By default, it is a NodePort service and can be accessed as: ```sh export NODE_PORT=$(kubectl get --namespace auth-apisix -o jsonpath="{.spec.ports[0].nodePort}" services auth-apisix-gateway) export NODE_IP=$(kubectl get nodes --namespace auth-apisix -o jsonpath="{.items[0].status.addresses[0].address}") # the authenticated endpoint published in APISIX gateway can be accessed as: http://$NODE_IP:$NODE_PORT/ -export accessUrl=http://$NODE_IP:$NODE_PORT/ - - +export accessUrl="http://$NODE_IP:$NODE_PORT/chatqna-oidc" ``` -
-Apisix helm chart provides configs to change the service type to other options like LoadBalancer (apisix.service.type) and externalTrafficPolicy to 'local'(apisix.service.externalTrafficPolicy). These can be added in values_apisix_gw.yaml

While accessing the published APIs, the HTTP Authorization header of the request should contain the Access token provided by Identity provider as 'Bearer <Access Token>'.

The access token, refresh token, userinfo, user roles and OIDC scopes assigned to user can be obtained by invoking OIDC auth endpoint through UI or token endpoint through curl and providing user credentials.

Below steps can be followed to get access token from keycloak and access the APISIX published ChatQnA API through curl ```sh -# Get access token for specified user from keycloak -export USER= -export PASSWORD= -export KEYCLOAK_ADDR= -export KEYCLOAK_REALM= -export KEYCLOAK_CLIENT_ID= -export KEYCLOAK_CLIENT_SECRET= - -#Invoke Keycloak's OIDC token endpoint to get access token, refresh token and expirt times. (Only Access token is used in the example below) +#Invoke Keycloak's OIDC token endpoint to get access token, refresh token and expiry times. (Only Access token is used in the example below) export TOKEN=$(curl -X POST http://${KEYCLOAK_ADDR}/realms/${KEYCLOAK_REALM}/protocol/openid-connect/token -H 'Content-Type: application/x-www-form-urlencoded' -d "grant_type=password&client_id=${KEYCLOAK_CLIENT_ID}&client_secret=${KEYCLOAK_CLIENT_SECRET}&username=${USER}&password=${PASSWORD}" | jq -r .access_token) -# follow instructions above to fetch the NODE_IP and NODE_PORT -export accessUrl="http://$NODE_IP:$NODE_PORT/chatqna-oidc" - # try without token. Shall get response: "Authorization required 401 error" -curl -X POST $accessUrl -d '{"text":"What is the revenue of Nike in 2023?","parameters":{"max_new_tokens":17, "do_sample": true}}' -sS -H 'Content-Type: application/json' -w " %{http_code}\n" +curl -X POST $accessUrl -d '{"messages": "What is the revenue of Nike in 2023?", "max_new_tokens":17, "do_sample": true}' -sS -H 'Content-Type: application/json' -w " %{http_code}\n" # try with token. Shall get the correct response from ChatQnA with http code 200 -curl -X POST $accessUrl -d '{"text":"What is the revenue of Nike in 2023?","parameters":{"max_new_tokens":17, "do_sample": true}}' -sS -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -w " %{http_code}\n" +curl -X POST $accessUrl -d '{"messages": "What is the revenue of Nike in 2023?", "max_new_tokens":17, "do_sample": true}' -sS -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -w " %{http_code}\n" ``` ## Uninstall ```sh -# Uninstall apisix +# Uninstall APISIX helm uninstall auth-apisix-crds --namespace auth-apisix helm uninstall auth-apisix --namespace auth-apisix ``` -The crds installed by apisix won't be deleted by helm uninstall. Need to manually delete those crds
+The crds installed by APISIX won't be deleted by helm uninstall. Need to manually delete those crds
All APISIX specific crds can be obtained by 'kubectl get crds | grep apisix'
Each crd can be manually deleted by 'kubectl delete crd/\'
diff --git a/authN-authZ/auth-apisix/templates/apisix-chatqna-route.yaml b/authN-authZ/auth-apisix/templates/apisix-chatqna-route.yaml index 61658f880..656baf077 100644 --- a/authN-authZ/auth-apisix/templates/apisix-chatqna-route.yaml +++ b/authN-authZ/auth-apisix/templates/apisix-chatqna-route.yaml @@ -28,7 +28,7 @@ spec: introspection_endpoint_auth_method: client_secret_basic scope: openid profile email bearer_only: true - realm: myrealm + realm: {{ .Values.oidc.realm }} - name: proxy-rewrite enable: true config: diff --git a/authN-authZ/auth-apisix/values_megaservice.yaml b/authN-authZ/auth-apisix/values_megaservice.yaml new file mode 100644 index 000000000..e66b6ff07 --- /dev/null +++ b/authN-authZ/auth-apisix/values_megaservice.yaml @@ -0,0 +1,20 @@ +# Copyright (C) 2024 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +# Identity provider OIDC config +oidc: + realm: ${KEYCLOAK_REALM} # replace with your realm name for OPEA apps + client_id: ${KEYCLOAK_CLIENT_ID} # replace with your oidc client id + client_secret: ${KEYCLOAK_CLIENT_SECRET} # your oidc client secret + discovery: http://${KEYCLOAK_ADDR}/realms/${KEYCLOAK_CLIENT_ID}/.well-known/openid-configuration + introspection_endpoint: http://${KEYCLOAK_ADDR}/realms/${KEYCLOAK_CLIENT_ID}/protocol/openid-connect/token/introspect + +# APISIX chatqna api config +chatqna: + namespace: default # namespace in which your chatqna service is running + hostname: your-hostname # 'Host' HTTP header from incoming request should match this. Wildcards like '*' allowed too + query_api: + path: /chatqna-oidc # This is the path that will be published in apisix and this should be used by UI to access the chatqna service + backend_service: chatqna # your kubernetes service name to access chatqna megaservice or gmc without ..svc.cluster.local + service_port: 8888 # port on which chatqna mega service or gmc is running + service_path: "/v1/chatqna" # path to access chatqna mega service or gmc backend diff --git a/authN-authZ/auth-istio/README.md b/authN-authZ/auth-istio/README.md index 96220ae0e..e91c63a7c 100644 --- a/authN-authZ/auth-istio/README.md +++ b/authN-authZ/auth-istio/README.md @@ -12,23 +12,46 @@ Here we use the chatQnA pipeline as an example. ## Prerequisite -Before composing an OPEA pipeline with authN & authZ, user need to install Istio to support this feature. Please follow the steps [here](https://istio.io/latest/docs/setup/install/istioctl/) for Istio installation. +**Istio installation** + +Before composing an OPEA pipeline with authN & authZ, user need to install Istio to support this feature. + +```bash +curl -L https://istio.io/downloadIstio | sh - +cd istio-1.23.2 ## change to your downloaded version +export PATH=$PWD/bin:$PATH +istioctl install +``` + +You can refer to [Istio installation](https://istio.io/latest/docs/setup/install/istioctl/) for detailed information. **Deploy chatQnA pipeline and enable Istio sidecar injection** -```sh +(Optional) Create a new namespace and you can enable istio sidecar injection for the namespace directly. -# deploy ChatQnA pipeline. You can either leverage GMC or the ChatQnA helm chart. +```sh kubectl create ns chatqa -# here's the command to leverage GMC custom resource for ChatQnA deployment. +kubectl label namespace chatqa istio-injection=enabled +``` + +Deploy ChatQnA pipeline. You can either leverage GMC, manifest or helm chart. + +```bash +# Option 1: GMC +# leverage GMC custom resource for ChatQnA deployment. kubectl apply -f $(pwd)/../../microservices-connector/config/samples/ChatQnA/chatQnA_dataprep_xeon.yaml -# please refer the doc https://github.com/opea-project/GenAIInfra/tree/main/helm-charts/chatqna for deployment with helm chart. -# and install under `chatqa` namespace + +# Option 2: helm chart +# please refer the doc https://github.com/opea-project/GenAIInfra/tree/main/helm-charts/chatqna for deployment with helm chart and install under `chatqa` namespace + +# Option 3: manifest +# Please refer to https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/kubernetes/intel for deployment with manifest and install under `chatqa` namespace # expose an environment variable to set the deployment method -# which will affect the configuration we use for authentication and authorization +# which will affect the configuration we use for authentication and authorization. If you use manifest, can also set to "helm-chart-based" export DEPLOY_METHOD= +# If you didn't enable istio sidecar injection for "chatqna" namespace, here we # patch the deployment to enable istio sidecar injection # for GMC based deployment, set the `deployment-name` to `router-service-deployment` # for helm chart based deployment, set the `deployment-name` to `chatqna` @@ -128,7 +151,7 @@ In this sample, we are going to test with the scenario that only privileged user Install Keycloak in the kubernetes cluster for user management. **Note:** Replace the admin password as your own in the command. ```bash -# make sure running under authN-authZ/auth-istio folder +# make sure running under authN-authZ folder kubectl apply -f $(pwd)/keycloak_install.yaml # get the ip and port to access keycloak. @@ -151,18 +174,63 @@ The user management is done via Keycloak and the configuration steps look like t 4. Create a new user name as `mary` and another user as `bob`. Set passwords for both users (set 'Temporary' to 'Off'). Select Role mapping on the top, assign the `user` role to `mary` and assign the `viewer` role to `bob`. -**Apply authentication and authorization policies to the pipeline endpoint based on OIDC provider** +5. Turn off all 'Required actions' buttons under the 'Authentication' section in Keycloak -Use the commands to apply the authentication and authorization rules. +**Trouble Shooting: https required** + +If you meet "https required" issue when you open the console, you can fix with the following steps: + +```bash +kubectl exec -it ${keycloak_pods_id} -- /bin/bash +cd /opt/keycloak/bin/ +./kcadm.sh config credentials --server ${KEYCLOAK_ADDR} --realm master --user admin ## need to type in password set before +./kcadm.sh update realms/master -s sslRequired=NONE --server ${KEYCLOAK_ADDR} +``` + +Then go to the Keycloak console and find the "Realm setting" for `apisix` realm, set "Require SSL" to "None". + +**Export the router service through istio ingress gateway** + +For authentication safegard, we should add a gateway for the service. Here we show how to set up the istio ingress gateway to control access to the chatQnA service. + +First export the router service through istio ingress gateway. ```bash -# export the router service through istio ingress gateway kubectl apply -f $(pwd)/$DEPLOY_METHOD/chatQnA_router_gateway.yaml +``` -# 'envsubst' is used to substitute envs in yaml. +Determine the ingress IP and ports and expose them as environment variables. + +```bash +export INGRESS_NAME=istio-ingressgateway +export INGRESS_NS=istio-system + +#run the following to determine if your Kubernetes cluster is in an environment that supports external load balancers: +kubectl get svc "$INGRESS_NAME" -n "$INGRESS_NS" + +# Case1: If your EXTERNAL-IP value is (or perpetually ), your environment does not provide an external load balancer for the ingress gateway. +# set the INGRESS_HOST to your host ip, +export INGRESS_HOST=${host_ip} +# set the INGRESS_PORT to the istio-ingressgateway svc port +export INGRESS_PORT=${gateway_svc_port} + +#Case2: If your environment supports external load balancers +export INGRESS_HOST=$(kubectl -n "$INGRESS_NS" get service "$INGRESS_NAME" -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +export INGRESS_PORT=$(kubectl -n "$INGRESS_NS" get service "$INGRESS_NAME" -o jsonpath='{.spec.ports[?(@.name=="http2")].port}') +``` + +You can refer to the [istio ingress gateway guide](https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/#determining-the-ingress-ip-and-ports) for more details about ingress gateway ip and ports. + +**Apply authentication and authorization policies to the pipeline endpoint based on OIDC provider** + +Use the commands to apply the authentication and authorization rules. + +```bash +# 'envsubst' is used to substitute envs in yaml if you launch with kubectl commands. # use 'sudo apt-get install gettext-base' to install envsubst if it does not exist on your machine # apply the authentication and authorization rule # these files will restrict user access with valid token (with valid issuer, username and realm role) +export REALM=istio envsubst < $(pwd)/$DEPLOY_METHOD/chatQnA_authN_keycloak.yaml | kubectl -n chatqa apply -f - envsubst < $(pwd)/$DEPLOY_METHOD/chatQnA_authZ_keycloak.yaml | kubectl -n chatqa apply -f - ``` @@ -181,7 +249,7 @@ export TOKENA=$(curl -X POST 'http://${KEYCLOAK_ADDR}/realms/istio/protocol/open export TOKENB=$(curl -X POST 'http://${KEYCLOAK_ADDR}/realms/istio/protocol/openid-connect/token' -H 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'grant_type=password' --data-urlencode 'client_id=istio' --data-urlencode 'username=bob' -d 'password=${PASSWORD}' -d 'scope=openid' -d 'response_type=id_token' | jq -r .access_token) ``` -Access the istio ingress gateway to reach the chatQnA service with different tokens. Follow the istio guide [here](https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/#determining-the-ingress-ip-and-ports) to determine the ingress IP and ports. +Access the istio ingress gateway to reach the chatQnA service with different tokens. ```bash # follow the guide above to fetch the $INGRESS_HOST and $INGRESS_PORT @@ -210,7 +278,7 @@ We are using a similar scenario here that only privileged users can access our c Here we take Keycloak as the sample OIDC Provider to use in the example. Follow the steps to install and configure Keycloak. ```bash -# make sure running under authN-authZ/auth-istio folder +# make sure running under authN-authZ folder kubectl apply -f $(pwd)/keycloak_install.yaml # get the ip and port to access keycloak. @@ -320,13 +388,27 @@ envsubst < $(pwd)/$DEPLOY_METHOD/chatQnA_authZ_oauth.yaml | kubectl apply -f - **Validate authentication and authorization with UI service** -Add both host names for ChatQnA UI and backend service into /etc/hosts +Add both host names for ChatQnA UI and backend service, edit the /etc/hosts file on the computer from where you are working/browsing to bypass the need for a DNS server to resolve your FQDN (fully qualified domain name). ```bash sudo sed -i '1i\127.0.0.1 chatqna-service.com' /etc/hosts sudo sed -i '1i\127.0.0.1 chatqna-ui.com' /etc/hosts ``` +By setting your routing for FQDN based services you can restrict access to only services you want to expose in your Kubernetes cluster. + Open browser with address `"chatqna-ui.com:${INGRESS_PORT}"` if using GMC based deployment. Otherwise, open the browser with address `"chatqna-service.com:${INGRESS_PORT}"`. Login with user `bob` and its credentials shall return a 403 error. Login with user `mary` and its credentials shall able to access the ChatQnA service. + +## Uninstall Istio + +After testing, you can uninstall Istio with following commands: + +```bash +istioctl uninstall --purge -y +kubectl delete namespace istio-system +kubectl label namespace chatqa istio-injection- +``` + +Can refer to [Istio uninstal](https://istio.io/latest/docs/setup/getting-started/#uninstall) for more information. diff --git a/authN-authZ/auth-istio/keycloak_install.yaml b/authN-authZ/keycloak_install.yaml similarity index 100% rename from authN-authZ/auth-istio/keycloak_install.yaml rename to authN-authZ/keycloak_install.yaml