Add example of AKS attestation and secret provisioning

examples/aks-attestation/README.md

@@ -0,0 +1,150 @@
+# Gramine Attestation Inside AKS cluster
+
+This guide demonstrates how Gramine DCAP attestation quote can be generated and verified from
+within an AKS cluster. Here, we provide an end-to-end example to help Cloud Solution Providers
+integrate Gramine’s RA-TLS attestation and secret provisioning feature with a confidential compute
+cluster managed by Azure Kubernetes Service. The necessary reference wrappers that will enable
+Gramine to use AKS components such as the AESMD and quote provider libraries are contributed.
+A microservice deployment is also provided for the RA-TLS verifier module that can be readily
+deployed to the AKS cluster.
+
+## Preparing client and server images
+
+This demonstration is created for ``gramine/CI-Examples/ra-tls-secret-prov`` sample.
+In order to create the below two images, user needs to download core [Gramine repository](https://github.com/gramineproject/gramine).
+
+### Creating server image
+
+1. Prepare server certificate:
+    - Create server certificate signed by your trusted root CA. Ensure "Common Name"
+      field in the server certificate corresponds to `<AKS-DNS-NAME>` used in step 5.
+    - Put trusted root CA certificate, server certificate, and server key in
+      `gramine/CI-Examples/ra-tls-secret-prov/certs` directory with existing naming convention.
+
+2. Make sure Gramine is built with `meson setup ... -Ddcap=enabled`.
+
+3. Create base ra-tls-secret-prov server image:
+
+    ```sh
+    $ cd gramine/CI-Examples/ra-tls-secret-prov
+    $ make clean && make dcap
+    $ cd gramine
+    $ docker build -t <aks-secret-prov-server-img> \
+        -f <path-to-gsc>/examples/aks-attestation/aks-secret-prov-server.dockerfile .
+    ```
+
+4. Push resulting image to Docker Hub or your preferred registry:
+
+    ```sh
+    $ docker tag <aks-secret-prov-server-img> \
+        <dockerhubusername>/<aks-secret-prov-server-img>
+    $ docker push <dockerhubusername>/<aks-secret-prov-server-img>
+    ```
+
+5. Deploy `<aks-secret-prov-server-img>` in AKS confidential compute cluster:
+    - Reference deployment file:
+        `gsc/examples/aks-attestation/aks-secret-prov-server-deployment.yaml`
+
+**NOTE**: Server can be deployed at a non-confidential compute node as well. However, in that case
+       QVE-based dcap verification will fail.
+
+### Creating client image
+
+1. Make sure Gramine is built with `meson setup ... -Ddcap=enabled`.
+
+2. Create base ra-tls-secret-prov min client image:
+
+    ```sh
+    $ cd gramine/CI-Examples/ra-tls-secret-prov
+    $ make clean && make secret_prov_min_client
+    $ cd gramine
+    $ docker build -t <base-secret-prov-client-img> \
+        -f <path-to-gsc>/examples/aks-attestation/aks-secret-prov-client.dockerfile .
+    ```
+
+3. Prepare client to connect with remote ra-tls-secret-prov server hosted inside AKS cluster:
+    - Provide server dns name `<AKS-DNS-NAME>` as `loader.env.SECRET_PROVISION_SERVERS` value
+      inside `gsc/examples/aks-attestation/aks-secret-prov-client.manifest` file.
+
+4. Create GSC image for ra-tls-secret-prov min client:
+
+    ```sh
+    $ cd gsc
+    $ openssl genrsa -3 -out enclave-key.pem 3072
+    $ ./gsc build <base-secret-prov-client-img> \
+        examples/aks-attestation/aks-secret-prov-client.manifest
+    $ ./gsc sign-image <base-secret-prov-client-img> enclave-key.pem
+    ```
+
+5. Push resulting image to Docker Hub or your preferred registry:
+
+    ```sh
+    $ docker tag <gsc-base-secret-prov-client-img> \
+        <dockerhubusername>/<aks-gsc-secret-prov-client-img>
+    $ docker push <dockerhubusername>/<aks-gsc-secret-prov-client-img>
+    ```
+
+6. Deploy `<aks-gsc-secret-prov-client-img>` in AKS confidential compute cluster:
+    - Reference deployment file:
+        `gsc/examples/aks-attestation/aks-secret-prov-client-deployment.yaml`
+
+**NOTE**: We recommend deploying GSC images on Ubuntu with Linux kernel version 5.11 or higher.
+
+## Deploying client and server images inside AKS Confidential Compute cluster
+
+AKS confidential compute cluster can be created using following
+[link](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-aks-get-started).
+
+Gramine performs out-of-proc mode DCAP quote generation. Out-of-proc mode quote generation requires aesmd
+service. To fulfill this requirement, AKS provides
+[sgxquotehelper daemonset](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-out-of-proc-attestation).
+This feature exposes aesmd service for the container node. The service will internally connect with
+az-dcap-client to fetch the platform collateral required for quote generation. In this demo, the
+``aks-secret-prov-client-deployment.yaml`` uses aesmd service exposed by AKS with the help of
+sgxquotehelper plugin.
+
+In the ra-tls-secret-prov example, the client will generate out-of-proc mode sgx quote that will be
+embedded inside RA-TLS certificate. On receiving the quote, the server will internally verify it
+using libsgx-dcap-quote-verify library via az-dcap-client library. Here,
+``aks-secret-prov-server-deployment.yaml`` will deploy a ra-tls-secret-prov server container inside
+ AKS cluster.
+
+### Deployment
+
+```sh
+$ kubectl apply -f aks-secret-prov-server-deployment.yaml
+```
+
+Once the server container is in running state, start the client container as shown below:
+
+```sh
+$ kubectl apply -f aks-secret-prov-client-deployment.yaml
+```
+
+At this stage, a successful RA-TLS verification would be completed, and the secrets have been
+provisioned from the server to the client container.
+
+## Checking SGX quote generation and verification
+
+Verify the client job is completed:
+
+```sh
+$ kubectl get pods
+```
+
+Receive logs to verify the secret has been provisioned to the client:
+
+```sh
+$ kubectl logs -l app=gsc-ra-tls-secret-prov-client --tail=50
+```
+
+### Expected Output
+
+`--- Received secret = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'`
+
+Delete both client and server containers
+
+```sh
+$ kubectl delete -f aks-secret-prov-server-deployment.yaml
+$ kubectl delete -f aks-secret-prov-client-deployment.yaml
+```

examples/aks-attestation/aks-secret-prov-client-deployment.yaml

@@ -0,0 +1,26 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+   name: secret-prov-client
+   labels:
+      app: secret-prov-client
+spec:
+   template:
+    metadata:
+       labels:
+           app: gsc-ra-tls-secret-prov-client
+    spec:
+      volumes:
+      - name: var-run-aesmd
+        hostPath:
+          path: /var/run/aesmd
+      containers:
+      - name: gsc-ra-tls-secret-prov-client-container
+        image: <dockerhubusername>/<aks-gsc-secret-prov-client-img>
+        resources:
+          limits:
+            kubernetes.azure.com/sgx_epc_mem_in_MiB: 10
+        volumeMounts:
+        - name: var-run-aesmd
+          mountPath: /var/run/aesmd
+      restartPolicy: Never

examples/aks-attestation/aks-secret-prov-client.dockerfile

@@ -0,0 +1,30 @@
+FROM ubuntu:18.04
+
+RUN apt-get update \
+    && env DEBIAN_FRONTEND=noninteractive apt-get install -y wget \
+    build-essential \
+    gnupg2 \
+    libcurl3-gnutls \
+    python3
+
+# Installing DCAP libraries
+
+RUN echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' \
+    > /etc/apt/sources.list.d/intel-sgx.list \
+    && wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key \
+    && apt-key add intel-sgx-deb.key
+
+RUN apt-get update \
+    && apt-get install -y libsgx-urts \
+    libsgx-dcap-ql \
+    libsgx-quote-ex
+
+# Build environment of this Dockerfile should point to the root of Gramine directory
+
+RUN mkdir -p /ra-tls-secret-prov
+
+COPY CI-Examples/ra-tls-secret-prov /ra-tls-secret-prov
+
+WORKDIR /ra-tls-secret-prov
+
+ENTRYPOINT ["/ra-tls-secret-prov/secret_prov_min_client"]

examples/aks-attestation/aks-secret-prov-client.manifest

@@ -0,0 +1,15 @@
+# Manifest file for ra-tls-secret-prov min client
+
+# Secret Provisioning library (client-side) is preloaded
+loader.env.LD_PRELOAD = "libsecret_prov_attest.so"
+
+loader.env.SECRET_PROVISION_SERVERS = "<AKS-DNS-NAME>:4433"
+loader.env.SECRET_PROVISION_CONSTRUCTOR = "1"
+loader.env.SECRET_PROVISION_CA_CHAIN_PATH = "certs/test-ca-sha256.crt"
+
+# Request remote attestation functionality from Gramine
+sgx.remote_attestation = true
+
+sgx.allowed_files.etchostname = "file:/etc/hostname"
+sgx.allowed_files.hosts = "file:/etc/hosts"
+sgx.allowed_files.resolv = "file:/etc/resolv.conf"

examples/aks-attestation/aks-secret-prov-server-deployment.yaml

@@ -0,0 +1,35 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ra-tls-secret-prov-server
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: ra-tls-secret-prov-server
+  template:
+    metadata:
+      labels:
+        app: ra-tls-secret-prov-server
+    spec:
+      containers:
+      - name: ra-tls-secret-prov-server-container
+        image: <dockerhubusername>/<aks-secret-prov-server-img>
+        ports:
+        - containerPort: 4433
+        resources:
+          limits:
+            kubernetes.azure.com/sgx_epc_mem_in_MiB: 10
+---
+apiVersion: v1
+kind: Service
+metadata:
+  annotations:
+    service.beta.kubernetes.io/azure-dns-label-name: <AKS-DNS-NAME>
+  name: ra-tls-secret-prov-server
+spec:
+  type: LoadBalancer
+  ports:
+  - port: 4433
+  selector:
+    app: ra-tls-secret-prov-server

examples/aks-attestation/aks-secret-prov-server.dockerfile

@@ -0,0 +1,36 @@
+FROM ubuntu:18.04
+
+RUN apt-get update \
+    && env DEBIAN_FRONTEND=noninteractive apt-get install -y \
+    build-essential \
+    gnupg2 \
+    libcurl3-gnutls \
+    libcurl4-openssl-dev \
+    python3 \
+    wget
+
+# Installing Azure DCAP Quote Provider Library (az-dcap-client).
+# Here, the version of az-dcap-client should be in sync with the az-dcap-client
+# version used for quote generation. User can replace the below package with the
+# latest package.
+
+RUN wget https://github.com/microsoft/Azure-DCAP-Client/releases/download/1.8/az-dcap-client_1.8_amd64_18.04.deb \
+    && dpkg -i az-dcap-client_1.8_amd64_18.04.deb
+
+# Installing DCAP Quote Verification Library
+RUN echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' \
+    > /etc/apt/sources.list.d/intel-sgx.list \
+    && wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key \
+    && apt-key add intel-sgx-deb.key
+
+RUN apt-get update && apt-get install -y libsgx-dcap-quote-verify
+
+# Build environment of this Dockerfile should point to the root of Gramine directory
+
+RUN mkdir -p /ra-tls-secret-prov
+
+COPY CI-Examples/ra-tls-secret-prov /ra-tls-secret-prov
+
+WORKDIR /ra-tls-secret-prov
+
+ENTRYPOINT ["/ra-tls-secret-prov/secret_prov_server_dcap"]