Postgres visibility tutorial (WIP)

docs/quickstart/visualization/postgresql.mdx

@@ -0,0 +1,135 @@
+---
+sidebar_position: 4
+title: PostgreSQL table-level access
+
+[//]: # (image: /img/quick-tutorials/aws-iam-eks/social.png) // TODO
+---
+
+import CodeBlock from "@theme/CodeBlock";
+import Tabs from "@theme/Tabs";
+import TabItem from "@theme/TabItem";
+
+Otterize visualizes table-level access to PostgreSQL databases, by collecting PostgreSQL audit logs.
+
+In this tutorial, we will:
+- Optionally, prepare a Kubernetes cluster and connect it to Otterize Cloud
+- Optionally, deploy a Cloud SQL for PostgreSQL instance
+- Deploy a PostgreSQL client pod that queries the Cloud SQL server
+- Configure the Cloud SQL server to enable database auditing and route the audit logs to a Pub/Sub destination
+- Create an Otterize database integration and configure visibility log collection
+- Create a `ClientIntents` resource allowing access from the PostgreSQL client pod to the Cloud SQL server
+
+:::info
+Visibility log collection is currently available only for PostgreSQL instances running on GCP Cloud SQL.
+:::
+
+## Prerequisites
+
+### 1. Prepare a Kubernetes cluster and connect it to Otterize Cloud
+Already have Otterize deployed with the database integration configured on your cluster? [Skip to the tutorial.](#tutorial)
+
+<details>
+  <summary>Prepare a Kubernetes cluster</summary>
+  {@include: ../../_common/cluster-setup.md}
+</details>
+
+<details>
+  <summary>Install Otterize in your cluster, <b>with</b> Otterize Cloud</summary>
+
+  #### Create an Otterize Cloud account
+
+  {@include: ../../_common/create-account.md}
+
+  #### Install Otterize OSS, connected to Otterize Cloud
+
+  {@include: ../../_common/install-otterize-from-cloud-with-enforcement.md}
+
+</details>
+
+### 2. Prepare a Cloud SQL for PostgreSQL instance
+Already have a Cloud SQL for PostgreSQL instance ready? [Skip to the tutorial.](#tutorial)
+<details>
+  <summary>Deploy a Cloud SQL for PostgreSQL instance</summary>
+
+  Follow the [installation instructions](https://cloud.google.com/sql/docs/postgres/create-instance) on the Google Cloud SQL documentation.
+
+  On the <b>Connections</b> configuration, make sure you check the <b>Public IP</b> option, to create a public IP address for your Cloud SQL instance.
+  On the `Authorized networks` configuration, add `0.0.0.0/0` to allow access from the internet.
+</details>
+
+## Tutorial
+
+### Deploy a PostgreSQL client
+
+Run the following commands, replacing `<PGHOST>`, `<PGUSER>` & `<PGPASSWORD>` with the IP address & credentials for your Cloud SQL for PostgreSQL instance:
+```shell
+kubectl create namespace otterize-tutorial-postgresql-visibility
+kubectl create secret generic psql-credentials -n otterize-tutorial-postgresql-visibility \
+    --from-literal=username="<PGUSER>" \
+    --from-literal=password="<PGPASSWORD>"
+kubectl create configmap psql-host -n otterize-tutorial-postgresql-visibility \
+    --from-literal=pghost="35.232.47.212" \
+    --from-literal=pgport=5432 \
+    --from-literal=pgdatabase=postgres
+kubectl apply -f ${ABSOLUTE_URL}/code-examples/postgresql-visibility/psql-client.yaml -n otterize-tutorial-postgresql-visibility
+```
+This will deploy a Kubernetes CronJob that will run a psql client once every 1 minute,
+connecting to your PostgrSQL server and running a basic query.
+
+### Configure the Cloud SQL server to enable database auditing
+Follow the instructions for configuring [audit for PostgreSQL using pgAudit](https://cloud.google.com/sql/docs/postgres/pg-audit) on GCP docs.
+
+### Route Cloud SQL audit logs to a Pub/Sub destination
+Follow the instructions for routing logs to [third-party integrations with Pub/Sub](https://cloud.google.com/logging/docs/export/pubsub#integrate-thru-pubsub) on GCP docs.
+- For the <b>third-party service account name</b>, use the Otterize Cloud service account:
+  ```shell
+  [email protected]
+  ```
+- Under <b>logs to include</b>, provide the following filter to include all audit logs generated by pgAudit
+  for the Cloud SQL instance you are using for this tutorial:
+  ```shell
+  resource.type="cloudsql_database"
+  protoPayload.request.@type="type.googleapis.com/google.cloud.sql.audit.v1.PgAuditEntry"
+  logName="projects/<GCP_PROJECT_ID>/logs/cloudaudit.googleapis.com%2Fdata_access"
+  resource.labels.database_id="<GCP_PROJECT_ID>:<CLOUDSQL_INSTANCE_NAME>"
+    ```
+
+### Create an Otterize database integration and configure visibility log collection
+- Navigate to the [Integrations page](https://app.otterize.com/integrations) on Otterize Cloud
+and click the `+ Add Integration` button to create a new integration.
+- Choose the <b>Database</b> integration type
+- Name your integration `otterize-tutorial-postgresql-visibility-db`
+- Follow the form to provide Otterize with your database information & credentials
+- Click `Test Connection` to ensure that Otterize Cloud is able to access your database instance
+- Under <b>Visibility settings</b>, choose to collect visibility logs using a <b>GCP Pub/Sub topic</b>
+- Enter your GCP Project ID & Topic name
+- Click `Test Visibility` to ensure that Otterize Cloud is able to subscribe to your Pub/Sub topic
+- Click `Add` to finish setting up your database integration
+
+At this point, Otterize's database integration will start collecting visibility logs from your Pub/Sub topic,
+and view them in your [Access Graph](https://app.staging.otterize.com/access-graph).
+
+If you followed the installation instructions for the psql-client application earlier in this tutorial,
+you should start seeing connections from the psql-client app to your Cloud SQL server after about one minute.
+
+[//]: # (TODO: add a screenshot here with some explanation about what we see - unknown client accesing PSQL server)
+
+### Create a `ClientIntents` resource allowing access from the PostgreSQL client pod to the Cloud SQL server
+
+```
+kubectl delete secret psql-credentials -n otterize-tutorial-postgresql-visibility
+kubectl patch cronjob psql-client -n otterize-tutorial-postgresql-visibility -p \
+  '{"spec": {"jobTemplate": {"spec": {"template": {"metadata": {"annotations": {"credentials-operator.otterize.com/user-password-secret-name": "psql-credentials"}}}}}}}'
+
+kubectl apply -f ${ABSOLUTE_URL}/code-examples/postgresql-visibility/psql-client-clientintents.yaml -n otterize-tutorial-postgresql-visibility
+
+```
+
+
+## Teardown
+
+To remove the deployed examples run:
+
+```bash
+kubectl delete namespace otterize-tutorial-postgresql-visibility
+```

static/code-examples/postgresql-visibility/psql-client-clientintents.yaml

@@ -0,0 +1,14 @@
+apiVersion: k8s.otterize.com/v1alpha3
+kind: ClientIntents
+metadata:
+  name: psql-client
+spec:
+  service:
+    name: psql-client
+  calls:
+    - name: otterize-tutorial-postgresql-visibility-db
+      type: database
+      databaseResources:
+        - databaseName: postgres
+          table:
+          operations:

static/code-examples/postgresql-visibility/psql-client.yaml

@@ -0,0 +1,60 @@
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: psql-client
+spec:
+  schedule: "* * * * *"
+  jobTemplate:
+    spec:
+      template:
+        metadata:
+          # Un-comment to generate postgres credentials using the Otterize credentials operator
+#          annotations:
+#            credentials-operator.otterize.com/user-password-secret-name: psql-credentials
+          labels:
+            app: psql-client
+        spec:
+          containers:
+            - name: psql-client
+              image: andreswebs/postgresql-client
+              imagePullPolicy: IfNotPresent
+              command:
+                - psql
+                - -c
+                - "select count(*),pg_sleep(10) from pg_catalog.pg_tables;"
+              env:
+                - name: PGHOST
+                  valueFrom:
+                    configMapKeyRef:
+                      name: psql-host
+                      key: pghost
+                - name: PGPORT
+                  valueFrom:
+                    configMapKeyRef:
+                      name: psql-host
+                      key: pgport
+                - name: PGDATABASE
+                  valueFrom:
+                    configMapKeyRef:
+                      name: psql-host
+                      key: pgdatabase
+                - name: PGUSER
+                  valueFrom:
+                    secretKeyRef:
+                      name: psql-credentials
+                      key: username
+                - name: PGPASSWORD
+                  valueFrom:
+                    secretKeyRef:
+                      name: psql-credentials
+                      key: password
+          volumes:
+            - name: psql-credentials
+              secret:
+                secretName: psql-credentials
+                items:
+                  - key: username
+                    path: username
+                  - key: password
+                    path: password
+          restartPolicy: Never