Skip to content

Commit

Permalink
MySQL integration tutorial (#240)
Browse files Browse the repository at this point in the history
  • Loading branch information
amitlicht committed May 21, 2024
1 parent 187e635 commit 8503e92
Show file tree
Hide file tree
Showing 18 changed files with 547 additions and 39 deletions.
8 changes: 8 additions & 0 deletions docs/features/mysql/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "MySQL",
"position": 5,
"collapsed": true,
"customProps": {
"image": "/img/icons/mysql.svg"
}
}
111 changes: 111 additions & 0 deletions docs/features/mysql/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
sidebar_position: 1
title: MySQL | Overview
hide_title: true
---

import DocsLinkCard from "@site/src/components/LinkCard";

export const mysql_tutorials = [
{
title: 'Just-in-time MySQL Access',
description: 'Learn how to manage just-in-time users and SQL GRANTs',
url: '/features/mysql/tutorials/mysql'
},
];

# MySQL

Otterize is able to create just-in-time username-and-password pairs for your service, providing them as a Kubernetes Secret that can be mounted to file or mapped to environment variables, as well as `GRANT`ing access to databases and tables, based on `ClientIntents` ([Intent-Based Access Control](/overview/intent-based-access-control)) declarations.

### Tutorials

To learn how to use the Intents Operator and Credentials Operator to enforce access using MySQL GRANTs, try one of these quickstart tutorials:

<DocsLinkCard items={mysql_tutorials} colSize={"sm"}/>



### How does Otterize work with MySQL?

The Otterize credentials operator will create a unique MySQL username-password combination for each service's use, exposed via a Kubernetes Secret. The service will use these credentials to connect to the database. `ClientIntents` will define the access required by that service. As the intents are applied, The Otterize intents operator will keep the database's list of users and GRANTs up to date so that the service is able to access it.

1. To get started, your cluster must have Otterize deployed.
2. You'll need to create a `MySQLServerConfig` in your cluster, providing a connection URL and admin-level credentials for Otterize to manage permissions in your database. Below is an example `MySQLServerConfig` resource.
```yaml
apiVersion: k8s.otterize.com/v1alpha3
kind: MySQLServerConfig
metadata:
name: mysql-tutorial-db # database instance name - should match the target in ClientIntents
spec:
address: <HOST:PORT> # Your MySQL servers address
credentials:
username: <USER> # Username Otterize will connect with & configure permissions as
password: <PASSWORD> # Password for above username
```

3. Each service can request a username-password Secret to be created, by annotating the Pod with `credentials-operator.otterize.com/user-password-secret-name`. Below is an example of that annotation and passing the generated credentials into a container with environmental variables.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: server
spec:
replicas: 1
selector:
matchLabels:
app: server
template:
metadata:
annotations:
# highlight-next-line
credentials-operator.otterize.com/user-password-secret-name: server-creds
labels:
app: server
spec:
serviceAccountName: server
containers:
- name: server
imagePullPolicy: Always
image: 'supercool/my-example-container'
ports:
- containerPort: 80
env:
- name: DB_SERVER_USER
valueFrom:
secretKeyRef:
name: server-creds
key: username
- name: DB_SERVER_PASSWORD
valueFrom:
secretKeyRef:
name: server-creds
key: password
```


4. Apply `ClientIntents` and the specified access will be `GRANT`ed to the service in the `ClientIntents`.


```yaml
apiVersion: k8s.otterize.com/v1alpha3
kind: ClientIntents
metadata:
name: client-intents-for-server
namespace: otterize-tutorial-mysql
spec:
service:
name: server
calls:
- name: mysql-tutorial-db # Same name as MySQLServerConfig metadata.name
type: database
databaseResources:
- databaseName: otterize-tutorial
table: example
operations:
- SELECT
- INSERT
```

5. Done!
45 changes: 45 additions & 0 deletions docs/features/mysql/reference.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
sidebar_position: 3
title: Reference
---

### MySQLServerConfig example (YAML)
```yaml
apiVersion: k8s.otterize.com/v1alpha3
kind: MySQLServerConfig
metadata:
name: otterize-tutorial-mysql # database instance name - should match the target in ClientIntents
spec:
address: <HOST:PORT> # Your MySQL servers address
credentials:
username: <USER> # Username Otterize will connect with & configure permissions as (typically 'admin')
password: <PASSWORD> # Password for above username
```

### ClientIntents example (YAML)

```yaml
apiVersion: k8s.otterize.com/v1alpha3
kind: ClientIntents
metadata:
name: client-intents-for-server
namespace: otterize-tutorial-mysql
spec:
service:
# Service requiring access to MySQL
name: server
calls:
# This name will need to match the MySQLServerConfig metadata.name field
- name: otterize-tutorial-mysql
type: database
databaseResources:
- databaseName: otterize_tutorial
# Optional table name, if omitted all tables will be granted access
table: example
# Operations being granted, options include SELECT, INSERT, UPDATE, DELETE, ALL
operations:
- SELECT
- INSERT
```


5 changes: 5 additions & 0 deletions docs/features/mysql/tutorials/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"label": "Tutorials",
"position": 2,
"collapsed": false
}
207 changes: 207 additions & 0 deletions docs/features/mysql/tutorials/mysql.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
---
sidebar_position: 2
title: Just-in-time MySQL access
image: /img/quick-tutorials/mysql/social.png
---

import CodeBlock from "@theme/CodeBlock";
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

export const Terminal = ({children}) => (
<div
style={{
backgroundColor: '#eee',
borderRadius: '5px',
fontSize: '12px',
fontWeight: '600',
color: 'darkgreen',
padding: '1rem',
fontFamily: 'monospace, monospace'
}}>
{children}
</div>
);


# Overview
This tutorial will deploy an example cluster to highlight Otterize's MySQL capabilities. Within that cluster is a client service that hits an endpoint on a server, which then connects to a database. The server runs two different database operations:
1. An `INSERT` operation to append a table within the database
2. A `SELECT` operation to validate the updates.

The server needs appropriate permissions to access the database. You could use one admin user for all services, which is insecure and is the cause for many security breaches. With Otterize, you can specify required access, and have Otterize create users and perform correctly scoped SQL GRANTs just in time, as the service spins up and down.

In this tutorial, we will:
* Optionally, spin up a MySQL database instance, based on Amazon RDS for MySQL. Alternatively, you could use any MySQL server of your choice.
* Deploy an example cluster
* Deploy Otterize in our cluster and give it access to our database instance
* Declare a ClientIntents resource for the server, specifying required access
* See that the required access has been granted

# Prerequisites

#### 1. Minikube Cluster
<details>
<summary>Prepare a Kubernetes cluster with Minikube</summary>

For this tutorial you'll need a local Kubernetes cluster. Having a cluster with a [CNI](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) that supports [NetworkPolicies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) isn't required for this tutorial, but is recommended so that your cluster works with other tutorials.

If you don't have the Minikube CLI, first [install it](https://minikube.sigs.k8s.io/docs/start/).

Then start your Minikube cluster with Calico, in order to enforce network policies.

```shell
minikube start --cpus=4 --memory 4096 --disk-size 32g --cni=calico
```
</details>

#### 2. Deploy Otterize
To deploy Otterize, head over to [Otterize Cloud](https://app.otterize.com) and associate a Kubernetes cluster on the [Integrations page](https://app.otterize.com/integrations), and follow the instructions. If you already have a Kubernetes cluster connected, skip this step.

#### 3. Deploy a MySQL database instance
Already have a MySQL database instance? [Skip to the tutorial.](#tutorial)

<details>
<summary>Deploy a MySQL database instance, based on Amazon RDS for MySQL</summary>

Follow the [installation instructions on the AWS RDS documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html#CHAP_GettingStarted.Creating.MySQL).

<li>
You may use the Free tier template for this tutorial.
</li>
<li>
Under "Settings", choose "Auto generate password". Make sure you save the generated password after the instance is created.
</li>
<li>
Under "Connectivity", enable public access to allow access from your Kubernetes cluster. Otterize will require that access to manage credentials for you.
Additionally, make sure you choose a security group that allows inbound access from the internet.
</li>

</details>



# Tutorial

### Setup MySQL database and table for the tutorial
Throughout this tutorial, we will refer to your MySQL host & credentials via environment variables, so make sure to set them up:
```shell
export MYSQLHOST=<YOURMYSQLHOST>
export MYSQLPASSWORD=<YOURPASSWORD>
```

Next, run the following command to create a database named `otterize_tutorial` and a table named `example` in your MySQL instance.
Our tutorial server will use this database and table to perform `INSERT` and `SELECT` operations.
```shell
export MYSQL_PWD=$MYSQLPASSWORD
curl ${ABSOLUTE_URL}/code-examples/mysql/db-setup.sql | mysql -u admin -h $MYSQLHOST --verbose
```

### Deploy tutorial services and request database credentials
Next, set up the namespace used for our tutorial and deploy the client & server services in it:

```shell
kubectl create namespace otterize-tutorial-mysql
kubectl apply -n otterize-tutorial-mysql -f ${ABSOLUTE_URL}/code-examples/mysql/client-server.yaml
kubectl patch deployment -n otterize-tutorial-mysql server --type='json' -p="[{\"op\": \"replace\", \"path\": \"/spec/template/spec/containers/0/env/0/value\", \"value\": \"$MYSQLHOST\"}]"
```


<details>
<summary>Expand to see the deployment YAML</summary>

```yaml
{@include: ../../../../static/code-examples/mysql/client-server.yaml}
```
</details>

Our server's Deployment spec specify an annotation on its Pod, which requests that the Otterize operator provision a username and password for it:
```yaml
template:
metadata:
annotations:
credentials-operator.otterize.com/user-password-secret-name: server-creds
```
This specifies that the secret `server-creds` will be populated with keys containing the username and password used by this pod to connect to the database.
The secret will only be created by the Otterize operator after it is integrated with your database by applying a MySQLServerConfig resources.


### View logs for the server
After the client, server, and database are up and running, we can see that the server does not have the appropriate access to the database by inspecting the logs with the following command.

```shell
kubectl logs -f -n otterize-tutorial-mysql deploy/server
```

Example log:
<Terminal>
Unable to perform INSERT operation
<br></br>
Unable to perform SELECT operation
</Terminal>


### Deploy a MySQLServerConfig to allow Otterize DB access
Let's apply a `MySQLServerConfig` so Otterize will know how to access our database instance:
```shell
kubectl apply -n otterize-tutorial-mysql -f ${ABSOLUTE_URL}/code-examples/mysql/mysqlserverconfig.yaml
kubectl patch mysqlserverconfig -n otterize-tutorial-mysql mysql-tutorial-db --type='json' -p="[{\"op\": \"replace\", \"path\": \"/spec/address\", \"value\": \"$MYSQLHOST\"}, {\"op\": \"replace\", \"path\": \"/spec/credentials/password\", \"value\": \"$MYSQLPASSWORD\"}]"
```

This applies the following `MySQLServerConfig` to your cluster, and patches it with your DB instance & credentials:


```yaml
{@include: ../../../../static/code-examples/mysql/mysqlserverconfig.yaml}
```

The above CRD tells Otterize how to access a database instance named `mysql-tutorial-db`, meaning that when intents
are applied requesting access permissions to `mysql-tutorial-db`, the Otterize operator will be able to configure
them.

In this tutorial, we use the admin user to grant Otterize permissions to create users and grant them access to the database.
In a production environment, it is recommended to create a dedicated user for Otterize, and grant it the necessary permissions to create and manage other users.

:::caution
The type MySQLServerConfig should be considered as sensitive and require high cluster privileges to access.
:::


### Define your ClientIntents

ClientIntents are Otterize’s way of defining access through unique relationships, which lead to perfectly scoped access. In this example, we provide our `server` workload the ability to insert and select records to allow it to access the database.

Below is our `intents.yaml` file. As you can see, it is scoped to our database named `otterize_tutorial` and our `example` table. We also have limited the access to just `SELECT` and `INSERT` operations. We could add more databases, tables, or operations if our service required more access.

Specifying the table and operations is optional. If you don't specify the table, access will be granted to all tables in the specified database. If you don't specify the operations, all operations will be allowed.
```yaml
{@include: ../../../../static/code-examples/mysql/clientintents.yaml}
```
We can now apply our intents. Behind the scenes, the Otterize operator created the user for our `server` workload and executed `GRANT` queries on the database, making our `SELECT` and `INSERT` errors disappear.

```shell
kubectl apply -n otterize-tutorial-mysql -f ${ABSOLUTE_URL}/code-examples/mysql/clientintents.yaml
```

### View logs for the server
We can now view the server logs once again. This time, we should see that the server has the appropriate access to the database:

```shell
kubectl logs -f -n otterize-tutorial-mysql deploy/server
```

Example log:
<Terminal>
Successfully INSERTED into our table

Successfully SELECTED, most recent value: 2024-04-30T13:20:46Z
</Terminal>

That’s it! If your service’s functionality changes, adding or removing access is as simple as updating your ClientIntents definitions. For fun, try altering the `operations` to just `SELECT` or `INSERT`.

# Teardown
To remove the deployed examples, run:
```shell
kubectl delete clientintents.k8s.otterize.com -n otterize-tutorial-mysql client-intents-for-server
kubectl delete namespace otterize-tutorial-mysql
```
4 changes: 2 additions & 2 deletions docs/features/postgresql/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ export const postgres_tutorials = [

# PostgreSQL

Otterize is able to create just-in-time username-and-password pairs for your service, providing them as a Kubernetes Secret that can be mounted to file or mapped to environment variables, as well as `GRANT`ing access to databases and tables, based on `ClientIntents` ([Intents-Based Access Control](/overview/intent-based-access-control)) declarations.
Otterize is able to create just-in-time username-and-password pairs for your service, providing them as a Kubernetes Secret that can be mounted to file or mapped to environment variables, as well as `GRANT`ing access to databases and tables, based on `ClientIntents` ([Intent-Based Access Control](/overview/intent-based-access-control)) declarations.
In addition, Otterize can map the access to your PostgreSQL database, showing you which service is accessing which database, table and which operation it's performing. This can be used to automatically generate the `ClientIntents` declarations.

### Tutorials
Expand Down Expand Up @@ -104,7 +104,7 @@ spec:
service:
name: server
calls:
- name: postgres-tutorial-db # Same name as our PostgresSQLServerConfig metadata.name
- name: postgres-tutorial-db # Same name as our PostgreSQLServerConfig metadata.name
type: database
databaseResources:
- databaseName: otterize-tutorial
Expand Down
Loading

0 comments on commit 8503e92

Please sign in to comment.