Skip to content

Commit

Permalink
Resolving comments
Browse files Browse the repository at this point in the history
  • Loading branch information
@davidgs
davidgs committed Aug 18, 2023
1 parent be01280 commit de4c8b6
Show file tree
Hide file tree
Showing 9 changed files with 13,464 additions and 123 deletions.
105 changes: 17 additions & 88 deletions docs/quick-tutorials/k8s-kafka-mtls.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,14 @@ title: Kafka access automation
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

This tutorial will walk you through declaring and applying intents to easily secure access to Kafka running inside a Kubernetes cluster,
automating the management of Kafka ACLs, and the generation and deployment of certificates for mTLS between Kafka and its clients.
This tutorial will walk you through declaring and applying intents to easily secure access to Kafka running inside a Kubernetes cluster, automating the management of Kafka ACLs, and the generation and deployment of certificates for mTLS between Kafka and its clients.

In this tutorial, we will:

- Deploy a Kafka broker with 2 topics, and two clients that call these topics.
- Protect just 1 of those topics with Otterize. See that access to that topic is blocked.
- Declare that one client pod intends to access that protected topic.
- See that an ACL was autogenerated to allow just that, while blocking calls to that topic from the other client.
- Also verify that mTLS credentials were autogenerated for all clients.
- See that an ACL was auto-generated to allow just that, while blocking calls to that topic from the other client.

## Prerequisites

Expand All @@ -30,7 +28,7 @@ You can now install (or reinstall) Otterize in your cluster, and optionally conn
1. See what's happening visually in your browser, through the "access graph";
2. Avoid using SPIRE (which can be installed with Otterize) for issuing certificates, as Otterize Cloud provides a certificate service.

So either forego browser visualization and:
So either forgo browser visualization and:

<details>
<summary>Install Otterize in your cluster, <b>without</b> Otterize Cloud (and no network policy management)</summary>
Expand Down Expand Up @@ -122,17 +120,15 @@ Our Kafka server will have 2 topics `transactions` and `mytopic` that our client
</Tabs>
</details>

We've chosen to allow unauthenticated access to both topics to begin with. Initially, we're not requiring declared intents so everything is open here. This will be the base state, from which we can gradually secure access to specific topics.
We've chosen not to allow unauthenticated access to both topics to begin with. We're also requiring declared intents, but since we have not turned enforcement on, everything is open. This will be the base state, from which we can gradually secure access to specific topics.

## Deploy clients

Our simple example consists of two client pods:
- One named "**client**".
- And one named "**client-other**".

These clients will eventually authenticate to Kafka using mTLS, but we haven't turned that on just yet. Otterize makes this easy, requiring just 3 simple changes to the client pod spec:
1. **Generate credentials**: add the `credentials-operator.otterize.com/tls-secret-name` annotation, which tells Otterize to generate mTLS credentials and store them in a Kubernetes secret whose name is the value of this annotation.
2. **Expose credentials in a volume**: add a volume containing this secret to the pod.
These clients are connecting to Kafka using mTLS, the credentials which they will receive from Otterize. Otterize makes this easy, requiring just 4 simple changes:
3. **Mount the volume**: mount the volume in every container in the pod.
4. **Enable mTLS**: configure the Kafka client library to require authentication.

Expand Down Expand Up @@ -250,14 +246,14 @@ This client should be able to access both topics:
[client] Read message from topic: mytopic - [client-other] Sent message 19 [mytopic]
```

As you can see, both `client` and `client-other` are currently able to access both `mytopic` and `transactions` topics.
As you can see, both `client` and `client-other` are currently able to access both `mytopic` and `transactions` topics. (We see `client` sending messages and reading messages sent by `client-other` so we know both are able to access both topics.)

3. **Open another terminal window [client-other]** and tail the client-other log:


This other client should also be able to access both topics:
```bash
kubectl logs -f --tail 1 -n otterize-tutorial-kafka-mtls deploy/client-other
```
This other client should also be able to access both topics:
```
[client-other] Loading mTLS certificates
[client-other] Connecting to Kafka
[client-other] Creating a producer and a consumer for - mytopic
Expand All @@ -275,42 +271,20 @@ If you've attached Otterize OSS to Otterize Cloud, you can now browse to your ac

![Access graph](/img/quick-tutorials/k8s-kafka-mtls/base.png)

You can see from the Access Graph that no clients are currently blocked because we haven't enabled any sort of enforcement yet.
The access graph shows, through its green and orange lines linking the services, that no clients are currently blocked because we haven't enabled any sort of enforcement yet. The orange lines indicate that, since we have not declared any intents for these clients, they *would* be blocked if we were to turn enforcement on.

<details>
<summary>Why do I see eight services?</summary>
<summary>Why do I see seven services?</summary>

In addition to the Kafka service and the 3 clients we deployed (yes, we snuck a 3rd client in there), the network mapper also picked up the calls between the intents operator and Kafka, and between Kafka and zookeeper, so those discovered intents are reflected in the access graph (in light blue).
In addition to the Kafka service and the 2 clients we deployed, the network mapper also picked up the calls between the intents operator and Kafka, and between Kafka and zookeeper, so those intents are reflected in the access graph.

We also deployed the Otterize kafka-watcher to monitor Kafka for new topics and intents, and that watcher communicates with the network-mapper in order to report the discovered intents (in grey).
We also deployed the Otterize kafka-watcher to monitor Kafka for new topics and intents, and that watcher communicates with the network-mapper in order to report the discovered intents.

The access graph also reflects an intent that's already been declared and applied, and was reported by the intents operator to the access graph: it's the intent which the intents operator created for itself to ensure it has access to Kafka. That's [automatically generated](/reference/configuration/intents-operator/configuration) by the intents operator when you apply the KafkaServerConfig: at that point the intents operator knows there is a Kafka service, and in order to ensure it can reach it and configure it, it declares its intent to do so. Specifically that will generate a network policy between the intents operator and the Kafka service, if network policies are in active enforcement and supported by your cluster, so the intents operator doesn't get blocked itself.

:::note
If you're running this tutorial in GKE, you'll also see the `konnectivity-agent` and the `metrics-server` as in the image above.
:::
</details>
<details>
<summary>Optional: tail the logs for client-authenticated</summary>
```bash
kubectl logs -f --tail 1 -n otterize-tutorial-kafka-mtls deploy/client-authenticated
```
```
[client-authenticated] Loading mTLS certificates
[client-authenticated] Connecting to Kafka
[client-authenticated] Creating a producer and a consumer for - mytopic
[client-authenticated] Sending messages
[client-authenticated] Creating a producer and a consumer for - transactions
[client-authenticated] Sending messages
[debug: client-authenticated] Sent message to [mytopic] - [client-authenticated] Sent message 1
[debug: client-authenticated] Sent message to [transactions] - [client-authenticated] Sent message 1
```
</details>

## Apply intents
Expand All @@ -329,70 +303,29 @@ We can apply intents for the `client` by applying the `client-intents.yaml` file
kubectl apply -f https://docs.otterize.com/code-examples/kafka-mtls/client-intents.yaml
```

If you go back to your access graph, you'll now see that the `client` has a solid green arrow connecting it to the Kafka broker. This is because the `client` has both declared its intent to access the Kafka broker, and it has been authenticated using mTLS.
If you go back to your access graph, you'll now see that the `client` has a solid green line connecting it to the Kafka broker. This is because the `client` has both declared its intent to access the Kafka broker, and it has been authenticated using mTLS.

If you click on that solid line, you will see that the declared intents match the discovered intents, so access is assured.

![client intents applied](../../static/img/quick-tutorials/k8s-kafka-mtls/client-intents.png)

2. At this point, since the Kafka server is not actually protected, the `client-other` can still access the topics. The line is dashed, not solid, indicating that it has no declared intents.
2. At this point, since the Kafka server is not actually protected, the `client-other` can still access the topics. The line is orange, indicating that it has no declared intents.

![Declared Intent](../../static/img/quick-tutorials/k8s-kafka-mtls/declared-intent.png)

3. Now let's make a change to the `kafkaserverconfig` to require intents for the 2 topics.
```bash
kubectl apply -f https://docs.otterize.com/code-examples/kafka-mtls/kafkaserverconfig-protect.yaml
```
<details>
<summary>Expand to see kafkaserverconfig-protect</summary>
<Tabs>
<TabItem value="kafkaserverconfig-protect.yaml" label="kafkaserverconfig-protect.yaml" default>
```yaml
{@include: ../../static/code-examples/kafka-mtls/kafkaserverconfig-protect.yaml}
```
</TabItem>
</Tabs>
</details>
Keep an eye on your Otterize Cloud access graph as you do this, as the 2 clients *without* declared intents should turn orange indicating that, if enforcement were turned on, they would be blocked. This is because we changed the KafkaServerConfig to require intents for those topics with `intentRequired: true`:
![Blocked Without Intent](../../static/img/quick-tutorials/k8s-kafka-mtls/blocked-without-intent.png)
4. Verify that an ACL for this client was configured on the Kafka broker:
```bash
kubectl logs -n kafka statefulset/kafka | grep "Processing Acl change" | grep mytopic | tail -n 1
```
You should see:
```
[2022-09-13 10:44:52,803] INFO Processing Acl change notification for
ResourcePattern(resourceType=TOPIC, name=mytopic, patternType=LITERAL),
versionedAcls : Set(User:ANONYMOUS has DENY permission for operations:
ALL from hosts: *, User:CN=client.otterize-tutorial-kafka-mtls,O=SPIRE,C=US has ALLOW permission
for operations: ALL from hosts: *), zkVersion : 6 (kafka.security.authorizer.AclAuthorizer)
```
If you've attached Otterize OSS to Otterize Cloud, go back to see the [access graph in your browser](https://app.otterize.com). Click on the Kafka service, and click at the bottom of it to focus on it and show its details:

![Access graph](/img/quick-tutorials/k8s-kafka-mtls/protected.png)

We can see what happened:
1. Kafka topic-specific intents from **[client]** are declared (solid green line).
2. Calls from **[client-other]** are not declared (orange line).
3. Looking at the Kafka service, we can see that **[client]** has specific access configured (via Kafka ACLs) to perform `all` operations on the `mytopic` topic.

Since discovered intents from the network mapper don't specify what specific topics and operations clients are performing (or attempting to perform), the access graph cannot show information on what is being blocked vs allowed (red vs green). That feature is in development.
Since discovered intents from the network mapper don't specify what specific topics and operations clients are performing (or attempting to perform), the access graph cannot show information on what is being blocked vs allowed (red vs green) on a per-topic basis. That feature is in development.

Also, the access graph shows information about the mTLS certificates (credentials) distributed to the various services, as long as [Cloud-managed credentials](/security#cryptographic-credentials) are being used. Visibility for certificates distributed through an in-cluster SPIRE is in development.

## Turn on protection

At this point, we haven't actually protected our Kafka broker. From everything we've done so far, we can see, however, that if we were to turn on protection, the `client-other` and `client-authenticated` would lose access to the broker.
At this point, we haven't actually protected our Kafka broker. From everything we've done so far, we can see, however, that if we were to turn on protection, the `client-other` would lose access to the broker.

Let's see that in action. Our clients that have not declared intents will be blocked from accessing the broker.

Expand Down Expand Up @@ -468,10 +401,6 @@ This allowed the *client* pod its access and protected `mytopic` from any uninte

</details>

:::tip Bonus tutorial
Try to create an intents file yourself for **client-other**, and apply it to allow this other client to access the topic.
:::
## What's next
- Follow [a more visual tutorial](/quick-visual-tutorials/visual-ibac-kafka-k8s) for securing Kafka with IBAC in a demo ecommerce application.
- Learn how to easily secure pod-to-pod access with IBAC using Kubernetes network policies, in [a hands-on tutorial](/quick-tutorials/k8s-network-policies) or [a more visual tutorial](/quick-visual-tutorials/visual-ibac-network-policies).
Expand Down
Loading

0 comments on commit de4c8b6

Please sign in to comment.