From 256c559d751e66955d288624a5c8415fca56b69b Mon Sep 17 00:00:00 2001 From: Ro'e Katz Date: Mon, 28 Aug 2023 17:48:36 +0300 Subject: [PATCH 1/7] Update docs with new field `credentialsProvider` that also supports "cert-manager" instead of the old `useCloudToGenerateTLSCredentials` --- .../credentials-operator/README.mdx | 2 +- .../credentials-operator/helm-chart.mdx | 30 ++++++++++++------- .../configuration/otterize-chart/README.mdx | 6 +++- docs/security/README.mdx | 6 ++-- 4 files changed, 29 insertions(+), 15 deletions(-) diff --git a/docs/reference/configuration/credentials-operator/README.mdx b/docs/reference/configuration/credentials-operator/README.mdx index 14b255b81..0fad5f519 100644 --- a/docs/reference/configuration/credentials-operator/README.mdx +++ b/docs/reference/configuration/credentials-operator/README.mdx @@ -38,7 +38,7 @@ To learn more, check out the documentation for [SPIRE](https://spiffe.io/docs/la ### SPIRE entry registration Once the operator [resolves the service name](#service-name-resolution-and-automatic-pod-labeling) for a pod, it labels the pod and registers an entry with the SPIRE server. -If configured to use Otterize Cloud credentials instead of SPIRE (`useCloudToGenerateTLSCredentials` in the Helm chart configuration), it registers the Otterize service with Otterize Cloud. +If configured to use Otterize Cloud credentials instead of SPIRE (`global.credentialsProvider=otterize-cloud` in the Helm chart configuration), it registers the Otterize service with Otterize Cloud. ### Credential generation The operator consults the label `credentials-operator.otterize.com/tls-secret-name`. If that label exists, the operator creates a secret named after the value of the label with X.509 credentials within, provided by SPIRE or Otterize Cloud, depending on how the credentials operator is configured. This way, the pod can get autogenerated credentials without modifying its code. diff --git a/docs/reference/configuration/credentials-operator/helm-chart.mdx b/docs/reference/configuration/credentials-operator/helm-chart.mdx index 98b8509e8..a52fe3e32 100644 --- a/docs/reference/configuration/credentials-operator/helm-chart.mdx +++ b/docs/reference/configuration/credentials-operator/helm-chart.mdx @@ -15,16 +15,17 @@ If you would like to deploy it on its own, add the Otterize Helm chart repositor # Parameters ## Global parameters -| Key | Description | Default | -|--------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|---------| -| `global.spire.serverServiceName` | If deployed with SPIRE, this key specifies SPIRE-server's service name. You should use either this **OR** `spire.serverAddress` (not both). | | -| `global.allowGetAllResources` | If defined overrides `allowGetAllResources`. | | | `false` | -| `global.commonAnnotations` | Annotations to add to all deployed objects | {} | -| `global.commonLabels` | Labels to add to all deployed objects | {} | -| `global.podAnnotations` | Annotations to add to all deployed pods | {} | -| `global.podLabels` | Labels to add to all deployed pods | {} | -| `global.serviceNameOverrideAnnotationName` | Which annotation to use (in the [service name resolution algorithm](/reference/service-identities#kubernetes-service-identity-resolution)) for setting a pod's service name, if not the default. Use this if you already have annotations on your pods that provide the correct service name. | `intents.otterize.com/service-name` | +| Key | Description | Default | +|----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|-----------| +| `global.credentialsProvider` | What provider should be used to generate certificates/credentials - `"spire"`, `"otterize-cloud"` or `"cert-manager"` | `"spire"` | +| `global.spire.serverServiceName` | If deployed with SPIRE, this key specifies SPIRE-server's service name. You should use either this **OR** `spire.serverAddress` (not both). | | +| `global.allowGetAllResources` | If defined overrides | `false` | +| `global.commonAnnotations` | Annotations to add to all deployed objects | {} | +| `global.commonLabels` | Labels to add to all deployed objects | {} | +| `global.podAnnotations` | Annotations to add to all deployed pods | {} | +| `global.podLabels` | Labels to add to all deployed pods | {} | +| `global.serviceNameOverrideAnnotationName` | Which annotation to use (in the [service name resolution algorithm](/reference/service-identities#kubernetes-service-identity-resolution)) for setting a pod's service name, if not the default. Use this if you already have annotations on your pods that provide the correct service name. | `intents.otterize.com/service-name` | ## SPIRE parameters @@ -38,14 +39,14 @@ If you would like to deploy it on its own, add the Otterize Helm chart repositor | Key | Description | Default | |-----------------------------|----------------------------|------------------------------| | `operator.image.repository` | Operator image repository. | `otterize` | -| `operator.image.image` | Operator image. | `credentials-operator` | +| `operator.image.image` | Operator image. | `credentials-operator` | | `operator.image.tag` | Operator image tag. | `latest` | | `operator.pullPolicy` | Operator pull policy. | `(none)` | ## Cloud parameters + | Key | Description | Default | |------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| -| `global.otterizeCloud.useCloudToGenerateTLSCredentials` | Use Otterize Cloud for certificate management instead of SPIRE | `false` | | `global.otterizeCloud.credentials.clientId` | Client ID for connecting to Otterize Cloud. | `(none)` | | `global.otterizeCloud.credentials.clientSecret` | Client secret for connecting to Otterize Cloud. | `(none)` | | `global.otterizeCloud.credentials.secretKeyRef.secretName` | If specified, the name of a pre-created Kubernetes Secret to be used instead of creating a secret with the value of clientSecret. | `(none)` | @@ -53,6 +54,13 @@ If you would like to deploy it on its own, add the Otterize Helm chart repositor | `global.otterizeCloud.apiAddress` | Overrides Otterize Cloud default API address. | `(none)` | | `global.otterizeCloud.apiExtraCAPEMSecret` | The name of a secret containing a single `CA.pem` file for an extra root CA used to connect to Otterize Cloud. The secret should be placed in the same namespace as the Otterize deployment. | `(none)` | +## cert-manager parameters + +| Key | Description | Default | +|--------------------------------|---------------------------------------------------------------------------------------------------------------|---------| +| `certManager.issuerName` | The cert-manager Issuer (or ClusterIssuer if `useClusterIssuer` is set) to be used for certificate generation | `""` | +| `certManager.useClusterIssuer` | Use ClusterIssuer instead of the namespace scoped Issuer. | `false` | + ## Common parameters | Key | Description | Default | |------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| diff --git a/docs/reference/configuration/otterize-chart/README.mdx b/docs/reference/configuration/otterize-chart/README.mdx index 2483eff20..3ca2cdd95 100644 --- a/docs/reference/configuration/otterize-chart/README.mdx +++ b/docs/reference/configuration/otterize-chart/README.mdx @@ -26,6 +26,7 @@ These parameters are used by multiple charts, and must be kept the same for the | `global.spiffe.CASubject.country` | SPIRE's CA certificates `Country` value. | `"US"` | | `global.spiffe.CASubject.organization` | SPIRE's CA certificates `Organization` Value. | `"SPIRE"` | | `global.spiffe.trustDomain` | The trust domain that SPIRE will use. | `"example.org"` | +| `global.credentialsProvider` | What provider should be used to generate certificates/credentials - `"spire"`, `"otterize-cloud"` or `"cert-manager"` | `"spire"` | | `global.spire.serverServiceName` | Name of the Kubernetes service that will be created for SPIRE-server. | | | `global.allowGetAllResources` | If defined overrides `allowGetAllResources` in subcharts. Gives get, list and watch permission to watch on all resources. This is used to resolve service names when pods have owners that are custom resources. When disabled, a limited set of permissions is used that only allows access to built-in Kubernetes resources that deploy Pods and Pods themselves - Deployments, StatefulSets, DaemonSets, ReplicaSets and Services. Resolving may not be able to complete if the owning resource is not one of those. | | | `global.telemetry.enabled` | If set to `false`, anonymous telemetries collection will be disabled | `true` | @@ -39,7 +40,6 @@ These parameters are used by multiple charts, and must be kept the same for the ## Cloud parameters | Key | Description | Default | |------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| -| `global.otterizeCloud.useCloudToGenerateTLSCredentials` | Use Otterize Cloud for certificate management instead of SPIRE. | `false` | | `global.otterizeCloud.credentials.clientId` | Client ID for connecting to Otterize Cloud. | `(none)` | | `global.otterizeCloud.credentials.clientSecret` | Client secret for connecting to Otterize Cloud. | `(none)` | | `global.otterizeCloud.credentials.secretKeyRef.secretName` | If specified, the name of a pre-created Kubernetes Secret to be used instead of creating a secret with the value of clientSecret. | `(none)` | @@ -78,6 +78,10 @@ Further information about `SPIRE` parameters can be found [in SPIRE's Helm chart All configurable parameters of the network mapper can be configured under the alias `networkMapper`. Further information about network mapper parameters can be found [in the network mapper's chart](https://github.com/otterize/helm-charts/tree/main/network-mapper). +## Credentials operator parameters +All configurable parameters of the credentials operator can be configured under the alias `credentialsOperator`. +Further information about network mapper parameters can be found [in the network mapper's chart](https://github.com/otterize/helm-charts/tree/main/credentials-operator). + ## Resource configuration | Component | Key | Default | |----------------------------|--------------------------------------|----------| diff --git a/docs/security/README.mdx b/docs/security/README.mdx index fc7426799..a5c1d591b 100644 --- a/docs/security/README.mdx +++ b/docs/security/README.mdx @@ -103,11 +103,13 @@ The service names in each namespace are set by the developers, or the platform t ### Cryptographic credentials -The cryptographic credentials created by the Otterize credentials operator (mTLS certificate and key pairs) are provided in one of two ways, depending on how the operator is configured: +The cryptographic credentials created by the Otterize credentials operator (mTLS certificate and key pairs) are provided in one of three ways, depending on how the operator is configured: 1. By a SPIRE server that is deployed alongside the credentials operator. 2. By the Otterize Cloud managed credentials service. That service is built on a Hashicorp Vault instance with a CA automatically created for you. Using Otterize Cloud for credentials means you do not need to deploy SPIRE on your cluster, which makes for a simpler and lightweight deployment. +3. By an independently deployed [cert-manager](https://github.com/cert-manager/cert-manager) operator. -Note that, by default, the first option (in-cluster SPIRE) is used, even when the credentials operator is connected to Otterize Cloud). The `useCloudToGenerateTLSCredentials` must be set to `true` to use Cloud-managed credentials; see the [Helm chart configuration for the credentials operator](/reference/configuration/credentials-operator/helm-chart#cloud-parameters) for more details. +Note that, by default, the first option (in-cluster SPIRE) is used, even when the credentials operator is connected to Otterize Cloud). +The `credentialsProvider` could be set to `otterize-cloud` or `cert-manager` to use the other options; see the [Helm chart configuration for the credentials operator](/reference/configuration/credentials-operator/helm-chart#cloud-parameters) for more details. The credentials operator watches for pods starting up in a Kubernetes cluster, and if mTLS credentials are requested, it uses the [resolved or declared service name](/reference/service-identities) plus the pod’s namespace to generate credentials for that service name, in that namespace. The operator is conceptually similar to a SPIRE agent in that it attests to the identity of pods. From eecbf8f2efa253f3b411ab4cbf332c6b1f960cdf Mon Sep 17 00:00:00 2001 From: Ori Shoshan Date: Tue, 12 Sep 2023 21:28:16 +0300 Subject: [PATCH 2/7] credentialsProvider -> certificateProvider --- docs/reference/configuration/credentials-operator/README.mdx | 2 +- .../reference/configuration/credentials-operator/helm-chart.mdx | 2 +- docs/reference/configuration/otterize-chart/README.mdx | 2 +- docs/security/README.mdx | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/reference/configuration/credentials-operator/README.mdx b/docs/reference/configuration/credentials-operator/README.mdx index 0fad5f519..bd3ee3a46 100644 --- a/docs/reference/configuration/credentials-operator/README.mdx +++ b/docs/reference/configuration/credentials-operator/README.mdx @@ -38,7 +38,7 @@ To learn more, check out the documentation for [SPIRE](https://spiffe.io/docs/la ### SPIRE entry registration Once the operator [resolves the service name](#service-name-resolution-and-automatic-pod-labeling) for a pod, it labels the pod and registers an entry with the SPIRE server. -If configured to use Otterize Cloud credentials instead of SPIRE (`global.credentialsProvider=otterize-cloud` in the Helm chart configuration), it registers the Otterize service with Otterize Cloud. +If configured to use Otterize Cloud credentials instead of SPIRE (`global.certificateProvider=otterize-cloud` in the Helm chart configuration), it registers the Otterize service with Otterize Cloud. ### Credential generation The operator consults the label `credentials-operator.otterize.com/tls-secret-name`. If that label exists, the operator creates a secret named after the value of the label with X.509 credentials within, provided by SPIRE or Otterize Cloud, depending on how the credentials operator is configured. This way, the pod can get autogenerated credentials without modifying its code. diff --git a/docs/reference/configuration/credentials-operator/helm-chart.mdx b/docs/reference/configuration/credentials-operator/helm-chart.mdx index a52fe3e32..de5d9e350 100644 --- a/docs/reference/configuration/credentials-operator/helm-chart.mdx +++ b/docs/reference/configuration/credentials-operator/helm-chart.mdx @@ -18,7 +18,7 @@ If you would like to deploy it on its own, add the Otterize Helm chart repositor | Key | Description | Default | |----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|-----------| -| `global.credentialsProvider` | What provider should be used to generate certificates/credentials - `"spire"`, `"otterize-cloud"` or `"cert-manager"` | `"spire"` | +| `global.certificateProvider` | What provider should be used to generate certificates - `"spire"`, `"otterize-cloud"` or `"cert-manager"` | `"spire"` | | `global.spire.serverServiceName` | If deployed with SPIRE, this key specifies SPIRE-server's service name. You should use either this **OR** `spire.serverAddress` (not both). | | | `global.allowGetAllResources` | If defined overrides | `false` | | `global.commonAnnotations` | Annotations to add to all deployed objects | {} | diff --git a/docs/reference/configuration/otterize-chart/README.mdx b/docs/reference/configuration/otterize-chart/README.mdx index 43a7b250c..d425c5666 100644 --- a/docs/reference/configuration/otterize-chart/README.mdx +++ b/docs/reference/configuration/otterize-chart/README.mdx @@ -26,7 +26,7 @@ These parameters are used by multiple charts, and must be kept the same for the | `global.spiffe.CASubject.country` | SPIRE's CA certificates `Country` value. | `"US"` | | `global.spiffe.CASubject.organization` | SPIRE's CA certificates `Organization` Value. | `"SPIRE"` | | `global.spiffe.trustDomain` | The trust domain that SPIRE will use. | `"example.org"` | -| `global.credentialsProvider` | What provider should be used to generate certificates/credentials - `"spire"`, `"otterize-cloud"` or `"cert-manager"` | `"spire"` | +| `global.certificateProvider` | What provider should be used to generate certificates/credentials - `"spire"`, `"otterize-cloud"` or `"cert-manager"` | `"spire"` | | `global.spire.serverServiceName` | Name of the Kubernetes service that will be created for SPIRE-server. | | | `global.allowGetAllResources` | If defined overrides `allowGetAllResources` in subcharts. Gives get, list and watch permission to watch on all resources. This is used to resolve service names when pods have owners that are custom resources. When disabled, a limited set of permissions is used that only allows access to built-in Kubernetes resources that deploy Pods and Pods themselves - Deployments, StatefulSets, DaemonSets, ReplicaSets and Services. Resolving may not be able to complete if the owning resource is not one of those. | | | `global.telemetry.enabled` | If set to `false`, anonymous telemetries collection will be disabled | `true` | diff --git a/docs/security/README.mdx b/docs/security/README.mdx index a5c1d591b..17be04129 100644 --- a/docs/security/README.mdx +++ b/docs/security/README.mdx @@ -109,7 +109,7 @@ The cryptographic credentials created by the Otterize credentials operator (mTLS 3. By an independently deployed [cert-manager](https://github.com/cert-manager/cert-manager) operator. Note that, by default, the first option (in-cluster SPIRE) is used, even when the credentials operator is connected to Otterize Cloud). -The `credentialsProvider` could be set to `otterize-cloud` or `cert-manager` to use the other options; see the [Helm chart configuration for the credentials operator](/reference/configuration/credentials-operator/helm-chart#cloud-parameters) for more details. +The `certificateProvider` could be set to `otterize-cloud` or `cert-manager` to use the other options; see the [Helm chart configuration for the credentials operator](/reference/configuration/credentials-operator/helm-chart#cloud-parameters) for more details. The credentials operator watches for pods starting up in a Kubernetes cluster, and if mTLS credentials are requested, it uses the [resolved or declared service name](/reference/service-identities) plus the pod’s namespace to generate credentials for that service name, in that namespace. The operator is conceptually similar to a SPIRE agent in that it attests to the identity of pods. From 67053382aa6f016fe2c5e19076ac219285545c76 Mon Sep 17 00:00:00 2001 From: Ori Shoshan Date: Tue, 12 Sep 2023 22:47:07 +0300 Subject: [PATCH 3/7] Credentials operator cert-manager docs --- .../credentials-operator/README.mdx | 48 ++++++++++++------- .../credentials-operator/helm-chart.mdx | 21 ++++---- docs/security/README.mdx | 4 +- 3 files changed, 44 insertions(+), 29 deletions(-) diff --git a/docs/reference/configuration/credentials-operator/README.mdx b/docs/reference/configuration/credentials-operator/README.mdx index bd3ee3a46..351fc5550 100644 --- a/docs/reference/configuration/credentials-operator/README.mdx +++ b/docs/reference/configuration/credentials-operator/README.mdx @@ -8,9 +8,39 @@ The Otterize credentials operator automatically resolves pods to dev-friendly se ## Deploying the credentials operator To deploy the operator, [use the Helm chart](/reference/configuration/credentials-operator/helm-chart). +To deploy with Otterize Cloud as the certificate provider, we recommend you [follow the instructions in Otterize Cloud](https://app.otterize.com/). +To deploy with cert-manager as the certificate provider, you must also [configure the Issuer name and whether it should look for a ClusterIssuer or an Issuer (namespace-scoped)](/reference/configuration/credentials-operator/helm-chart#cert-manager-parameters). + ## Acquiring mTLS credentials using the credentials operator The credentials operator is controlled using annotations placed on pods. To have it provision credentials and place them in Secrets, you must specify `credentials-operator.otterize.com/tls-secret-name`. +## How does the credentials operator work? + +### 1. SPIRE entry registration +This step only happens if the operator is configured to use SPIRE for certificate generation. Once the operator [resolves the service name](#service-name-resolution-and-automatic-pod-labeling) for a pod, it labels the pod so that SPIRE can find it, registers an entry with the SPIRE server for that label. + +### 2. Certificate generation +The operator consults the annotation `credentials-operator.otterize.com/tls-secret-name`. If that annotation exists, the operator creates a secret named after the value of the label with X.509 credentials within, provided by cert-manager, Otterize Cloud or SPIRE, depending on how the credentials operator is configured. + +#### cert-manager +The operator creates a cert-manager [`Certificate`](https://cert-manager.io/docs/usage/certificate/) resource, which will create a Kubernetes Secret with the name specified by the value of the annotation `credentials-operator.otterize.com/tls-secret-name`. The common name and DNS names in the certificate is the identity of the service, as resolved by the [service identity resolution algorithm](#3-service-name-resolution-and-automatic-pod-labeling), i.e. `servicename.namespace`. + +The operator will use a ClusterIssuer or an Issuer to create the Certificate resource, which it expects to find in the same namespace as the pods. The Issuer is configured at deploy time, using the [Helm chart](/reference/configuration/credentials-operator/helm-chart). + +In the event that the default approver controller in `cert-manager` is [disabled](https://cert-manager.io/docs/concepts/certificaterequest/#approver-controller), the credentials operator will auto-approve its own [`CertificateRequests`](https://cert-manager.io/docs/concepts/certificaterequest/) + +#### Otterize Cloud +The operator requests certificates from Otterize Cloud, which internally manages them in Hashicorp Vault. The certificates are then placed within a Kubernetes Secret named with the value of the annotation `credentials-operator.otterize.com/tls-secret-name`. + +#### SPIRE +Once the operator has registered the pod with SPIRE, which happens automatically for a pod that has the `credentials-operator.otterize.com/tls-secret-name` annotation upon pod startup. The credentials operator then acquires the SVID and certificates for the CA chain and places them within a Kubernetes Secret. The SVID and DNS names in the certificate is the identity of the service, as resolved by the [service identity resolution algorithm](#3-service-name-resolution-and-automatic-pod-labeling), i.e. `servicename.namespace`. + +### 3. Service name resolution and automatic pod labeling +The operator performs service name resolution for each pod, through the same algorithm used by the [intents operator](/reference/configuration/intents-operator) and [network mapper](/reference/configuration/network-mapper). [Learn more on how the resolution happens](/reference/service-identities#kubernetes-service-identity-resolution). +The value resulting from this resolution process is the service name, which is then used to create credentials for the service, in the format ``. For example: `payments-svc.staging`. + + + ## SPIRE workload registrar When using SPIRE, the operator registers every pod (even those without annotations). Alongside the credentials operator, you could use SPIRE agents and the SPIRE SDK to work with the same SPIRE server. @@ -32,20 +62,4 @@ To learn more, check out the documentation for [SPIRE](https://spiffe.io/docs/la | `credentials-operator.otterize.com/truststore-file-name` | Truststore key name in the secret. When mounted, this is the filename for the truststore. Only used when cert-type is `jks`. | `truststore.jks` | | `credentials-operator.otterize.com/jks-password` | Password for the JKS truststore and keystore. Only used when cert-type is `jks`. | `password` | | `credentials-operator.otterize.com/restart-pod-on-certificate-renewal` | A pod with this annotation (no matter the value) will be restarted after certificate renewal, along with any replicas. Should be ideally set through the pod owner's `template` spec so it will persist between restarts. | N/A | -| `intents.otterize.com/service-name` | Used for [service identity resolution](/reference/service-identities#kubernetes-service-identity-resolution).| | - -## How does the credentials operator work? - -### SPIRE entry registration -Once the operator [resolves the service name](#service-name-resolution-and-automatic-pod-labeling) for a pod, it labels the pod and registers an entry with the SPIRE server. -If configured to use Otterize Cloud credentials instead of SPIRE (`global.certificateProvider=otterize-cloud` in the Helm chart configuration), it registers the Otterize service with Otterize Cloud. - -### Credential generation -The operator consults the label `credentials-operator.otterize.com/tls-secret-name`. If that label exists, the operator creates a secret named after the value of the label with X.509 credentials within, provided by SPIRE or Otterize Cloud, depending on how the credentials operator is configured. This way, the pod can get autogenerated credentials without modifying its code. - -When using SPIRE, once the operator has registered the pod with SPIRE, which happens automatically upon startup, the pod can use the SPIRE Workload API to generate credentials for the SVID `.`. - -### Service name resolution and automatic pod labeling -The operator performs service name resolution for each pod. You can read more about the resolution algorithm [here](/reference/service-identities#kubernetes-service-identity-resolution). -The value resulting from this resolution process forms the value of the label `credentials-operator.otterize.com/service-name`. - +| `intents.otterize.com/service-name` | Used for [service identity resolution](/reference/service-identities#kubernetes-service-identity-resolution).| | \ No newline at end of file diff --git a/docs/reference/configuration/credentials-operator/helm-chart.mdx b/docs/reference/configuration/credentials-operator/helm-chart.mdx index de5d9e350..1dcd2554a 100644 --- a/docs/reference/configuration/credentials-operator/helm-chart.mdx +++ b/docs/reference/configuration/credentials-operator/helm-chart.mdx @@ -27,12 +27,12 @@ If you would like to deploy it on its own, add the Otterize Helm chart repositor | `global.podLabels` | Labels to add to all deployed pods | {} | | `global.serviceNameOverrideAnnotationName` | Which annotation to use (in the [service name resolution algorithm](/reference/service-identities#kubernetes-service-identity-resolution)) for setting a pod's service name, if not the default. Use this if you already have annotations on your pods that provide the correct service name. | `intents.otterize.com/service-name` | -## SPIRE parameters +## cert-manager parameters -| Key | Description | Default | -|-----------------------|----------------------------------------------------------------------------------------------------------------|------------------------| -| `spire.serverAddress` | Specify the SPIRE-server's address. You should use either this OR `global.spire.serverServiceName` (not both). | | -| `spire.socketsPath` | SPIRE sockets path. The operator will expect to find agent.sock in the host-mounted folder | `"/run/spire/sockets"` | +| Key | Description | Default | +|--------------------------------|---------------------------------------------------------------------------------------------------------------|---------| +| `certManager.issuerName` | The cert-manager Issuer (or ClusterIssuer if `useClusterIssuer` is set) to be used for certificate generation | `""` | +| `certManager.useClusterIssuer` | Use ClusterIssuer instead of the namespace scoped Issuer. | `true` | ## Operator parameters @@ -54,13 +54,14 @@ If you would like to deploy it on its own, add the Otterize Helm chart repositor | `global.otterizeCloud.apiAddress` | Overrides Otterize Cloud default API address. | `(none)` | | `global.otterizeCloud.apiExtraCAPEMSecret` | The name of a secret containing a single `CA.pem` file for an extra root CA used to connect to Otterize Cloud. The secret should be placed in the same namespace as the Otterize deployment. | `(none)` | -## cert-manager parameters -| Key | Description | Default | -|--------------------------------|---------------------------------------------------------------------------------------------------------------|---------| -| `certManager.issuerName` | The cert-manager Issuer (or ClusterIssuer if `useClusterIssuer` is set) to be used for certificate generation | `""` | -| `certManager.useClusterIssuer` | Use ClusterIssuer instead of the namespace scoped Issuer. | `false` | +## SPIRE parameters + +| Key | Description | Default | +|-----------------------|----------------------------------------------------------------------------------------------------------------|------------------------| +| `spire.serverAddress` | Specify the SPIRE-server's address. You should use either this OR `global.spire.serverServiceName` (not both). | | +| `spire.socketsPath` | SPIRE sockets path. The operator will expect to find agent.sock in the host-mounted folder | `"/run/spire/sockets"` | ## Common parameters | Key | Description | Default | |------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| diff --git a/docs/security/README.mdx b/docs/security/README.mdx index 17be04129..fc358c7ff 100644 --- a/docs/security/README.mdx +++ b/docs/security/README.mdx @@ -104,9 +104,9 @@ The service names in each namespace are set by the developers, or the platform t ### Cryptographic credentials The cryptographic credentials created by the Otterize credentials operator (mTLS certificate and key pairs) are provided in one of three ways, depending on how the operator is configured: -1. By a SPIRE server that is deployed alongside the credentials operator. +1. By a [cert-manager](https://github.com/cert-manager/cert-manager) deployment in the same cluster. 2. By the Otterize Cloud managed credentials service. That service is built on a Hashicorp Vault instance with a CA automatically created for you. Using Otterize Cloud for credentials means you do not need to deploy SPIRE on your cluster, which makes for a simpler and lightweight deployment. -3. By an independently deployed [cert-manager](https://github.com/cert-manager/cert-manager) operator. +3. By a SPIRE server that is deployed alongside the credentials operator. Note that, by default, the first option (in-cluster SPIRE) is used, even when the credentials operator is connected to Otterize Cloud). The `certificateProvider` could be set to `otterize-cloud` or `cert-manager` to use the other options; see the [Helm chart configuration for the credentials operator](/reference/configuration/credentials-operator/helm-chart#cloud-parameters) for more details. From 83617b85d6f4565d8f9302c1554455867bf8f1d6 Mon Sep 17 00:00:00 2001 From: Ori Shoshan Date: Tue, 12 Sep 2023 22:59:45 +0300 Subject: [PATCH 4/7] Update tutorials --- .../configuration/credentials-operator/README.mdx | 10 +++++----- .../ibac-for-kafka/interactive/all.yaml | 7 ++++++- static/code-examples/kafka-mtls/all.yaml | 12 ++++++++++++ static/code-examples/kafka-mtls/helm/values.yaml | 2 ++ 4 files changed, 25 insertions(+), 6 deletions(-) diff --git a/docs/reference/configuration/credentials-operator/README.mdx b/docs/reference/configuration/credentials-operator/README.mdx index 351fc5550..2c612c15f 100644 --- a/docs/reference/configuration/credentials-operator/README.mdx +++ b/docs/reference/configuration/credentials-operator/README.mdx @@ -55,11 +55,11 @@ To learn more, check out the documentation for [SPIRE](https://spiffe.io/docs/la | `credentials-operator.otterize.com/dns-names` | If set, overrides the list of subject alternative names in the certificate. Should include the hostname of Kubernetes services that will be used to access this pod. | N/A | | `credentials-operator.otterize.com/cert-ttl` | Override for the expiration time for the certificate in seconds. | If deployed with the bundled SPIRE server, 1 day (86400). | | `credentials-operator.otterize.com/cert-type` | Type of the credential bundle - `pem` or `jks`. | `pem` | -| `credentials-operator.otterize.com/cert-file-name` | Certificate key name in the secret. When mounted, this is the filename for the certificate (when using spire it's the SVID file). Only used when cert-type is `pem`. | `svid.pem` | -| `credentials-operator.otterize.com/ca-file-name` | Bundle (certificate chain bundle) key name in the secret. When mounted, this is the filename for the certificate chain. Only used when cert-type is `pem`. | `bundle.pem` | -| `credentials-operator.otterize.com/key-file-name` | Private key key name in the secret. When mounted, this is the filename for the private key. Only used when cert-type is `pem`. | `key.pem` | -| `credentials-operator.otterize.com/keystore-file-name` | Keystore key name in the secret. When mounted, this is the filename for the keystore. Only used when cert-type is `jks`. | `keystore.jks` | -| `credentials-operator.otterize.com/truststore-file-name` | Truststore key name in the secret. When mounted, this is the filename for the truststore. Only used when cert-type is `jks`. | `truststore.jks` | +| `credentials-operator.otterize.com/cert-file-name` | Certificate key name in the secret. When mounted, this is the filename for the certificate (when using SPIRE it's the SVID file). Only used when cert-type is `pem`. Not supported when `certificateProvider` is `cert-manager`. | `svid.pem` | +| `credentials-operator.otterize.com/ca-file-name` | Bundle (certificate chain bundle) key name in the secret. When mounted, this is the filename for the certificate chain. Only used when cert-type is `pem`. Not supported when `certificateProvider` is `cert-manager`. | `bundle.pem` | +| `credentials-operator.otterize.com/key-file-name` | Private key key name in the secret. When mounted, this is the filename for the private key. Only used when cert-type is `pem`. Not supported when `certificateProvider` is `cert-manager`. | `key.pem` | +| `credentials-operator.otterize.com/keystore-file-name` | Keystore key name in the secret. When mounted, this is the filename for the keystore. Only used when cert-type is `jks`. Not supported when `certificateProvider` is `cert-manager`. | `keystore.jks` | +| `credentials-operator.otterize.com/truststore-file-name` | Truststore key name in the secret. When mounted, this is the filename for the truststore. Only used when cert-type is `jks`. Not supported when `certificateProvider` is `cert-manager`. | `truststore.jks` | | `credentials-operator.otterize.com/jks-password` | Password for the JKS truststore and keystore. Only used when cert-type is `jks`. | `password` | | `credentials-operator.otterize.com/restart-pod-on-certificate-renewal` | A pod with this annotation (no matter the value) will be restarted after certificate renewal, along with any replicas. Should be ideally set through the pod owner's `template` spec so it will persist between restarts. | N/A | | `intents.otterize.com/service-name` | Used for [service identity resolution](/reference/service-identities#kubernetes-service-identity-resolution).| | \ No newline at end of file diff --git a/static/code-examples/ibac-for-kafka/interactive/all.yaml b/static/code-examples/ibac-for-kafka/interactive/all.yaml index ba36cd2e8..c8222cbe9 100644 --- a/static/code-examples/ibac-for-kafka/interactive/all.yaml +++ b/static/code-examples/ibac-for-kafka/interactive/all.yaml @@ -52,4 +52,9 @@ spec: name: interactive-properties - name: otterize-credentials secret: - secretName: interactive-credentials-secret \ No newline at end of file + secretName: interactive-credentials-secret + items: + - key: truststore.jks + path: kafka.truststore.jks + - key: keystore.jks + path: kafka.keystore.jks \ No newline at end of file diff --git a/static/code-examples/kafka-mtls/all.yaml b/static/code-examples/kafka-mtls/all.yaml index 52ccc83ad..ae0eaf15a 100644 --- a/static/code-examples/kafka-mtls/all.yaml +++ b/static/code-examples/kafka-mtls/all.yaml @@ -31,6 +31,13 @@ spec: - name: otterize-credentials secret: secretName: client-credentials-secret + items: + - key: tls.crt + path: cert.pem + - key: tls.key + path: key.pem + - key: ca.crt + path: ca.pem - name: ephemeral emptyDir: { } --- @@ -62,5 +69,10 @@ spec: - name: otterize-credentials secret: secretName: client-other-credentials-secret + items: + - key: tls.crt + path: cert.pem + - key: tls.key + path: key.pem - name: ephemeral emptyDir: { } diff --git a/static/code-examples/kafka-mtls/helm/values.yaml b/static/code-examples/kafka-mtls/helm/values.yaml index 5b39621dd..5f019007c 100644 --- a/static/code-examples/kafka-mtls/helm/values.yaml +++ b/static/code-examples/kafka-mtls/helm/values.yaml @@ -26,6 +26,8 @@ auth: existingSecrets: - kafka-tls-secret password: password + jksTruststore: truststore.jks + jksKeystoreSAN: keystore.jks authorizerClassName: kafka.security.authorizer.AclAuthorizer # Allocate resources resources: From 3d4c605da6fb0a139504cb5ffd7ef9d6d54658d0 Mon Sep 17 00:00:00 2001 From: Ori Shoshan Date: Wed, 13 Sep 2023 21:39:00 +0300 Subject: [PATCH 5/7] fixups --- .../credentials-operator/README.mdx | 28 ++++++++----------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/docs/reference/configuration/credentials-operator/README.mdx b/docs/reference/configuration/credentials-operator/README.mdx index 2c612c15f..888f30c36 100644 --- a/docs/reference/configuration/credentials-operator/README.mdx +++ b/docs/reference/configuration/credentials-operator/README.mdx @@ -3,7 +3,7 @@ sidebar_position: 3 title: Credentials operator --- -The Otterize credentials operator automatically resolves pods to dev-friendly service names, registers them with a SPIRE server or with Otterize Cloud, and provisions credentials as Kubernetes Secrets. +The Otterize credentials operator automatically resolves pods to dev-friendly service names and provisions credentials (certificates) for the services from cert-manager, Otterize Cloud or SPIRE as Kubernetes Secrets. ## Deploying the credentials operator To deploy the operator, [use the Helm chart](/reference/configuration/credentials-operator/helm-chart). @@ -14,18 +14,20 @@ To deploy with cert-manager as the certificate provider, you must also [configur ## Acquiring mTLS credentials using the credentials operator The credentials operator is controlled using annotations placed on pods. To have it provision credentials and place them in Secrets, you must specify `credentials-operator.otterize.com/tls-secret-name`. -## How does the credentials operator work? +## How does the credentials operator provision credentials? -### 1. SPIRE entry registration -This step only happens if the operator is configured to use SPIRE for certificate generation. Once the operator [resolves the service name](#service-name-resolution-and-automatic-pod-labeling) for a pod, it labels the pod so that SPIRE can find it, registers an entry with the SPIRE server for that label. +The credentials operator performs two steps in order to issue certificates. -### 2. Certificate generation -The operator consults the annotation `credentials-operator.otterize.com/tls-secret-name`. If that annotation exists, the operator creates a secret named after the value of the label with X.509 credentials within, provided by cert-manager, Otterize Cloud or SPIRE, depending on how the credentials operator is configured. +### Step 1: SPIRE entry registration +This step only happens if the operator is configured to use SPIRE for certificate generation. Once the operator [resolves the service name](#service-name-resolution-and-automatic-pod-labeling) for a pod, it labels the pod so that SPIRE can find it, and registers an entry with the SPIRE server for that label. + +### Step 2: Certificate generation +The operator consults the annotation `credentials-operator.otterize.com/tls-secret-name`. If that annotation exists, the operator creates a secret named after the value of the label. That secret contains X.509 credentials within, provided by cert-manager, Otterize Cloud or SPIRE, depending on how the credentials operator is configured. #### cert-manager -The operator creates a cert-manager [`Certificate`](https://cert-manager.io/docs/usage/certificate/) resource, which will create a Kubernetes Secret with the name specified by the value of the annotation `credentials-operator.otterize.com/tls-secret-name`. The common name and DNS names in the certificate is the identity of the service, as resolved by the [service identity resolution algorithm](#3-service-name-resolution-and-automatic-pod-labeling), i.e. `servicename.namespace`. +The operator creates a cert-manager [`Certificate`](https://cert-manager.io/docs/usage/certificate/) resource, which will create a Kubernetes Secret with the name specified by the value of the annotation `credentials-operator.otterize.com/tls-secret-name`. The common name and DNS names in the certificate are values that represent the identity of the service, as resolved by the [service identity resolution algorithm](#3-service-name-resolution-and-automatic-pod-labeling), i.e. `servicename.namespace`. -The operator will use a ClusterIssuer or an Issuer to create the Certificate resource, which it expects to find in the same namespace as the pods. The Issuer is configured at deploy time, using the [Helm chart](/reference/configuration/credentials-operator/helm-chart). +The operator will use a [`ClusterIssuer`](https://cert-manager.io/docs/concepts/issuer/) or an [`Issuer`](https://cert-manager.io/docs/concepts/issuer/) to create the Certificate resource, which it expects to find in the same namespace as the `Pod` with the annotation. The `Issuer` is configured at deploy time, using the [Helm chart](/reference/configuration/credentials-operator/helm-chart). In the event that the default approver controller in `cert-manager` is [disabled](https://cert-manager.io/docs/concepts/certificaterequest/#approver-controller), the credentials operator will auto-approve its own [`CertificateRequests`](https://cert-manager.io/docs/concepts/certificaterequest/) @@ -35,19 +37,13 @@ The operator requests certificates from Otterize Cloud, which internally manages #### SPIRE Once the operator has registered the pod with SPIRE, which happens automatically for a pod that has the `credentials-operator.otterize.com/tls-secret-name` annotation upon pod startup. The credentials operator then acquires the SVID and certificates for the CA chain and places them within a Kubernetes Secret. The SVID and DNS names in the certificate is the identity of the service, as resolved by the [service identity resolution algorithm](#3-service-name-resolution-and-automatic-pod-labeling), i.e. `servicename.namespace`. -### 3. Service name resolution and automatic pod labeling -The operator performs service name resolution for each pod, through the same algorithm used by the [intents operator](/reference/configuration/intents-operator) and [network mapper](/reference/configuration/network-mapper). [Learn more on how the resolution happens](/reference/service-identities#kubernetes-service-identity-resolution). -The value resulting from this resolution process is the service name, which is then used to create credentials for the service, in the format ``. For example: `payments-svc.staging`. - - - ## SPIRE workload registrar -When using SPIRE, the operator registers every pod (even those without annotations). +When using SPIRE, the operator registers every pod with the SPIRE server (even those without annotations). Alongside the credentials operator, you could use SPIRE agents and the SPIRE SDK to work with the same SPIRE server. To learn more, check out the documentation for [SPIRE](https://spiffe.io/docs/latest/spire-about/spire-concepts/). Note that to use the credentials operator, you do not need to work directly with SPIRE or SPIRE agents, and can do everything completely using annotations and Kubernetes Secrets. -### Pod annotations +## Pod annotations | Annotation | Description | Default | |---------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------| From 8b4f5143e0813390f85f1025c85e8e58b8f2e1d0 Mon Sep 17 00:00:00 2001 From: Ori Shoshan Date: Thu, 14 Sep 2023 22:15:26 +0300 Subject: [PATCH 6/7] Fix tutorial --- static/code-examples/kafka-mtls/all.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/static/code-examples/kafka-mtls/all.yaml b/static/code-examples/kafka-mtls/all.yaml index ae0eaf15a..3b9254785 100644 --- a/static/code-examples/kafka-mtls/all.yaml +++ b/static/code-examples/kafka-mtls/all.yaml @@ -74,5 +74,7 @@ spec: path: cert.pem - key: tls.key path: key.pem + - key: ca.crt + path: ca.pem - name: ephemeral emptyDir: { } From 4a61e9d60449002cbe36254d899e5996ead767da Mon Sep 17 00:00:00 2001 From: Ori Shoshan Date: Thu, 14 Sep 2023 22:24:30 +0300 Subject: [PATCH 7/7] Fixes --- docs/quick-tutorials/k8s-kafka-mtls.mdx | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/quick-tutorials/k8s-kafka-mtls.mdx b/docs/quick-tutorials/k8s-kafka-mtls.mdx index 30096bd37..fcfd62165 100644 --- a/docs/quick-tutorials/k8s-kafka-mtls.mdx +++ b/docs/quick-tutorials/k8s-kafka-mtls.mdx @@ -56,7 +56,6 @@ Of course you can also choose to combine them — after all, Kafka is just a #### Install Otterize OSS, connected to Otterize Cloud {@include: ../_common/install-otterize-from-cloud-with-enforcement-and-kafka-watcher.md} -Make sure you also include the **`--set intentsOperator.operator.enableNetworkPolicyCreation=false`** flag. as well. ## Install Kafka @@ -129,8 +128,9 @@ Our simple example consists of two client pods: - And one named "**client-other**". These clients are connecting to Kafka using mTLS, the credentials which they will receive from Otterize. Otterize makes this easy, requiring just 4 simple changes: +1. **Generate credentials**: add the `credentials-operator.otterize.com/tls-secret-name` annotation, which tells Otterize to generate mTLS credentials and store them in a Kubernetes secret whose name is the value of this annotation. +2. **Expose credentials in a volume**: add a volume containing this secret to the pod. 3. **Mount the volume**: mount the volume in every container in the pod. -4. **Enable mTLS**: configure the Kafka client library to require authentication.
Expand to see this structure @@ -319,8 +319,6 @@ We can see what happened: 2. Calls from **[client-other]** are not declared (orange line). 3. Looking at the Kafka service, we can see that **[client]** has specific access configured (via Kafka ACLs) to perform `all` operations on the `mytopic` topic. -Since discovered intents from the network mapper don't specify what specific topics and operations clients are performing (or attempting to perform), the access graph cannot show information on what is being blocked vs allowed (red vs green) on a per-topic basis. That feature is in development. - Also, the access graph shows information about the mTLS certificates (credentials) distributed to the various services, as long as [Cloud-managed credentials](/security#cryptographic-credentials) are being used. Visibility for certificates distributed through an in-cluster SPIRE is in development. ## Turn on protection @@ -330,7 +328,7 @@ At this point, we haven't actually protected our Kafka broker. From everything w Let's see that in action. Our clients that have not declared intents will be blocked from accessing the broker. -We need to turn enforcement on in our cluster by updating our Otterize helm configuration. +We need to turn enforcement on in our cluster by updating our Otterize Helm configuration. ```bash helm upgrade --install otterize otterize/otterize-kubernetes -n otterize-system --create-namespace \ @@ -338,6 +336,7 @@ helm upgrade --install otterize otterize/otterize-kubernetes -n otterize-system --set global.otterizeCloud.credentials.clientSecret= \ --set intentsOperator.operator.mode=defaultActive \ --set global.otterizeCloud.useCloudToGenerateTLSCredentials=true \ + --set intentsOperator.operator.enableNetworkPolicyCreation=false --set networkMapper.kafkawatcher.enable=true \ --set networkMapper.kafkawatcher.kafkaServers={"kafka-0.kafka"} ```