Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated Automatically Rotating Certs Docs #1757

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ shared across clusters, and an *issuer certificate*, which is specific to the
cluster.

While Linkerd automatically rotates the per-proxy TLS certificates, it does not
rotate the issuer certificate. In this doc, we'll describe how to set up
automatic rotation of the issuer certificate and its corresponding private key
using the cert-manager project.
rotate the issuer certificate. Linkerd's out-of-the-box installations generate
static self-signed certificates with a validity of one year but require manual
rotation by the user to prevent expiry. While this setup is convenient for quick
start testing, it's not advisable nor recommended for production environments.

{{< trylpt >}}

Expand All @@ -24,17 +25,23 @@ for making TLS credentials from external sources available to Kubernetes
clusters.

Cert-manager is very flexible. You can configure it to pull certificates from
secrets managemenet solutions such as [Vault](https://www.vaultproject.io). In
secrets managemenet solutions such as [Vault](https://www.vaultproject.io). In
this guide, we'll focus on a self-sufficient setup: we will configure
cert-manager to act as an on-cluster
[CA](https://en.wikipedia.org/wiki/Certificate_authority) and have it re-issue
Linkerd's issuer certificate and private key on a periodic basis, derived from
the trust anchor.
the trust anchor. Additionally, we will use trust-manager create a trust bundle
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"to create"

which will allow Linkerd to verify the authenticity of certificates issued by
cert-manager.

### Cert manager as an on-cluster CA

As a first step, [install cert-manager on your
cluster](https://cert-manager.io/docs/installation/).
As a first step,
[install cert-manager on your cluster](https://cert-manager.io/docs/installation/),
then
[install trust-manager](https://cert-manager.io/docs/trust/trust-manager/installation/)
and configure it to use "linkerd" as the
[trust namespace](https://cert-manager.io/docs/trust/trust-manager/installation/#trust-namespace).

Next, create the namespace that cert-manager will use to store its
Linkerd-related resources. For simplicity, we suggest reusing the default
Expand All @@ -44,46 +51,111 @@ Linkerd control plane namespace:
kubectl create namespace linkerd
```

#### Save the signing key pair as a Secret
#### Give cert-manager necessary RBAC permissions

Next, using the [`step`](https://smallstep.com/cli/) tool, create a signing key
pair and store it in a Kubernetes Secret in the namespace created above:
By default cert-manager will only create certificate secrets in the namespace
where it is installed. Linkerd, however, requires the cert secrets to be created
in the linkerd namespace. To do this we will create a `ServiceAccount` for
cert-manager in the linkerd namespace with the required permissions.

```bash
step certificate create root.linkerd.cluster.local ca.crt ca.key \
--profile root-ca --no-password --insecure &&
kubectl create secret tls \
linkerd-trust-anchor \
--cert=ca.crt \
--key=ca.key \
--namespace=linkerd
kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
name: cert-manager
namespace: linkerd
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: cert-manager-secret-creator
namespace: linkerd
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "get", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cert-manager-secret-creator-binding
namespace: linkerd
subjects:
- kind: ServiceAccount
name: cert-manager
namespace: linkerd
roleRef:
kind: Role
name: cert-manager-secret-creator
apiGroup: rbac.authorization.k8s.io
EOF
```

For a longer-lived trust anchor certificate, pass the `--not-after` argument
to the step command with the desired value (e.g. `--not-after=87600h`).
#### Create the trust root ClusterIssuer

To begin, create a self-signing `ClusterIssuer` for the Linkerd trust root
certificate.

```bash
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: linkerd-trust-root-issuer
spec:
selfSigned: {}
EOF
```

#### Create an Issuer referencing the secret
#### Create a trust root certificate

With the Secret in place, we can create a cert-manager "Issuer" resource that
references it:
Now create a cert-manager `Certificate` resource which uses the
previously-created `ClusterIssuer`:

```bash
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
kind: Certificate
metadata:
name: linkerd-trust-anchor
namespace: linkerd
spec:
commonName: root.linkerd.cluster.local
isCA: true
duration: 87600h0m0s
renewBefore: 87264h0m0s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh. I had thought this was "renew, at latest, when this many hours are remaining"?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're correct. updating to clarify.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just double checked their docs: https://cert-manager.io/docs/usage/certificate/#reissuance-triggered-by-expiry-renewal

It looks like the original blurb we had on the site originally (where I copied this blurb from) is accurate: "If spec.renewBefore has been set, it will be spec.renewBefore amount of time before expiry. cert-manager will set Certificate's status.RenewalTime to the time when the renewal will be attempted."

issuerRef:
name: linkerd-trust-root-issuer
kind: ClusterIssuer
privateKey:
algorithm: ECDSA
secretName: linkerd-trust-anchor
EOF
```

#### Create Linkerd identity issuer

Using the previously-generated trust root certificate, create a Linkerd identity
`Issuer`:

```bash
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: linkerd-identity-issuer
namespace: linkerd
spec:
ca:
secretName: linkerd-trust-anchor
EOF
```

#### Create a Certificate resource referencing the Issuer
#### Create a Certificate resource referencing the issuer

Finally, we can create a cert-manager "Certificate" resource which uses this
Issuer to generate the desired certificate:
Next, create a Linkerd identity issuer certificate which will act as an
intermediary signing CA for all Linkerd mTLS proxy certificates:

```bash
kubectl apply -f - <<EOF
Expand All @@ -93,67 +165,131 @@ metadata:
name: linkerd-identity-issuer
namespace: linkerd
spec:
secretName: linkerd-identity-issuer
duration: 48h
renewBefore: 25h
issuerRef:
name: linkerd-trust-anchor
kind: Issuer
commonName: identity.linkerd.cluster.local
dnsNames:
- identity.linkerd.cluster.local
isCA: true
privateKey:
algorithm: ECDSA
usages:
- cert sign
- crl sign
- server auth
- client auth
- cert sign
- crl sign
- server auth
- client auth
duration: 28h0m0s
renewBefore: 25h0m0s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above re renewBefore, but also, duration should be 48 hours, correct?

issuerRef:
name: linkerd-identity-issuer
kind: Issuer
privateKey:
algorithm: ECDSA
secretName: linkerd-identity-issuer
EOF
```

(In the YAML manifest above, the `duration` key instructs cert-manager to
consider certificates as valid for `48` hours and the `renewBefore` key indicates
that cert-manager will attempt to issue a new certificate `25` hours before
expiration of the current one. These values can be customized to your liking.)
consider certificates as valid for `48` hours and the `renewBefore` key
indicates that cert-manager will attempt to issue a new certificate `25` hours
before expiration of the current one. These values can be customized to your
liking.)

#### Create Linkerd trust bundle

Lastly, we will also need to create a trust bundle which will allow Linkerd's
identity controller to verify the authenticity of certificates issued by
cert-manager:

```bash
kubectl apply -f - <<EOF
apiVersion: trust.cert-manager.io/v1alpha1
kind: Bundle
metadata:
name: linkerd-identity-trust-roots
namespace: linkerd
spec:
sources:
- secret:
name: "linkerd-trust-anchor"
key: "ca.crt"
target:
configMap:
key: "ca-bundle.crt"
EOF
```

## Summary & Validation

Below are the resources created for managing Linkerd identity certificates with
cert-manager:

At this point, cert-manager can now use this Certificate resource to obtain TLS
credentials, which will be stored in a secret named `linkerd-identity-issuer`.
To validate your newly-issued certificate, you can run:
- Namespace: `linkerd` (to store certificates and secrets)
- RBAC Permissions: ServiceAccount, Role, and RoleBinding in the `linkerd`
namespace for cert-manager
- ClusterIssuer: `linkerd-trust-root-issuer` (self-signed ClusterIssuer)
- Certificate: `linkerd-trust-anchor` (in the `linkerd` namespace, referencing
`linkerd-trust-root-issuer`)
- Issuer: `linkerd-identity-issuer` (to manage Linkerd identity certificates)
- Certificate: `linkerd-identity-issuer` (in the `linkerd` namespace, acting as
an intermediary signing CA)
- Trust Bundle: `linkerd-identity-trust-roots` (to allow Linkerd's identity
controller to verify certificate authenticity)

To validate creation and status, run the following commands:

```bash
kubectl get secret linkerd-identity-issuer -o yaml -n linkerd
# Check namespace creation
kubectl get namespaces linkerd

# Check RBAC permissions
kubectl get serviceaccount,role,rolebinding -n linkerd

# Check ClusterIssuer creation
kubectl get clusterissuers linkerd-trust-root-issuer

# Check Certificate creation
kubectl get certificates -n linkerd

# Check Issuer creation
kubectl get issuers.cert-manager.io -n linkerd

# Check Trust Bundle creation
kubectl get bundles -n linkerd
```

Now we just need to inform Linkerd to consume these credentials.
## Consuming cert-manager identity certificates

To have Linkerd consume cert-manager created certificates you will need to add
the following to your values file or pass them in as flags at runtime.

| Field | Value |
| ------------------------ | ----------------- |
| `identity.externalCA` | true |
| `identity.issuer.scheme` | kubernetes.io/tls |

## Using these credentials with CLI installation
### Using these credentials with CLI installation

For CLI installation, the Linkerd control plane should be installed with the
`--identity-external-issuer` flag, which instructs Linkerd to read certificates
from the `linkerd-identity-issuer` secret. Whenever certificate and key stored
in the secret are updated, the `identity` service will automatically detect
this change and reload the new credentials.
in the secret are updated, the `identity` service will automatically detect this
change and reload the new credentials.

Voila! We have set up automatic rotation of Linkerd's control plane TLS
credentials.

## Using these credentials with a Helm installation
### Using these credentials with a Helm installation

For installing with Helm, first install the `linkerd-crds` chart:

```bash
helm install linkerd-crds -n linkerd --create-namespace linkerd/linkerd-crds
```

Then install the `linkerd-control-plane` chart, setting the
`identityTrustAnchorsPEM` to the value of `ca.crt` in the
`linkerd-identity-issuer` Secret:
Then install the `linkerd-control-plane` chart:

```bash
helm install linkerd-control-plane -n linkerd \
--set-file identityTrustAnchorsPEM=ca.crt \
--set identity.externalCA=true \
--set identity.issuer.scheme=kubernetes.io/tls \
linkerd/linkerd-control-plane
```
Expand Down Expand Up @@ -199,5 +335,5 @@ following instructions to install and configure Linkerd to use it.

## See also

* [Automatically Rotating Webhook TLS Credentials](../automatically-rotating-webhook-tls-credentials/)
* [Manually rotating Linkerd's trust anchor credentials](../manually-rotating-control-plane-tls-credentials/)
- [Automatically Rotating Webhook TLS Credentials](../automatically-rotating-webhook-tls-credentials/)
- [Manually rotating Linkerd's trust anchor credentials](../manually-rotating-control-plane-tls-credentials/)
Loading