Custom api-server Certificates

[sempex]

Hi everyone,

We have our own Certificate Authority (CA), and I would like to use a custom certificate for the API server created by that CA. This will ensure seamless integration between all surrounding systems and Kubernetes.

Does anyone have any concepts or guidance on how to achieve this? I couldn't find any relevant information in the Talos documentation.

Thank you!

[smira]

I'm not quite sure what you're looking for exactly. The kube-apiserver is both verifying and issuing certificates, so it needs a CA. That CA by default in Talos is generated as a self-signed one, but you could try to replace it with a CA which is signed by your root CA.

If you're looking just to use client kubeconfig signed by your internal CA, you can simply add its certificate to the list of accepted CAs.

[tux-box]

How do I get that CA so I can import it into my local trusted CA store?
What format is it in?
Thanks.

[smira]

e.g.

# talosctl -n 172.20.0.2 get kubernetesrootsecrets -o yaml | yq .spec.issuingCA
crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJpRENDQVMrZ0F3SUJBZ0lRR1dPWjhXZ3EvbWo2WGxkdFFzRHhiekFLQmdncWhrak9QUVFEQWpBVk1STXcKRVFZRFZRUUtFd3ByZFdKbGNtNWxkR1Z6TUI0WERUSTBNRFl5T0RFM01UTXhORm9YRFRNME1EWXlOakUzTVRNeApORm93RlRFVE1CRUdBMVVFQ2hNS2EzVmlaWEp1WlhSbGN6QlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VICkEwSUFCS0M4ZTZUTDV5bGw1UjBGVnJVNktsRjBIQ09POTYyekQrZEJNQW4rNlRqUXQyWDlJaWFScW1RVmVSSlMKZkN6dURiQ0cvUG1WVit6anlHam5Da3lzT0RxallUQmZNQTRHQTFVZER3RUIvd1FFQXdJQ2hEQWRCZ05WSFNVRQpGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFCkZnUVVNODNkYVJvbTd1QWNyWCtMeWhhN0lBNjN6LzR3Q2dZSUtvWkl6ajBFQXdJRFJ3QXdSQUlnYTZzclRmODkKekRjTmFZVzJ3SnI3YXRaa1JDeGVmU1I4L0hQNVlXbjBhRWdDSUh2S2cwaFNGc2J2QkJJYUx2VGZwd2ovYWZHRgpXWVRtTFJ2WExhUmNmNHRGCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
key: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUlrKzV3blhYeWUvQ2IwWmJCTWl3am9TYk5xZ0U1Sk16R2M3MVczRjM3WmFvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFb0x4N3BNdm5LV1hsSFFWV3RUb3FVWFFjSTQ3M3JiTVA1MEV3Q2Y3cE9OQzNaZjBpSnBHcQpaQlY1RWxKOExPNE5zSWI4K1pWWDdPUElhT2NLVEt3NE9nPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=

that gives your Kubernetes API server issuing CA certificate and private key (this is sensitive, I'm sharing output here as it's a single-use dev cluster).

These are base64-encoded PEM-encoded files, so you can e.g. use openssl to decode them fully:

# talosctl -n 172.20.0.2 get kubernetesrootsecrets -o yaml | yq .spec.issuingCA.crt | base64 -d | openssl x509 -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            19:63:99:f1:68:2a:fe:68:fa:5e:57:6d:42:c0:f1:6f
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: O = kubernetes
        Validity
            Not Before: Jun 28 17:13:14 2024 GMT
            Not After : Jun 26 17:13:14 2034 GMT
        Subject: O = kubernetes
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:a0:bc:7b:a4:cb:e7:29:65:e5:1d:05:56:b5:3a:
                    2a:51:74:1c:23:8e:f7:ad:b3:0f:e7:41:30:09:fe:
                    e9:38:d0:b7:65:fd:22:26:91:aa:64:15:79:12:52:
                    7c:2c:ee:0d:b0:86:fc:f9:95:57:ec:e3:c8:68:e7:
                    0a:4c:ac:38:3a
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Subject Key Identifier: 
                33:CD:DD:69:1A:26:EE:E0:1C:AD:7F:8B:CA:16:BB:20:0E:B7:CF:FE
    Signature Algorithm: ecdsa-with-SHA256
    Signature Value:
        30:44:02:20:6b:ab:2b:4d:ff:3d:cc:37:0d:69:85:b6:c0:9a:
        fb:6a:d6:64:44:2c:5e:7d:24:7c:fc:73:f9:61:69:f4:68:48:
        02:20:7b:ca:83:48:52:16:c6:ef:04:12:1a:2e:f4:df:a7:08:
        ff:69:f1:85:59:84:e6:2d:1b:d7:2d:a4:5c:7f:8b:45

[Clockwork-Muse]

Yeah, but if you're spinning up clusters with something like matchbox, you shouldn't be putting private key info inside the ignition file.

I think one of the things people will be looking for here is integration with tools like EJBCA or Hashicorp's Vault, where certificates are autogenerated (and often quite short lived). However, that represents a chicken-and-egg problem - namely, how do you securely register the cluster to those tools, and how do you verify that the Talos cluster you're talking to is the correct one.

For me, what I want is for the management API to use a certificate that is signed by my CA, and not have a cluster-specific root CA, and additionally to not ever expose any private keys (especially for any intermediate CA - which may not be available if it uses a TPM). I'm spinning up a bare-metal cluster that I have console/display access to, so I have some additional opportunities for interaction that may not be available in all cases....

What I would want is the ability to tell the cluster "When you spin up, generate the private keys and CSRs, then talk to this remote endpoint (with this known root CA cert) to get the CSRs signed".
(Of course, the only way currently I can validate the CSR matches the cluster is to look at a monitor, but still....)

[veshant]

I am trying to use the cluster acceptedCAs to accomplish this with no success. It seems like the rotate CA uses it so it should work

https://www.talos.dev/v1.8/advanced/ca-rotation/#manual-steps-for-kubernetes-api-ca-rotation