Use mTLS instead of Kerberos for KRaft inter-node traffic #1066

Description

With Kerberos enabled, the CONTROLLER listener currently uses SASL_SSL/GSSAPI. Brokers and controllers authenticate to each other with Kerberos principals (#1024). The INTERNAL listener already uses mTLS with the pod certificates of the internalSecretClass in every configuration.

Kerberos principals are a poor fit for identifying the Kafka nodes. With the default sasl.kerberos.principal.to.local.rules, every node principal maps to the same short name, so it can't be pinned to a specific node. Pod certificate subjects can.

Proposal:

• Make the CONTROLLER listener always SSL, and require client authentication when Kerberos, TLS client authentication or OPA is configured.
• Use Kerberos only on the client-facing broker listeners. Controllers get no keytab or JAAS configuration.
• Make super.users contain only the pod certificate subjects of the nodes.

Issue checklist

This is a simple checklist of things to bear in mind when creating a new issue.

• Describe the use-case: As far as possible, use the pattern "As a [type of user], I would like [feature/functionality] to be able to do [specific action]." This helps identify the feature and the problem it addresses.
• Indicate importance and urgency: Use a scale (e.g., low, medium, high) to indicate the level of importance and urgency.
• Work-around: If there is a known work-around, describe it briefly.
• Environment: Describe the environment where the issue occurs (e.g., SDP version, K8S version, etc.).