Option for Kafkatopic Resource to not delete the actual data if resource is delete

[chilicat]

Hi All,

I was wondering if it is possible to use KafkaTopic resources to create/configure topics but leave the data intact when deleted.

We require that because we see the deployment of Kubernetes resources as ephemeral, and it is not uncommon that our customers to delete and reinstall our Helm charts (which contain the KafkaTopic resources) with the expectation that the data still exists.

[chilicat]

I found this unidirectional topic operator page: https://strimzi.io/blog/2023/11/02/unidirectional-topic-operator/

Would I get the expected result by disabling the finializer?

 topicOperatorContainer:
        env:
          - name: STRIMZI_USE_FINALIZERS
            value: "false"

The documentation states: In development and test environments, you can disable the use of finalizers
Apart from the data is not deleted, would I expect other side effects by disabling the finalizer?

[scholzj]

No. the finalizer makes sure the data are deleted when the KafkaTopic is deleted. But disabling them just means that the data might not get deleted in case of some race conditions, outages etc.

If you want to delete the KafkaTopic resource without deleting the data, you should follow this to "unmanage" the topic first before deleting it: https://strimzi.io/docs/operators/latest/full/deploying.html#proc-converting-managed-topics-str

[chilicat]

Thank you Jakub for the fast answer.

I guess I could implement in our tooling to set the unmanaged annotation before the cluster cleanup. Of course, I still have a lot of edge cases where our customer can lose data.

How does the community feel about a new feature in the KafkaTopic spec where you can indicate that the data should not be deleted?

[scholzj]

TBH, I do not really follow your use-case here. So not sure how easy or hard that would be to do. I guess as an alternative you can also disable the topic deletion in Kafka and then nobody would be able to delete any topics. That might be easier, but might have some side effects.