Failed to call webhook: Post "https://secrets-injector.default.svc:443/inject?timeout=10s": no endpoints available for service

[gel-hidden]

Your environment

Secret Injector Version: 1.1.0
Kubernetes Version: Client Version: v1.31.2, Kustomize Version: v5.4.2, Server Version: v1.30.9-gke.1009000

What happened?

I used the 1password/secrets-injector helm chart, and everything worked like a charm. But a few hours later I saw this log:

ERROR 2025-02-12T01:22:31.531145922Z [resource.labels.containerName: secrets-injector] Failed to listen and serve webhook server: http: Server closed

And the deployment has no pods, and it seems to fail to create the replica set:

9m25s       Warning   FailedCreate        replicaset/secrets-injector-59bcf97785     Error creating: Internal error occurred: failed calling webhook "secrets-injector.1password.com": failed to call webhook: Post "https://secrets-injector.default.svc:443/inject?timeout=10s": no endpoints available for service "secrets-injector"

Is it some strange circular dependency? Because the service that should serve the webhook is part of the deployment that cannot call the webhooK?

[gel-hidden]

#32 (comment)

Doing this, ie deleting the webhook, got everything to work again.

[gel-hidden]

The strange part is that when I manually removed (killed) the pod, it was unable to restart. It seems that the issue arises when the injector is in a namespace labeled with secrets-injection=enabled, causing it to become dependent on itself. However, moving it to a different namespace without this label resolves the problem.