fix(kubewarden-defaults): set failurePolicy: Ignore if mode == "monitor"

[Malet]

Description

Currently the default set of policies are applied in monitor mode, but cause all pods to be rejected if the kubewarden service is down. This should not be the case if the user's intention is to monitor instead of protect.

Test

I've added a couple of tests to a new suite for kubewarden-defaults to check this behaves as expected

Additional Information

Tradeoff

N/A

Potential improvement

Prevent catastrophic failure when kubewarden service is unavailable if in monitor mode

[viccuad]

Hi, many thanks for this contribution and providing a test case!
I personally like this, yet I look forward to the other reviewers opinion because we may want to keep it false as default since this case is not usual.

Out of curiosity, could you expand on the cases where Kubewarden is down? The PolicyServers rollouts should provide with availability in any case. We can schedule follow-up work if needed.

[flavio]

That sounds good to me. Thanks for having extended the test suite too!

[Malet]

Out of curiosity, could you expand on the cases where Kubewarden is down? The PolicyServers rollouts should provide with availability in any case. We can schedule follow-up work if needed.

Our particular issue seems to have been caused by a deadlock between the HNC webhook and kubewarden when a nodepool was updated. I'm sure there are plenty of other scenarios where it is not available/schedulable (resource constraints etc.).