Flaky test: TestKongPluginBindingManaged

[pmalek]

Problem statement

TestKongPluginBindingManaged is flaky.

https://app.buildpulse.io/@Kong/gateway-operator?q=TestKongPluginBindingManaged&select%5B%5D=&show_type=default&sort=disruptiveness&daterange=October+7+-+October+21&starting_at=2024-10-07+09%3A56%3A32+UTC&ending_at=2024-10-21+09%3A56%3A32+UTC

https://github.com/Kong/gateway-operator/actions/runs/11437427757/job/31816903033

    kongpluginbinding_managed_test.go:442: deployed new KongService test-vfv69/kongservice-a4b8e2dd resource
    kongpluginbinding_managed_test.go:449: deployed new KongRoute test-vfv69/kongroute-00fceb73 resource
    kongpluginbinding_managed_test.go:456: deployed new KongConsumer test-vfv69/consumer-stzsh resource
    kongpluginbinding_managed_test.go:464: waiting for 2 KongPluginBindings to be created
    kongpluginbinding_managed_test.go:491: checking that managed KongPlugin test-vfv69/rate-limiting-kp-nlz49 gets plugin-in-use finalizer added
    kongpluginbinding_managed_test.go:506: delete managed kongPluginBinding test-vfv69/rate-limiting-kp-nlz49-bmsrn, then check it gets recreated
    kongpluginbinding_managed_test.go:506: delete managed kongPluginBinding test-vfv69/rate-limiting-kp-nlz49-shwzh, then check it gets recreated
    kongpluginbinding_managed_test.go:511: 
        	Error Trace:	/home/runner/work/gateway-operator/gateway-operator/test/envtest/watch.go:68
        	            				/home/runner/work/gateway-operator/gateway-operator/test/envtest/kongpluginbinding_managed_test.go:511
        	Error:      	2 KongPluginBindings were not recreated
        	Test:       	TestKongPluginBindingManaged/binding_to_KongConsumer,_KongService_and_KongRoute
        	Messages:   	Last object received:
        	            	&v1alpha1.KongPluginBinding{
        	            	    TypeMeta:   v1.TypeMeta{},
        	            	    ObjectMeta: v1.ObjectMeta{
        	            	        Name:                       "rate-limiting-kp-nlz49-qbmrp",
        	            	        GenerateName:               "rate-limiting-kp-nlz49-",
        	            	        Namespace:                  "test-vfv69",
        	            	        SelfLink:                   "",
        	            	        UID:                        "39d8e158-11d0-4c08-aacc-d6410df8c406",
        	            	        ResourceVersion:            "402",
        	            	        Generation:                 1,
        	            	        CreationTimestamp:          time.Date(2024, time.October, 21, 9, 43, 47, 0, time.Local),
        	            	        DeletionTimestamp:          (*v1.Time)(nil),
        	            	        DeletionGracePeriodSeconds: (*int64)(nil),
        	            	        Labels:                     {},
        	            	        Annotations:                {},
        	            	        OwnerReferences:            {
        	            	            {
        	            	                APIVersion:         "configuration.konghq.com/v1",
        	            	                Kind:               "KongPlugin",
        	            	                Name:               "rate-limiting-kp-nlz49",
        	            	                UID:                "899ddef2-45ac-47d0-aea3-ecfc6ee2deec",
        	            	                Controller:         (*bool)(nil),
        	            	                BlockOwnerDeletion: &bool(true),
        	            	            },
        	            	        },
        	            	        Finalizers:    nil,
        	            	        ManagedFields: {
        	            	            {
        	            	                Manager:    "envtest.test",
        	            	                Operation:  "Update",
        	            	                APIVersion: "configuration.konghq.com/v1alpha1",
        	            	                Time:       time.Date(2024, time.October, 21, 9, 43, 47, 0, time.Local),
        	            	                FieldsType: "FieldsV1",
        	            	                FieldsV1:   &v1.FieldsV1{
        	            	                    Raw: {0x7b, 0x22, 0x66, 0x3a, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x22, 0x3a, 0x7b, 0x22, 0x66, 0x3a, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6f, 0x77, 0x6e, 0x65, 0x72, 0x52, 0x65, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x73, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x6b, 0x3a, 0x7b, 0x5c, 0x22, 0x75, 0x69, 0x64, 0x5c, 0x22, 0x3a, 0x5c, 0x22, 0x38, 0x39, 0x39, 0x64, 0x64, 0x65, 0x66, 0x32, 0x2d, 0x34, 0x35, 0x61, 0x63, 0x2d, 0x34, 0x37, 0x64, 0x30, 0x2d, 0x61, 0x65, 0x61, 0x33, 0x2d, 0x65, 0x63, 0x66, 0x63, 0x36, 0x65, 0x65, 0x32, 0x64, 0x65, 0x65, 0x63, 0x5c, 0x22, 0x7d, 0x22, 0x3a, 0x7b, 0x7d, 0x7d, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x73, 0x70, 0x65, 0x63, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x63, 0x6f, 0x6e, 0x74, 0x72, 0x6f, 0x6c, 0x50, 0x6c, 0x61, 0x6e, 0x65, 0x52, 0x65, 0x66, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6b, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63, 0x65, 0x64, 0x52, 0x65, 0x66, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63, 0x65, 0x22, 0x3a, 0x7b, 0x7d, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x74, 0x79, 0x70, 0x65, 0x22, 0x3a, 0x7b, 0x7d, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x52, 0x65, 0x66, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6b, 0x69, 0x6e, 0x64, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x7b, 0x7d, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x63, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x52, 0x65, 0x66, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x7b, 0x7d, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x72, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x66, 0x22, 0x3a, 0x7b, 0x22, 0x2e, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x67, 0x72, 0x6f, 0x75, 0x70, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6b, 0x69, 0x6e, 0x64, 0x22, 0x3a, 0x7b, 0x7d, 0x2c, 0x22, 0x66, 0x3a, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x7b, 0x7d, 0x7d, 0x7d, 0x7d, 0x7d},
        	            	                },
        	            	                Subresource: "",
        	            	            },
        	            	        },
        	            	    },
        	            	    Spec: v1alpha1.KongPluginBindingSpec{
        	            	        PluginReference: v1alpha1.PluginRef{
        	            	            Name: "rate-limiting-kp-nlz49",
        	            	            Kind: &"KongPlugin",
        	            	        },
        	            	        Targets: v1alpha1.KongPluginBindingTargets{
        	            	            RouteReference:         &v1alpha1.TargetRefWithGroupKind{Name:"kongroute-00fceb73", Kind:"KongRoute", Group:"configuration.konghq.com"},
        	            	            ServiceReference:       (*v1alpha1.TargetRefWithGroupKind)(nil),
        	            	            ConsumerReference:      &v1alpha1.TargetRef{Name:"consumer-stzsh"},
        	            	            ConsumerGroupReference: (*v1alpha1.TargetRef)(nil),
        	            	        },
        	            	        ControlPlaneRef: &v1alpha1.ControlPlaneRef{
        	            	            Type:                 "konnectNamespacedRef",
        	            	            KonnectID:            (*string)(nil),
        	            	            KonnectNamespacedRef: &v1alpha1.KonnectNamespacedRef{Name:"cp-f050f08c", Namespace:"test-vfv69"},
        	            	        },
        	            	    },
        	            	    Status: v1alpha1.KongPluginBindingStatus{
        	            	        Konnect:    (*v1alpha1.KonnectEntityStatusWithControlPlaneRef)(nil),
        	            	        Conditions: {
        	            	            {
        	            	                Type:               "Programmed",
        	            	                Status:             "Unknown",
        	            	                ObservedGeneration: 0,
        	            	                LastTransitionTime: time.Date(1970, time.January, 1, 0, 0, 0, 0, time.Local),
        	            	                Reason:             "Pending",
        	            	                Message:            "Waiting for controller",
        	            	            },
        	            	        },
        	            	    },
        	            	}

=== FAIL: test/envtest TestKongPluginBindingManaged/binding_to_KongConsumerGroup,_KongService_and_KongRoute (30.13s)
    kongpluginbinding_managed_test.go:647: deployed new KongService test-vfv69/kongservice-4ab31a55 resource
    kongpluginbinding_managed_test.go:654: deployed new KongRoute test-vfv69/kongroute-f411750b resource
    kongpluginbinding_managed_test.go:661: deployed new KongConsumerGroup test-vfv69/consumer-group-7df97f0e resource
    kongpluginbinding_managed_test.go:669: waiting for 2 KongPluginBindings to be created
    kongpluginbinding_managed_test.go:696: checking that managed KongPlugin test-vfv69/rate-limiting-kp-nlz49 gets plugin-in-use finalizer added
    kongpluginbinding_managed_test.go:700: 
        	Error Trace:	/home/runner/work/gateway-operator/gateway-operator/test/envtest/watch.go:70
        	            				/home/runner/work/gateway-operator/gateway-operator/test/envtest/kongpluginbinding_managed_test.go:700
        	Error:      	KongPlugin wasn't updated to get plugin-in-use finalizer added
        	Test:       	TestKongPluginBindingManaged/binding_to_KongConsumerGroup,_KongService_and_KongRoute

[pmalek]

Happened again in https://github.com/Kong/gateway-operator/actions/runs/11498797771/job/32005372179#step:5:888