add metadata to agones webhook autoscaler request #3957

swermin · 2024-08-20T06:20:23Z

What type of PR is this?

Uncomment only one /kind <> line, press enter to put that in a new line, and remove leading whitespace from that line:

/kind breaking
/kind bug
/kind cleanup
/kind documentation

/kind feature

/kind hotfix
/kind release

What this PR does / Why we need it:
This PR extends the webhook autoscaler with metadata, so that the the service retrieving the request can use the lables/annotations to do the calculation

Which issue(s) this PR fixes:
Closes #3951

Special notes for your reviewer:

google-cla · 2024-08-20T06:20:27Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

agones-bot · 2024-08-20T06:30:13Z

Build Failed 😭

Build Id: 5ade9d2e-d20e-4cad-b76b-697960ab7ff0

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

igooch

Thanks for the contribution. Could you please also add unit and e2e tests?

gongmax · 2024-09-19T17:05:36Z

@swermin kindly ping on this, could you resolve the conflicts and address the above comments?

swermin · 2024-09-20T09:50:09Z

Yes ofc! I have unit tests ready and just left with the e2e tests. I’ll push the current changes and focus on the e2e tests today and this weekend if needed!

agones-bot · 2024-09-22T19:36:24Z

Build Failed 😭

Build Id: 166b3f1b-5297-4b2a-bc9b-3b0ec925abe5

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-09-22T20:12:56Z

Build Failed 😭

Build Id: 984531ec-7bcc-4fa4-bf36-089d5719a947

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

swermin · 2024-09-24T18:05:02Z

@igooch do you have any good suggestions on how to do the e2e tests for this?
Current default webhook autoscaler is using the example image to test the scaling. I feel like adding support in that image feels a bit weird but I have no good idea on how to add something

agones-bot · 2024-09-24T18:16:49Z

Build Failed 😭

Build Id: 0ef393db-4a59-403f-8729-4a86e54fb6d3

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

igooch · 2024-10-08T19:48:54Z

I just got back from being out of office, so I'm still catching up.

For the e2e tests, do you need to make changes to examples/autoscaler-webhook/main.go? Depending on the test you may be able to simply add MetaData to the default similar to https://github.com/googleforgames/agones/blob/c26f7371980418b56b816b13d265c9cc9d092f32/test/e2e/fleetautoscaler_test.go#L549C14-L549C38. If you do need to make changes to the examples/autoscaler-webhook/main.go, then a maintainer will need to build and push a new example autoscaler webhook image with the changes. That being said, it looks like the unit test is failing.

From the most recent build:

        	Test:       	TestApplyWebhookPolicyWithMetadata
        	            	actual  : 5
        	            	expected: 11
        	Error:      	Not equal: 
        	Error Trace:	/go/src/agones.dev/agones/pkg/fleetautoscalers/fleetautoscalers_test.go:618
    fleetautoscalers_test.go:618: 
--- FAIL: TestApplyWebhookPolicyWithMetadata (0.01s)

You'll need to run ~/agones/build$ make gen-crd-code to generate the deepcopy for pkg/apis/autoscaling/v1/fleetautoscaler.go, but even with that the test is still failing, so there's another issue going on.

agones-bot · 2024-10-21T20:10:12Z

Build Succeeded 🥳

Build Id: 00518f36-9edb-49bc-8c2a-b755dd8f48d1

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.45.0-dev-9e74cc5
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.45.0-dev-9e74cc5
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.45.0-dev-9e74cc5-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.45.0-dev-9e74cc5
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.45.0-dev-9e74cc5
Linux C++ SDK (build): agonessdk-1.45.0-dev-9e74cc5-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.45.0-dev-9e74cc5.zip

A preview of the website (the last 30 builds are retained):

https://9e74cc5-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3957/head:pr_3957 && git checkout pr_3957
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.45.0-dev-9e74cc5

igooch · 2024-12-13T19:05:12Z

/gcbrun

agones-bot · 2024-12-13T20:11:02Z

Build Succeeded 🥳

Build Id: eb8ecbc4-2d69-463d-befa-65337072b6f2

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.46.0-dev-2b8baf3
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.46.0-dev-2b8baf3
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.46.0-dev-2b8baf3-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.46.0-dev-2b8baf3
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.46.0-dev-2b8baf3
Linux C++ SDK (build): agonessdk-1.46.0-dev-2b8baf3-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.46.0-dev-2b8baf3.zip

A preview of the website (the last 30 builds are retained):

https://2b8baf3-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3957/head:pr_3957 && git checkout pr_3957
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.46.0-dev-2b8baf3

markmandel · 2025-02-01T19:04:11Z

site/content/en/docs/Reference/fleetautoscaler.md

@@ -374,6 +374,8 @@ type FleetAutoscaleRequest struct {
 	Namespace string `json:"namespace"`
 	// The Fleet's status values
 	Status v1.FleetStatus `json:"status"`
+	// Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.


Super late to this party, but new additions will need a feature shortcode. See https://agones.dev/site/docs/contribute/#within-a-page for an example.

Also an addition here: https://github.com/googleforgames/agones/blob/main/site/content/en/docs/Guides/feature-stages.md?plain=1#L41 (also feature shortcode - may need to copy the whole table).

This is because docs get published intermediately, so we need to hold back documentation on unreleased features.

markmandel · 2025-02-01T19:05:06Z

Added a note on the issue as well, but my suggestion is that we let this PR through as is (with the above docs changes), and then follow up with e2e test PR.

swermin · 2025-02-03T16:17:37Z

Added a note on the issue as well, but my suggestion is that we let this PR through as is (with the above docs changes), and then follow up with e2e test PR.

Biggest problem with the e2e test is that I need to specify a version of the container to use. So there is no way for me to do the change without having to do two different PRs (at least what I know of)

I tried multiple different ways of getting a e2e test in without having to separate the builds but I cannot seem to find a solution :(

markmandel · 2025-02-09T23:35:42Z

site/content/en/docs/Guides/feature-stages.md

@@ -37,7 +37,8 @@ The current set of `alpha` and `beta` feature gates:
 | [Rolling Update Fixes](https://github.com/googleforgames/agones/issues/3688)                                                | `RollingUpdateFix`                 | Disabled | `Alpha` | 1.41.0 |
 | [Multiple dynamic port ranges](https://github.com/googleforgames/agones/issues/1911)                                        | `PortRanges`                       | Disabled | `Alpha` | 1.41.0 |
 | [Port Policy None](https://github.com/googleforgames/agones/issues/3804)                                                    | `PortPolicyNone`                   | Disabled | `Alpha` | 1.41.0 |
-| [Scheduled Fleet Autoscaling](https://github.com/googleforgames/agones/issues/3008)                                         | `ScheduledAutoscaler`              | Disabled | `Alpha` | 1.43.0 |
+| [Scheduled Fleet Autoscaling](https://github.com/googleforgames/agones/issues/3951)                                         | `ScheduledAutoscaler`              | Disabled | `Alpha` | 1.43.0 |


feature shortcode here too please, and I think this will be good to go 👍🏻

Oh I missed that one, good catch!

I've added the feature gate here, how to I test it in a good way? You did mention that it may require me copying the whole table

markmandel · 2025-02-09T23:36:47Z

/gcbrun

agones-bot · 2025-02-10T01:24:14Z

Build Succeeded 🥳

Build Id: 5c7f95bc-ac04-420a-86c9-1cac805cf8f8

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.47.0-dev-b1d60cf
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.47.0-dev-b1d60cf
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.47.0-dev-b1d60cf-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.47.0-dev-b1d60cf
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.47.0-dev-b1d60cf
Linux C++ SDK (build): agonessdk-1.47.0-dev-b1d60cf-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.47.0-dev-b1d60cf.zip

A preview of the website (the last 30 builds are retained):

https://b1d60cf-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3957/head:pr_3957 && git checkout pr_3957
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.47.0-dev-b1d60cf

markmandel · 2025-02-10T01:55:45Z

pkg/apis/autoscaling/v1/fleetautoscaler.go

@@ -316,6 +316,8 @@ type FleetAutoscaleRequest struct {
 	Namespace string `json:"namespace"`
 	// The Fleet's status values
 	Status agonesv1.FleetStatus `json:"status"`
+	// Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.
+	MetaData *metav1.ObjectMeta `json:"metadata,omitempty"`


🤔 I'm looking at this again, and wondering if this should be all the ObjectMeta - which is a lot, and we copy a chunk of it here already (Name and Namespace for example) - so we are also duplicating some effort.

Should it just be labels and annotations? Do we need everything?

Admittedly, in the ticket it says "metadata" but in my head I thought of it as "labels and annotations" and didn't clarify.

WDYT?

My initial thought was to just do labels and annotations, but it was ”easier” to add the whole metadata.
Now that you raising it it makes more sense to just have labels and annotations.
I’ll update this PR accordingly

While the object is still called MetaData we only ever copy over annotations, labels, and name. Do you want it to be more explicit in that the fields should be called Annotations and Labels?

if runtime.FeatureEnabled(runtime.FeatureFleetAutoscaleRequestMetaData) { faReq.Request.MetaData = &metav1.ObjectMeta{ Name: f.ObjectMeta.Name, Labels: f.ObjectMeta.Labels, Annotations: f.ObjectMeta.Annotations, } }

I knew that I did think of the whole metadata copying and the object itself being too big

While the object is still called MetaData we only ever copy over annotations, labels, and name. Do you want it to be more explicit in that the fields should be called Annotations and Labels?

if runtime.FeatureEnabled(runtime.FeatureFleetAutoscaleRequestMetaData) { faReq.Request.MetaData = &metav1.ObjectMeta{ Name: f.ObjectMeta.Name, Labels: f.ObjectMeta.Labels, Annotations: f.ObjectMeta.Annotations, } }

Yeah - I think this is the way to go (Only having labels and Annotations on the Request struct, not the full ObjectMeta):

We control then what is sent.

People aren't wondering why the rest of ObjectMeta isn't populated anyway.

That is a fair point, I’ve updated the code to reflect this

…Meta

markmandel · 2025-02-24T00:53:07Z

site/content/en/docs/Guides/feature-stages.md

@@ -38,6 +38,9 @@ The current set of `alpha` and `beta` feature gates:
 | [Multiple dynamic port ranges](https://github.com/googleforgames/agones/issues/1911)                                        | `PortRanges`                       | Disabled | `Alpha` | 1.41.0 |
 | [Port Policy None](https://github.com/googleforgames/agones/issues/3804)                                                    | `PortPolicyNone`                   | Disabled | `Alpha` | 1.41.0 |
 | [Scheduled Fleet Autoscaling](https://github.com/googleforgames/agones/issues/3008)                                         | `ScheduledAutoscaler`              | Disabled | `Alpha` | 1.43.0 |
+{{% feature publishVersion="1.48.0" %}}


Mid table items tend to render badly, you may need to copy the whole table. But I'll run it and we can see what it looks like in preview.

(make site-server will give you a local version if you would like to try it).

https://1d43bbe-dot-preview-dot-agones-images.appspot.com/site/docs/guides/feature-stages/#feature-gates

Going to have to copy the whole block, but otherwise, I think this is good to go.

markmandel · 2025-02-24T00:53:37Z

/gcbrun

Thanks for your patience!

agones-bot · 2025-02-24T01:59:41Z

Build Succeeded 🥳

Build Id: a29b3a1f-88f6-411f-ab57-37b08c7ad17a

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.48.0-dev-1d43bbe
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.48.0-dev-1d43bbe
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.48.0-dev-1d43bbe-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.48.0-dev-1d43bbe
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.48.0-dev-1d43bbe
Linux C++ SDK (build): agonessdk-1.48.0-dev-1d43bbe-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.48.0-dev-1d43bbe.zip

A preview of the website (the last 30 builds are retained):

https://1d43bbe-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3957/head:pr_3957 && git checkout pr_3957
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.48.0-dev-1d43bbe

markmandel · 2025-02-24T16:31:02Z

/gcbrun

agones-bot · 2025-02-24T17:02:21Z

Build Failed 😭

Build Id: 2e11f722-e322-4bd7-a76f-9579725cdde2

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

swermin · 2025-02-24T18:21:56Z

I don’t understand why the build failed 🧐

markmandel · 2025-02-24T18:28:44Z

Looks like a flake in https://console.cloud.google.com/cloud-build/builds/2e11f722-e322-4bd7-a76f-9579725cdde2;step=26?project=agones-images submit-upgrade-test-cloud-build. @igooch you might want to look into that.

agones-bot · 2025-02-24T18:58:00Z

Build Failed 😭

Build Id: 50a3d4db-0e7a-438c-be30-3af541ecb9ad

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

igooch · 2025-02-24T19:05:30Z

We're looking into how to pipe through relevant logs to the build output without spamming the build output file. It looks like the issue was a failure to pull the build image:

2025/02/24 17:01:25 sdk-client-test on  sdk-client-test-dm5x8 has failed. Latest event: message Failed to pull image "us-docker.pkg.dev/agones-images/ci/sdk-client-test:1.42.0": rpc error: code = DeadlineExceeded desc = failed to pull and unpack image "us-docker.pkg.dev/agones-images/ci/sdk-client-test:1.42.0": failed to resolve reference "us-docker.pkg.dev/agones-images/ci/sdk-client-test:1.42.0": failed to do request: Head "https://us-docker.pkg.dev/v2/agones-images/ci/sdk-client-test/manifests/1.42.0": dial tcp 74.125.195.82:443: i/o timeout

But the image exists:

And can be pulled using docker:

me@me:~$ docker pull us-docker.pkg.dev/agones-images/ci/sdk-client-test:1.42.0
1.42.0: Pulling from agones-images/ci/sdk-client-test
688513194d7a: Already exists
bfb59b82a9b6: Already exists
efa9d1d5d3a2: Already exists
a62778643d56: Already exists
7c12895b777b: Already exists
3214acf345c0: Already exists
5664b15f108b: Already exists
0bab15eea81d: Already exists
4aa0ea1413d3: Already exists
da7816fa955e: Already exists
9aee425378d2: Already exists
daf4059f7961: Pull complete
Digest: sha256:60811eb6965ded7236dc76bea020d4a84250ecab7a3ca483c237207712bb97e2
Status: Downloaded newer image for us-docker.pkg.dev/agones-images/ci/sdk-client-test:1.42.0
us-docker.pkg.dev/agones-images/ci/sdk-client-test:1.42.0

So this looks to be a flake, although an odd one.

igooch · 2025-02-24T19:05:36Z

/gcbrun

agones-bot · 2025-02-24T20:38:56Z

Build Succeeded 🥳

Build Id: c5cbe07f-f6fd-47dd-92f1-a94e63d056a5

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.48.0-dev-de9f9f2
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.48.0-dev-de9f9f2
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.48.0-dev-de9f9f2-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.48.0-dev-de9f9f2
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.48.0-dev-de9f9f2
Linux C++ SDK (build): agonessdk-1.48.0-dev-de9f9f2-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.48.0-dev-de9f9f2.zip

A preview of the website (the last 30 builds are retained):

https://de9f9f2-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3957/head:pr_3957 && git checkout pr_3957
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.48.0-dev-de9f9f2

github-actions bot added kind/feature New features for Agones size/S labels Aug 20, 2024

swermin force-pushed the feature/extend-webhook-autoscaler branch from a970b12 to 9ec84ce Compare August 24, 2024 18:32

igooch reviewed Sep 9, 2024

View reviewed changes

swermin force-pushed the feature/extend-webhook-autoscaler branch from 9ec84ce to f56ba00 Compare September 22, 2024 19:28

swermin force-pushed the feature/extend-webhook-autoscaler branch from b9ab8fd to 8d0c86a Compare September 24, 2024 18:05

swermin force-pushed the feature/extend-webhook-autoscaler branch from 8d0c86a to 9e74cc5 Compare October 21, 2024 19:04

markmandel mentioned this pull request Feb 1, 2025

Extend Webhook autoscaler to send fleet metadata with the request #3951

Closed

markmandel reviewed Feb 1, 2025

View reviewed changes

swermin force-pushed the feature/extend-webhook-autoscaler branch 2 times, most recently from 0c9b515 to 315d70d Compare February 3, 2025 23:44

swermin requested a review from markmandel February 5, 2025 12:54

markmandel reviewed Feb 9, 2025

View reviewed changes

markmandel reviewed Feb 10, 2025

View reviewed changes

swermin added 5 commits February 17, 2025 19:55

add metadata to agones webhook autoscaler request

9880a7f

add feature shortcode for the documentation

7c0811d

add feature to feature stages

52673c8

revert swap of links

643efb5

update feature gates to reflect 1.48 instead of 1.47

b819b17

swermin force-pushed the feature/extend-webhook-autoscaler branch from b0aedbf to b819b17 Compare February 17, 2025 18:55

use explicit fields for labels/annotations instead of wrapping Object…

1d43bbe

…Meta

markmandel reviewed Feb 24, 2025

View reviewed changes

copy the whole table due to feature gates not printing it correctly

de9f9f2

markmandel approved these changes Feb 24, 2025

View reviewed changes

markmandel merged commit 1e7a8e4 into googleforgames:main Feb 24, 2025
4 checks passed

add metadata to agones webhook autoscaler request #3957

add metadata to agones webhook autoscaler request #3957

Conversation

swermin commented Aug 20, 2024

google-cla bot commented Aug 20, 2024

agones-bot commented Aug 20, 2024

igooch left a comment

Choose a reason for hiding this comment

gongmax commented Sep 19, 2024

swermin commented Sep 20, 2024

agones-bot commented Sep 22, 2024

agones-bot commented Sep 22, 2024

swermin commented Sep 24, 2024 • edited Loading

agones-bot commented Sep 24, 2024

igooch commented Oct 8, 2024 • edited Loading

agones-bot commented Oct 21, 2024

igooch commented Dec 13, 2024

agones-bot commented Dec 13, 2024

Choose a reason for hiding this comment

markmandel commented Feb 1, 2025

swermin commented Feb 3, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markmandel commented Feb 9, 2025

agones-bot commented Feb 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markmandel commented Feb 24, 2025

agones-bot commented Feb 24, 2025

markmandel commented Feb 24, 2025

agones-bot commented Feb 24, 2025

swermin commented Feb 24, 2025

markmandel commented Feb 24, 2025

agones-bot commented Feb 24, 2025

igooch commented Feb 24, 2025

igooch commented Feb 24, 2025

agones-bot commented Feb 24, 2025

swermin commented Sep 24, 2024 •

edited

Loading

igooch commented Oct 8, 2024 •

edited

Loading