Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support GKE E2E tests and use (actual) E2E tests to gate releases #3035

Merged
merged 13 commits into from
Nov 9, 2022

Conversation

rainest
Copy link
Contributor

@rainest rainest commented Oct 7, 2022

What this PR does / why we need it:

  • Adds GKE and existing cluster support to E2E tests.
  • Uses the E2E tests as the release job condition for creating the release tag not yet, currently just a test variant because I can't easily test with the release workflow.

We need this because we aren't currently running the test suite we want to during releases. It also requires some refactoring of the E2E tests themselves, which is a good opportunity to refactor them and KTF to address some ease of use issues. Tests cannot easily load cluster-specific functionality (loading images, adding a KIND configuration, etc.) without abusing the environment addon functionality (we have addons that fail if loaded on an unsupported cluster, which we don't want). Kong/kubernetes-testing-framework#403 seems an appropriate means of allowing for this.

Which issue this PR fixes:
Part of #2790
Fix #2287 (I think this actually got fixed earlier, but whatever, it's still open and this obsoletes it).
Fix #3041

Special notes for your reviewer:

Convert tests other than the webhook tests to the new E2E environment
helper.

Refactor image loads to always succeed during builder construction.
@rainest rainest temporarily deployed to Configure ci November 1, 2022 20:26 Inactive
@rainest rainest temporarily deployed to Configure ci November 1, 2022 20:34 Inactive
@rainest rainest temporarily deployed to Configure ci November 1, 2022 20:49 Inactive
@rainest rainest marked this pull request as ready for review November 1, 2022 20:51
@rainest rainest requested a review from a team as a code owner November 1, 2022 20:51
@rainest rainest temporarily deployed to Configure ci November 1, 2022 21:19 Inactive
test/e2e/helpers_test.go Outdated Show resolved Hide resolved
test/e2e/helpers_test.go Outdated Show resolved Hide resolved
@rainest rainest temporarily deployed to Configure ci November 2, 2022 18:56 Inactive
Co-authored-by: Grzegorz Burzyński <[email protected]>
@rainest rainest temporarily deployed to Configure ci November 2, 2022 19:00 Inactive
@rainest rainest enabled auto-merge (squash) November 2, 2022 19:03
@rainest rainest temporarily deployed to Configure ci November 2, 2022 19:23 Inactive
@rainest rainest temporarily deployed to Configure ci November 2, 2022 19:23 Inactive
pmalek
pmalek previously approved these changes Nov 2, 2022
.github/workflows/e2e.yaml Show resolved Hide resolved
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:05 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:05 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:05 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:05 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:06 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:19 Inactive
@rainest
Copy link
Contributor Author

rainest commented Nov 9, 2022

Discovered fun facts about environment notifications and the current E2E test job layout. 500+ low-content notifications in the timeline, very useful.

https://github.com/Kong/kubernetes-ingress-controller/actions/runs/3424207046/jobs/5703617497 is the latest run with the changes to accept and use a (single value) matrix. This also mitigates an unrelated bug in the Makefile that I can't and don't really want to figure out:

16:16:38-0800 esenin $ grep -A1 ^test.e2e.gke Makefile
test.e2e.gke:          
	CLUSTER_NAME="e2e-$(uuidgen)" \

16:16:43-0800 esenin $ CLUSTER_NAME="e2e-$(uuidgen)"; echo $CLUSTER_NAME
e2e-69ba2dd0-0578-415c-8e7c-638351a73d22              

16:16:47-0800 esenin $ make test.e2e.gke                           
CLUSTER_NAME="e2e-" \  
	KUBERNETES_CLUSTER_NAME="" go run hack/e2e/cluster/deploy/main.go \
	KONG_TEST_CLUSTER="gke:" \

I originally copied the target from the existing GKE target, which appears to be broken. uuidgen output apparently disappears into the void when make runs it, and that variable isn't being copied properly to boot. I've added a name generator to the hack code because I don't much want to think about bash, and the names are mostly just UUIDs anyway.

@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:27 Inactive
@rainest rainest temporarily deployed to Configure ci November 9, 2022 00:27 Inactive
@rainest rainest merged commit c6344be into main Nov 9, 2022
@rainest rainest deleted the chore/e2e-release branch November 9, 2022 02:53
czeslavo added a commit that referenced this pull request Nov 9, 2022
Consolidate E2E environment builders into helper.

Add GKE support for E2E tests.

Add GKE E2E workflow and make target.

Refactor image loads to always succeed during builder construction.

Change final release test from integration to E2E.

Co-authored-by: Grzegorz Burzyński <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add actions to run e2e/integration on k8s clusters of cloud providers Refactor E2E image loads
4 participants