Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use ephemeral S3 buckets for E2E tests #17157

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Use ephemeral S3 buckets for E2E tests #17157

wants to merge 3 commits into from
+263 −14

Conversation

ameukam
Copy link
Member

@ameukam ameukam commented Dec 28, 2024

Use S3 buckets created during the lifecycle of a test instead of a
static one.

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 28, 2024
@k8s-ci-robot k8s-ci-robot requested a review from hakman December 28, 2024 23:16
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Dec 28, 2024
rifelpet
rifelpet reviewed Dec 28, 2024
View reviewed changes
Copy link
Member

@rifelpet rifelpet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a second bucket for the cluster's oidc discovery store, we can reuse this logic but it will need a policy that allows anonymous read access.

@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from a39430a to e1415f9 Compare January 11, 2025 12:33
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign olemarkus for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hakman
Copy link
Member

hakman commented Jan 12, 2025

/test pull-kops-e2e-k8s-aws-calico

@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from e1415f9 to 7e72ef4 Compare January 15, 2025 14:26
@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from 7e72ef4 to ef45e62 Compare February 7, 2025 17:02
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 7, 2025
@ameukam
Copy link
Member Author

ameukam commented Feb 7, 2025

/test pull-kops-e2e-k8s-aws-calico

@ameukam
Copy link
Member Author

ameukam commented Feb 7, 2025

/retest

@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from ef45e62 to 54d6ea3 Compare February 7, 2025 17:53
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 7, 2025
@hakman
Copy link
Member

hakman commented Feb 8, 2025

/test pull-kops-kubernetes-e2e-ubuntu-gce-build

@ameukam ameukam mentioned this pull request Feb 12, 2025
Open
@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from 54d6ea3 to 3680629 Compare February 21, 2025 22:05
@ameukam
Copy link
Member Author

ameukam commented Feb 21, 2025

/test pull-kops-test

@ameukam
Copy link
Member Author

ameukam commented Feb 21, 2025

/test pull-kops-e2e-k8s-aws-amazonvpc

@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from 3680629 to 1c64e02 Compare February 21, 2025 22:32
@ameukam
Copy link
Member Author

ameukam commented Feb 21, 2025

/test pull-kops-e2e-k8s-aws-amazonvpc

@ameukam
Use ephemeral S3 buckets for E2E tests
d73a286 
Use S3 buckets created during the lifecycle of a test instead of a
static one and provide the capability to make them read-only public.

Signed-off-by: Arnaud Meukam <[email protected]>
@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from 1c64e02 to d73a286 Compare February 23, 2025 19:10
@ameukam
Copy link
Member Author

ameukam commented Feb 23, 2025

/test pull-kops-e2e-k8s-aws-amazonvpc

@ameukam
Copy link
Member Author

ameukam commented Feb 23, 2025

/test pull-kops-e2e-k8s-aws-calico
/test pull-kops-e2e-k8s-aws-cilium-1-31

xmudrii
xmudrii suggested changes Feb 27, 2025
View reviewed changes
@ameukam ameukam marked this pull request as ready for review February 28, 2025 16:45
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 28, 2025
@ameukam
Copy link
Member Author

ameukam commented Feb 28, 2025

/label tide/merge-method-squash

@k8s-ci-robot k8s-ci-robot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Feb 28, 2025
@xmudrii @ameukam
Improve ephemeral S3 buckets implementation for tests
cc3e2a1 
Signed-off-by: Marko Mudrinić <[email protected]>
Signed-off-by: Arnaud Meukam <[email protected]>
@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch from 255229d to cc3e2a1 Compare February 28, 2025 17:08
@ameukam
Copy link
Member Author

ameukam commented Mar 1, 2025

/retest

@ameukam
Copy link
Member Author

ameukam commented Mar 1, 2025

/test pull-kops-e2e-cni-amazonvpc
/test pull-kops-e2e-cni-amazonvpc-ipv6

@ameukam ameukam force-pushed the kubetest-aws-s3-ephemeral-buckets branch 2 times, most recently from bbddbc5 to cc3e2a1 Compare March 4, 2025 11:01
@xmudrii
Base S3 bucket name on ProwJob ID
7d6f7e2 
Signed-off-by: Marko Mudrinić <[email protected]>
@xmudrii
Copy link
Member

xmudrii commented Mar 4, 2025

/test pull-kops-e2e-cni-amazonvpc
/test pull-kops-e2e-cni-amazonvpc-ipv6

@xmudrii
Copy link
Member

xmudrii commented Mar 4, 2025

/test pull-kops-kubernetes-e2e-ubuntu-gce-build

@ameukam
Copy link
Member Author

ameukam commented Mar 4, 2025

/retest

@ameukam
Copy link
Member Author

ameukam commented Mar 4, 2025

@rifelpet @hakman PTAL.
We may need to override some contexts.

@k8s-ci-robot
Copy link
Contributor

@ameukam: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kops-e2e-cni-flannel a39430a link true /test pull-kops-e2e-cni-flannel
pull-kops-e2e-gce-cni-calico e1415f9 link true /test pull-kops-e2e-gce-cni-calico
pull-kops-e2e-gce-cni-kindnet e1415f9 link true /test pull-kops-e2e-gce-cni-kindnet
pull-kops-e2e-k8s-aws-amazonvpc-u2404 7e72ef4 link true /test pull-kops-e2e-k8s-aws-amazonvpc-u2404
pull-kops-e2e-cni-amazonvpc-ipv6 7d6f7e2 link false /test pull-kops-e2e-cni-amazonvpc-ipv6

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BenTheElder BenTheElder BenTheElder left review comments

@rifelpet rifelpet rifelpet left review comments

@xmudrii xmudrii xmudrii requested changes

@hakman hakman Awaiting requested review from hakman

@olemarkus olemarkus Awaiting requested review from olemarkus

Assignees

@xmudrii xmudrii

Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ameukam @k8s-ci-robot @hakman @xmudrii @BenTheElder @rifelpet