Skip to content

Run e2e-tests in CI#1210

Merged
adombeck merged 41 commits intomainfrom
e2e-tests-in-ci
Feb 4, 2026
Merged

Run e2e-tests in CI#1210
adombeck merged 41 commits intomainfrom
e2e-tests-in-ci

Conversation

@adombeck
Copy link
Contributor

Run our new e2e-tests in the GitHub CI.

UDENG-7426

@codecov
Copy link

codecov bot commented Jan 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.62%. Comparing base (c17b4f5) to head (b4bf91b).
⚠️ Report is 42 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1210      +/-   ##
==========================================
+ Coverage   87.59%   87.62%   +0.03%     
==========================================
  Files          91       91              
  Lines        6231     6231              
  Branches      111      111              
==========================================
+ Hits         5458     5460       +2     
+ Misses        717      715       -2     
  Partials       56       56              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@denisonbarbosa denisonbarbosa force-pushed the e2e-tests-in-ci branch 3 times, most recently from 5fd0acc to 4201248 Compare January 27, 2026 13:56
@adombeck adombeck force-pushed the e2e-tests-in-ci branch 3 times, most recently from 9f93b5a to 0e7717f Compare February 3, 2026 20:02
@adombeck adombeck marked this pull request as ready for review February 3, 2026 23:39
@adombeck
Copy link
Contributor Author

adombeck commented Feb 3, 2026

@denisonbarbosa I rebased and squashed the fixup commits. We're good to merge from my side.

@adombeck
Copy link
Contributor Author

adombeck commented Feb 4, 2026

@denisonbarbosa I now see that there are still some tmp: commits of yours. You might want to check if you want to drop those or rename them.

denisonbarbosa and others added 12 commits February 4, 2026 06:55
This makes our life easier when trying to upload the test outputs as
artifacts on CI.
The syntax used in this execution was flawed.
This is a QoL change to help us fine tune the artifacts in CI.
We were using only the base name to create the image file, which could
cause conflicts if we ever try to setup both releases.
The 'await' command on virsh is not available on Noble, which is our
runner in github, so we need to use another way to check for the domain
status.
Update the commands to also run `sudo apt-get update` and add some
dependencies that were missing from the lists
This workflow downloads an Ubuntu image, sets up some of the
aplications and configurations that we will need in the VM in order to
be able to run the E2E tests and then caches it for later use to save us
some time when running the tests.

Co-authored-by: Adrian Dombeck <adrian.dombeck@canonical.com>
fix output artifact naming

ci/e2e-tests: Use env vars instead of config file

The config file is useful when running the tests locally, but it only
sets environment variables, so in the CI it's simpler to just set the
environment variables directly.

ci/e2e-tests: Fix indentation
This caused the provisioning to fail
We were padding the generated code with trailing 0s, but we should have
been padding with leading 0s. This commit fixes that.
The code is generated according to the current time and is valid for 30
seconds. This means that if we generate the code just before the time
window changes, it might be invalid by the time we use it.
To avoid this, we make sure the time is safely within a new window before
generating the code.
denisonbarbosa and others added 27 commits February 4, 2026 06:55
This was stopping the entire test execution when a test failed and we
don't want that
We recently changed the error message that is shown when the broker
fails to reach the provider, so we need to update this check on the
tests as well.
Sometimes machinectl login hangs and it causes the tests to fail. Let's
try reopening it a couple of times (in case of failure) to see if we are
able to get more consistent behavior
systemd only supports sending the journal via VSOCK since v256. In the
GitHub CI the newest Ubuntu release we can use on the runners (both
GitHub-hosted or self-hosted) is 24.04, which has systemd 255.
It doesn't only "Build and Cache Ubuntu VM", it runs the e2e-tests and
does everything required to do so.
On GitHub Runners, the VM can't connect to the host via TCP. This commit
instead makes the host connect to the VM to retrieve the journal.
The tests are also useful when targeting other branches than main.
There are not many files in the repo which don't affect the e2e-tests
(once we run them with authd and broker artifacts built from the branch)
but it's still worthwhile to add include/exclude paths because the tests
are very resource intensive.
To avoid duplication between the msentraid and google e2e-tests.
So that we automatically re-provision the VM image if anything changed
in the relevant files.
It's not only testing authd-oidc-brokers but also authd.
We ignored errors during cloud-init and happily continued provisioning a
broken VM.
Should save some time and bandwidth.
@adombeck adombeck merged commit b6cb564 into main Feb 4, 2026
21 of 26 checks passed
@adombeck adombeck deleted the e2e-tests-in-ci branch February 4, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants