Convert test workflow to container-based testing#1208
Conversation
23e7e8c to
d19c0ec
Compare
|
This version of the container CI setup runs on all of the expected platforms. The failures on Fedora 43 and 44 amd64 are expected... they are probably the result of glibc changes, and illustrating those failures is largely the point of this change. The failures on arm are not expected. It looks like the tests that 1840407 marked skipped aren't being skipped in the arm container. I'll have to look into that later. |
|
Very cool @gordonmessmer - funny that we both got to work on that almost at the time 😄 Can you mark the failing tests as skipped on CI then ? So we can still have a 🟢 build for this PR, and tackle the known failures in individual followup issues/PRs ? |
|
FWIW in my own tests in #1206 I remember Arch was also not playing well for the test suite. |
4059975 to
ef33e72
Compare
I suppose I should make a habit of looking at open PRs to see if the thing I want to do is already in progress, but.... -_-
If you'd like, I can squash these commits and then work on disabling some tests. Or you can squash these and merge in its current condition, and then disable tests in another PR. Either way, I think it makes sense to disable tests in a separate commit so that it's easier to revert later. I think that the PR is now working as expected. All of the Ubuntu jobs are green. Fedora 42 on amd64 is green. Fedora 43 and rawhide fail, but that's expected. Not entirely sure why f42:arm64 fails. :-/ |
Yeah, that's fine. As long as we track the disabled tests in an issue or somewhere so we don't forget they fail (and point it if someone comes and asks why it fails) I can de-dust my arm64 dev box and install a fedora43 vm over the weekend and have a look. But in the meantime I see no reason not to merge this PR as-is when all lights come back green |
|
I'll send a commit to disable some tests shortly... Are you able to reopen and rebase #1197 ? It fixes some of the tests currently failing. |
Refactor the CI workflow to build and run tests inside Docker containers instead of directly on GitHub runners. This adds support for testing on Fedora releases in addition to Ubuntu 22.04 and 24.04.
d6febdd to
e28dc31
Compare
e28dc31 to
049d3dd
Compare
049d3dd to
1a098de
Compare
|
Checking back in... Let me know if you'd like any more changes to this PR. All the tests were green on the last run. After merging this, I'd like to try to get #1197 merged, and then restore some of the temporarily disabled tests. |
hugsy
left a comment
There was a problem hiding this comment.
awesome work, this is much better now
just pointed out a couple of lint thingy, you can batch-commit them if you'd like
|
done ! and sorry for the long wait 😞 |
1a098de to
1cbda7a
Compare
Easier to squash those changes into the "disable temporarily" commit locally. :) Thanks! |
1cbda7a to
dd318ff
Compare
Refactor the CI workflow to build and run tests inside Docker containers instead of directly on GitHub runners. This adds support for testing on Fedora releases in addition to Ubuntu 22.04 and 24.04.
Description
This change uses container images for tests so that platforms other than the ones provided directly by GitHub can be tested.
Testing in container images makes it easier to support a broader range of platforms, but more importantly it allows gef to test and adapt to changes in glibc, gcc, and Python as they are released, rather than re-synchronizing with those projects every two years.
Checklist