Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make UI tests work on github actions #153

Open
ensonic opened this issue Mar 2, 2024 · 2 comments
Open

Make UI tests work on github actions #153

ensonic opened this issue Mar 2, 2024 · 2 comments

Comments

@ensonic
Copy link
Member

ensonic commented Mar 2, 2024

We still can't run the UI tests and this makes the whole CI run super slow (as each test runs into a timeout).

Locally the tests mostly work:

  • debian testing: 84%: Checks: 113, Failures: 0, Errors: 18
  • opensuse tumbleweed: ...

The test setup believes it has successfully setup the xvfb:

0:24:22.152876977 72611 0x55f86d997c70 INFO                bt-check tests/bt-check-ui.c:221:check_setup_test_display: display 0x55f86d9e85e0,":0" is active

It is not even clear if the xvfb setup is to blame. A lot of the tests have these as the last log lines:

0:26:02.382873958 72703 0x55f86d997c70 INFO                 bt-edit src/ui/edit/main-page-sequence.c:1413:sequence_table_init: preparing sequence table
0:26:02.383991664 72703 0x55f86d997c70 INFO                 bt-core src/lib/core/settings.c:408:bt_settings_make: return cached settings object 0x55f86dd10560,ref_ct=3,floating=0
0:26:02.384494252 72703 0x55f86d997c70 INFO      GST_PLUGIN_LOADING ../gst/gstplugin.c:987:_priv_gst_plugin_load_file_for_registry: plugin "/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstplayback.so" loaded
0:26:02.384515702 72703 0x55f86d997c70 INFO     GST_ELEMENT_FACTORY ../gst/gstelementfactory.c:489:gst_element_factory_create_with_properties: creating element "playbin"

The test itself seem to actually work and finish quickly (we see cleanup markers in the log). It feels more like the something in the test hardness hangs (e.g. taking down the xvfb server)

Completely alternative idea:

We could try to use a matrix to run groups of tests in parallel.
https://stackoverflow.com/questions/59175332/using-output-from-a-previous-job-in-a-new-one-in-a-github-action
https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs

ensonic referenced this issue Mar 2, 2024
It just made it even slower
@ensonic ensonic changed the title Use a test matrix for ci Make UI tests work on github actions Mar 2, 2024
ensonic pushed a commit that referenced this issue Mar 2, 2024
@ensonic
Copy link
Member Author

ensonic commented Mar 2, 2024

With the skipped UI test, CI run finishes in ~6m (instead of 45m !):

https://github.com/Buzztrax/buzztrax/actions/runs/8123130041

ensonic added a commit that referenced this issue Mar 9, 2024
Run only the ui test with increased log levels to debug test infra
issues on CI.

See #153
ensonic added a commit that referenced this issue Mar 9, 2024
Run only the ui test with increased log levels to debug test infra
issues on CI.

See #153
ensonic added a commit that referenced this issue Mar 9, 2024
Run only the ui test with increased log levels to debug test infra
issues on CI.

See #153
@ensonic
Copy link
Member Author

ensonic commented Mar 9, 2024

One of the test that also fail locally on debian is:

BT_CHECKS="test_bt_edit_application_new_song" make bt_edit.check

When I run it with BT_CHECK_NO_XVFB=1 no window appears. Maybe this runs too quickly to even get any event loop to kick in?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant