Allow throttling testing based on PR labels #182
Conversation
Avoid redundant testing for Make builds, on the assumption that CMake handles most of them: - Skip all GPU and HVX testing - Skip the error, warning, performance, and tutorials (basically, just correctness and generator) - Continue to skip python & apps where we used to - Continue to skip wasm for Make entirely
Followup from recent Halide meeting, this changes build/test defaults to require opt-in for GPU and/or HVX testing, based on labels:
We vary testing depending on how a PR is labeled:
- if 'buildbot_test_nothing' is present, no testing is done and the rest are ignored. ('skip_buildbots' is a synonym for this.)
- if 'buildbot_test_everything' is present, we do all possible testing.
- Otherwise, we default to doing (at least) baseline CPU testing on all targets.
- if 'buildbot_test_gpu' is present, we also test all GPU targets (cuda, opencl, metal, d3d12, etc)
- if 'buildbot_test_hvx' is present, we also test Hexagon
This has been (lightly) tested on a local buildbot and seems pretty close, but needs more testing on a proper setup.
|
It would be nice if these labels corresponded to something that already exists: eg. the actual target/feature names, the app names, or the CTest label names. It's fine to have meta-labels like We also still need to do the work of labeling the GPU-enabled correctness tests and the like, so we don't re-run the exact same test with irrelevant target flags. I'm also not sold on the naming convention. Having |
|
How bad would it be to dust this off and land it? |
Followup from recent Halide meeting, this changes build/test defaults to require opt-in for GPU and/or HVX testing, based on labels:
With this in place, we vary testing depending on how a PR is labeled:
This has been (lightly) tested on a local buildbot and seems pretty close, but needs more testing on a proper setup.