ci: use different git repo id for releases [backport 4.6]#17972
ci: use different git repo id for releases [backport 4.6]#17972dd-octo-sts[bot] wants to merge 1 commit into4.6from
Conversation
## Description When a flaky test is detected on a release branch (e.g. `4.8`, `4.10`), it is quarantined across **all** branches — including `main`. This causes problems in two directions: 1. **Release branches don't receive flaky-test-fix backports from `main`**, so a test that gets fixed and unquarantined on `main` may remain quarantined (or vice versa) on a release branch indefinitely. 2. **New flakes introduced during release activity** (cherry-picks, backports, release-specific patches) pollute the main pool and can suppress test failures on `main` and feature branches. This change uses a separate [Datadog Test Optimization](https://docs.datadoghq.com/tests/) Flaky Test Management pool for release branches by overriding `DD_GIT_REPOSITORY_URL` to `https://github.com/DataDog/dd-trace-py-release.git` in GitLab CI pipelines that run on release branches. The Datadog backend uses `git.repository_url` (surfaced in the UI as `@git.repository.id_v2`) as the partition key for Flaky Test Management. By giving release-branch pipelines a distinct URL, their quarantine list is fully independent from the one used by `main`, merge queues, and feature branches. ### How it works - **Release branches** (`X.Y` / `X.YZ` pattern, e.g. `4.8`, `4.10`): `DD_GIT_REPOSITORY_URL` is set to `https://github.com/DataDog/dd-trace-py-release.git` vian`workflow:rules:variables`, which overrides the global variable for the entire pipeline. - **All other pipelines** (`main`, merge queues, feature branches): continue using `https://github.com/DataDog/dd-trace-py.git` unchanged. ## Testing The change is CI-only. It can be validated by triggering a pipeline on a release branch (e.g. `4.8`) and confirming that test results appear under `@git.repository.id_v2:github.com/datadog/dd-trace-py-release` in the Datadog Test Optimization UI, while a pipeline on `main` still shows `@git.repository.id_v2:github.com/datadog/dd-trace-py`. ## Risks - **Quarantine list starts empty for release branches.** Any tests already quarantined on release branches under the old pool ID will not carry over — they will re-run and be re-detected as flaky. This is a one-time, bounded cost. - No application code is changed; no test behaviour is affected outside of the flaky-test quarantine logic. ## Additional Notes `workflow:rules:variables` in GitLab CI takes precedence over the global `variables:` block, so the override is not silently clobbered. `DD_GIT_REPOSITORY_URL` is not redefined in any included job template or project-level CI/CD variable, so the behaviour is deterministic. Co-authored-by: federico.mon <federico.mon@datadoghq.com> (cherry picked from commit e3bdccd) Co-authored-by: Federico Mon <federico.mon@datadoghq.com>
Codeowners resolved as |
Performance SLOsComparing candidate unknown (unknown) with baseline 4.6 (ce51055) 📈 Performance Regressions (2 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 106.064µs (SLO: <130.000µs 📉 -18.4%) vs baseline: +3.2% Memory: ✅ 43.646MB (SLO: <46.000MB -5.1%) vs baseline: +5.5% ✅ add_inplace_aspectTime: ✅ 102.993µs (SLO: <130.000µs 📉 -20.8%) vs baseline: +0.8% Memory: ✅ 43.647MB (SLO: <46.000MB -5.1%) vs baseline: +4.8% ✅ add_inplace_noaspectTime: ✅ 28.298µs (SLO: <40.000µs 📉 -29.3%) vs baseline: -0.2% Memory: ✅ 43.707MB (SLO: <46.000MB -5.0%) vs baseline: +4.8% ✅ add_noaspectTime: ✅ 49.128µs (SLO: <70.000µs 📉 -29.8%) vs baseline: -0.3% Memory: ✅ 43.723MB (SLO: <46.000MB -4.9%) vs baseline: +5.2% ✅ bytearray_aspectTime: ✅ 248.537µs (SLO: <400.000µs 📉 -37.9%) vs baseline: -0.6% Memory: ✅ 43.734MB (SLO: <46.000MB -4.9%) vs baseline: +5.6% ✅ bytearray_extend_aspectTime: ✅ 661.757µs (SLO: <800.000µs 📉 -17.3%) vs baseline: +0.7% Memory: ✅ 43.408MB (SLO: <46.000MB -5.6%) vs baseline: +4.2% ✅ bytearray_extend_noaspectTime: ✅ 269.093µs (SLO: <400.000µs 📉 -32.7%) vs baseline: -0.9% Memory: ✅ 43.903MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ bytearray_noaspectTime: ✅ 142.044µs (SLO: <300.000µs 📉 -52.7%) vs baseline: +2.0% Memory: ✅ 43.708MB (SLO: <46.000MB -5.0%) vs baseline: +5.0% ✅ bytes_aspectTime: ✅ 216.643µs (SLO: <300.000µs 📉 -27.8%) vs baseline: ~same Memory: ✅ 43.806MB (SLO: <46.000MB -4.8%) vs baseline: +6.2% ✅ bytes_noaspectTime: ✅ 133.787µs (SLO: <200.000µs 📉 -33.1%) vs baseline: ~same Memory: ✅ 43.690MB (SLO: <46.000MB -5.0%) vs baseline: +4.8% ✅ bytesio_aspectTime: ✅ 3.819ms (SLO: <5.000ms 📉 -23.6%) vs baseline: ~same Memory: ✅ 43.602MB (SLO: <46.000MB -5.2%) vs baseline: +5.5% ✅ bytesio_noaspectTime: ✅ 319.996µs (SLO: <420.000µs 📉 -23.8%) vs baseline: ~same Memory: ✅ 43.674MB (SLO: <46.000MB -5.1%) vs baseline: +4.8% ✅ capitalize_aspectTime: ✅ 88.693µs (SLO: <300.000µs 📉 -70.4%) vs baseline: +0.1% Memory: ✅ 43.617MB (SLO: <46.000MB -5.2%) vs baseline: +5.0% ✅ capitalize_noaspectTime: ✅ 258.271µs (SLO: <300.000µs 📉 -13.9%) vs baseline: +2.3% Memory: ✅ 43.629MB (SLO: <46.000MB -5.2%) vs baseline: +4.5% ✅ casefold_aspectTime: ✅ 88.814µs (SLO: <500.000µs 📉 -82.2%) vs baseline: ~same Memory: ✅ 43.660MB (SLO: <46.000MB -5.1%) vs baseline: +5.5% ✅ casefold_noaspectTime: ✅ 310.934µs (SLO: <500.000µs 📉 -37.8%) vs baseline: -1.0% Memory: ✅ 43.624MB (SLO: <46.000MB -5.2%) vs baseline: +4.5% ✅ decode_aspectTime: ✅ 87.342µs (SLO: <100.000µs 📉 -12.7%) vs baseline: +0.1% Memory: ✅ 43.338MB (SLO: <46.000MB -5.8%) vs baseline: +4.1% ✅ decode_noaspectTime: ✅ 156.256µs (SLO: <210.000µs 📉 -25.6%) vs baseline: +0.3% Memory: ✅ 43.739MB (SLO: <46.000MB -4.9%) vs baseline: +5.1% ✅ encode_aspectTime: ✅ 85.329µs (SLO: <200.000µs 📉 -57.3%) vs baseline: +0.5% Memory: ✅ 43.303MB (SLO: <46.000MB -5.9%) vs baseline: +4.3% ✅ encode_noaspectTime: ✅ 145.879µs (SLO: <200.000µs 📉 -27.1%) vs baseline: +0.2% Memory: ✅ 43.699MB (SLO: <46.000MB -5.0%) vs baseline: +4.8% ✅ format_aspectTime: ✅ 14.623ms (SLO: <19.200ms 📉 -23.8%) vs baseline: -0.2% Memory: ✅ 43.618MB (SLO: <46.000MB -5.2%) vs baseline: +4.4% ✅ format_map_aspectTime: ✅ 16.494ms (SLO: <21.500ms 📉 -23.3%) vs baseline: +0.4% Memory: ✅ 43.792MB (SLO: <46.000MB -4.8%) vs baseline: +5.0% ✅ format_map_noaspectTime: ✅ 373.267µs (SLO: <500.000µs 📉 -25.3%) vs baseline: -0.3% Memory: ✅ 43.648MB (SLO: <46.000MB -5.1%) vs baseline: +4.8% ✅ format_noaspectTime: ✅ 309.084µs (SLO: <500.000µs 📉 -38.2%) vs baseline: +0.8% Memory: ✅ 43.693MB (SLO: <46.000MB -5.0%) vs baseline: +4.7% ✅ index_aspectTime: ✅ 124.784µs (SLO: <300.000µs 📉 -58.4%) vs baseline: +4.4% Memory: ✅ 43.670MB (SLO: <46.000MB -5.1%) vs baseline: +5.3% ✅ index_noaspectTime: ✅ 40.412µs (SLO: <300.000µs 📉 -86.5%) vs baseline: +0.3% Memory: ✅ 43.724MB (SLO: <46.000MB -4.9%) vs baseline: +4.9% ✅ join_aspectTime: ✅ 215.298µs (SLO: <300.000µs 📉 -28.2%) vs baseline: -0.1% Memory: ✅ 43.694MB (SLO: <46.000MB -5.0%) vs baseline: +5.7% ✅ join_noaspectTime: ✅ 143.861µs (SLO: <300.000µs 📉 -52.0%) vs baseline: -1.4% Memory: ✅ 43.655MB (SLO: <46.000MB -5.1%) vs baseline: +4.9% ✅ ljust_aspectTime: ✅ 574.957µs (SLO: <700.000µs 📉 -17.9%) vs baseline: 📈 +12.5% Memory: ✅ 43.697MB (SLO: <46.000MB -5.0%) vs baseline: +5.6% ✅ ljust_noaspectTime: ✅ 262.284µs (SLO: <300.000µs 📉 -12.6%) vs baseline: +0.5% Memory: ✅ 43.647MB (SLO: <46.000MB -5.1%) vs baseline: +4.9% ✅ lower_aspectTime: ✅ 302.820µs (SLO: <500.000µs 📉 -39.4%) vs baseline: -0.2% Memory: ✅ 43.668MB (SLO: <46.000MB -5.1%) vs baseline: +5.3% ✅ lower_noaspectTime: ✅ 238.697µs (SLO: <300.000µs 📉 -20.4%) vs baseline: -0.8% Memory: ✅ 43.828MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ lstrip_aspectTime: ✅ 0.278ms (SLO: <3.000ms 📉 -90.7%) vs baseline: -0.8% Memory: ✅ 43.392MB (SLO: <46.000MB -5.7%) vs baseline: +4.2% ✅ lstrip_noaspectTime: ✅ 0.181ms (SLO: <3.000ms 📉 -94.0%) vs baseline: +1.0% Memory: ✅ 43.595MB (SLO: <46.000MB -5.2%) vs baseline: +4.5% ✅ modulo_aspectTime: ✅ 14.294ms (SLO: <18.750ms 📉 -23.8%) vs baseline: -0.3% Memory: ✅ 43.580MB (SLO: <46.000MB -5.3%) vs baseline: +4.6% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.800ms (SLO: <19.350ms 📉 -23.5%) vs baseline: ~same Memory: ✅ 43.768MB (SLO: <46.000MB -4.9%) vs baseline: +4.9% ✅ modulo_aspect_for_bytesTime: ✅ 14.434ms (SLO: <18.900ms 📉 -23.6%) vs baseline: +0.2% Memory: ✅ 43.714MB (SLO: <46.000MB -5.0%) vs baseline: +4.7% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.658ms (SLO: <19.150ms 📉 -23.5%) vs baseline: ~same Memory: ✅ 43.739MB (SLO: <46.000MB -4.9%) vs baseline: +4.7% ✅ modulo_noaspectTime: ✅ 0.362ms (SLO: <3.000ms 📉 -87.9%) vs baseline: -0.2% Memory: ✅ 43.667MB (SLO: <46.000MB -5.1%) vs baseline: +4.8% ✅ replace_aspectTime: ✅ 18.430ms (SLO: <24.000ms 📉 -23.2%) vs baseline: ~same Memory: ✅ 43.759MB (SLO: <46.000MB -4.9%) vs baseline: +5.4% ✅ replace_noaspectTime: ✅ 289.010µs (SLO: <300.000µs -3.7%) vs baseline: +1.3% Memory: ✅ 43.743MB (SLO: <46.000MB -4.9%) vs baseline: +5.0% ✅ repr_aspectTime: ✅ 319.925µs (SLO: <420.000µs 📉 -23.8%) vs baseline: -0.2% Memory: ✅ 43.659MB (SLO: <46.000MB -5.1%) vs baseline: +5.4% ✅ repr_noaspectTime: ✅ 46.676µs (SLO: <90.000µs 📉 -48.1%) vs baseline: -0.4% Memory: ✅ 43.616MB (SLO: <46.000MB -5.2%) vs baseline: +4.8% ✅ rstrip_aspectTime: ✅ 391.587µs (SLO: <500.000µs 📉 -21.7%) vs baseline: -0.6% Memory: ✅ 43.558MB (SLO: <46.000MB -5.3%) vs baseline: +4.6% ✅ rstrip_noaspectTime: ✅ 188.391µs (SLO: <300.000µs 📉 -37.2%) vs baseline: +0.8% Memory: ✅ 43.700MB (SLO: <46.000MB -5.0%) vs baseline: +5.0% ✅ slice_aspectTime: ✅ 182.136µs (SLO: <300.000µs 📉 -39.3%) vs baseline: +0.3% Memory: ✅ 43.681MB (SLO: <46.000MB -5.0%) vs baseline: +5.6% ✅ slice_noaspectTime: ✅ 54.306µs (SLO: <90.000µs 📉 -39.7%) vs baseline: +0.4% Memory: ✅ 43.513MB (SLO: <46.000MB -5.4%) vs baseline: +4.5% ✅ stringio_aspectTime: ✅ 4.399ms (SLO: <5.000ms 📉 -12.0%) vs baseline: 📈 +14.1% Memory: ✅ 43.631MB (SLO: <46.000MB -5.1%) vs baseline: +5.5% ✅ stringio_noaspectTime: ✅ 353.439µs (SLO: <500.000µs 📉 -29.3%) vs baseline: -0.7% Memory: ✅ 43.706MB (SLO: <46.000MB -5.0%) vs baseline: +4.9% ✅ strip_aspectTime: ✅ 278.585µs (SLO: <350.000µs 📉 -20.4%) vs baseline: -0.6% Memory: ✅ 43.582MB (SLO: <46.000MB -5.3%) vs baseline: +5.3% ✅ strip_noaspectTime: ✅ 183.330µs (SLO: <240.000µs 📉 -23.6%) vs baseline: -0.4% Memory: ✅ 43.700MB (SLO: <46.000MB -5.0%) vs baseline: +4.8% ✅ swapcase_aspectTime: ✅ 342.616µs (SLO: <500.000µs 📉 -31.5%) vs baseline: ~same Memory: ✅ 43.701MB (SLO: <46.000MB -5.0%) vs baseline: +5.5% ✅ swapcase_noaspectTime: ✅ 276.306µs (SLO: <400.000µs 📉 -30.9%) vs baseline: -0.1% Memory: ✅ 43.778MB (SLO: <46.000MB -4.8%) vs baseline: +5.2% ✅ title_aspectTime: ✅ 329.399µs (SLO: <500.000µs 📉 -34.1%) vs baseline: -1.0% Memory: ✅ 43.707MB (SLO: <46.000MB -5.0%) vs baseline: +5.6% ✅ title_noaspectTime: ✅ 262.290µs (SLO: <400.000µs 📉 -34.4%) vs baseline: -0.4% Memory: ✅ 43.592MB (SLO: <46.000MB -5.2%) vs baseline: +4.5% ✅ translate_aspectTime: ✅ 504.959µs (SLO: <700.000µs 📉 -27.9%) vs baseline: -0.1% Memory: ✅ 43.696MB (SLO: <46.000MB -5.0%) vs baseline: +5.4% ✅ translate_noaspectTime: ✅ 425.230µs (SLO: <500.000µs 📉 -15.0%) vs baseline: -1.5% Memory: ✅ 43.623MB (SLO: <46.000MB -5.2%) vs baseline: +4.9% ✅ upper_aspectTime: ✅ 302.217µs (SLO: <500.000µs 📉 -39.6%) vs baseline: +0.8% Memory: ✅ 43.569MB (SLO: <46.000MB -5.3%) vs baseline: +5.5% ✅ upper_noaspectTime: ✅ 235.826µs (SLO: <400.000µs 📉 -41.0%) vs baseline: ~same Memory: ✅ 43.680MB (SLO: <46.000MB -5.0%) vs baseline: +5.0% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 518.563µs (SLO: <700.000µs 📉 -25.9%) vs baseline: 📈 +19.6% Memory: ✅ 43.391MB (SLO: <46.000MB -5.7%) vs baseline: +5.0% ✅ ospathbasename_noaspectTime: ✅ 435.968µs (SLO: <700.000µs 📉 -37.7%) vs baseline: +0.4% Memory: ✅ 43.380MB (SLO: <46.000MB -5.7%) vs baseline: +4.3% ✅ ospathjoin_aspectTime: ✅ 619.255µs (SLO: <700.000µs 📉 -11.5%) vs baseline: -0.3% Memory: ✅ 43.391MB (SLO: <46.000MB -5.7%) vs baseline: +4.5% ✅ ospathjoin_noaspectTime: ✅ 628.442µs (SLO: <700.000µs 📉 -10.2%) vs baseline: +0.6% Memory: ✅ 43.450MB (SLO: <46.000MB -5.5%) vs baseline: +5.2% ✅ ospathnormcase_aspectTime: ✅ 346.340µs (SLO: <700.000µs 📉 -50.5%) vs baseline: -0.6% Memory: ✅ 43.304MB (SLO: <46.000MB -5.9%) vs baseline: +4.7% ✅ ospathnormcase_noaspectTime: ✅ 359.547µs (SLO: <700.000µs 📉 -48.6%) vs baseline: +1.1% Memory: ✅ 43.470MB (SLO: <46.000MB -5.5%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 483.616µs (SLO: <700.000µs 📉 -30.9%) vs baseline: -1.1% Memory: ✅ 43.411MB (SLO: <46.000MB -5.6%) vs baseline: +5.0% ✅ ospathsplit_noaspectTime: ✅ 498.581µs (SLO: <700.000µs 📉 -28.8%) vs baseline: ~same Memory: ✅ 43.313MB (SLO: <46.000MB -5.8%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 376.494µs (SLO: <700.000µs 📉 -46.2%) vs baseline: -0.2% Memory: ✅ 43.486MB (SLO: <46.000MB -5.5%) vs baseline: +4.8% ✅ ospathsplitdrive_noaspectTime: ✅ 72.542µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -0.2% Memory: ✅ 43.450MB (SLO: <46.000MB -5.5%) vs baseline: +5.2% ✅ ospathsplitext_aspectTime: ✅ 457.505µs (SLO: <700.000µs 📉 -34.6%) vs baseline: -1.0% Memory: ✅ 43.332MB (SLO: <46.000MB -5.8%) vs baseline: +4.4% ✅ ospathsplitext_noaspectTime: ✅ 470.701µs (SLO: <700.000µs 📉 -32.8%) vs baseline: +0.6% Memory: ✅ 43.352MB (SLO: <46.000MB -5.8%) vs baseline: +4.2% 🟡 Near SLO Breach (1 suite)🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 3.389ms (SLO: <4.750ms 📉 -28.7%) vs baseline: -0.2% Memory: ✅ 55.448MB (SLO: <66.500MB 📉 -16.6%) vs baseline: +5.0% ✅ appsec-postTime: ✅ 2.876ms (SLO: <6.750ms 📉 -57.4%) vs baseline: -0.1% Memory: ✅ 55.747MB (SLO: <66.500MB 📉 -16.2%) vs baseline: +4.7% ✅ appsec-telemetryTime: ✅ 3.408ms (SLO: <4.750ms 📉 -28.2%) vs baseline: +1.2% Memory: ✅ 55.470MB (SLO: <66.500MB 📉 -16.6%) vs baseline: +5.0% ✅ debuggerTime: ✅ 1.844ms (SLO: <2.000ms -7.8%) vs baseline: -0.4% Memory: ✅ 48.943MB (SLO: <51.500MB -5.0%) vs baseline: +4.6% ✅ iast-getTime: ✅ 1.840ms (SLO: <2.000ms -8.0%) vs baseline: ~same Memory: ✅ 46.052MB (SLO: <49.000MB -6.0%) vs baseline: +5.2% ✅ profilerTime: ✅ 1.881ms (SLO: <2.100ms 📉 -10.4%) vs baseline: ~same Memory: ✅ 51.916MB (SLO: <52.500MB 🟡 -1.1%) vs baseline: +5.1% ✅ resource-renamingTime: ✅ 3.379ms (SLO: <3.650ms -7.4%) vs baseline: +0.2% Memory: ✅ 55.378MB (SLO: <60.000MB -7.7%) vs baseline: +4.7% ✅ tracerTime: ✅ 3.387ms (SLO: <3.650ms -7.2%) vs baseline: +0.4% Memory: ✅ 55.429MB (SLO: <60.000MB -7.6%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 3.370ms (SLO: <3.650ms -7.7%) vs baseline: -0.4% Memory: ✅ 55.396MB (SLO: <60.000MB -7.7%) vs baseline: +4.8%
|
Backport #17942 to 4.6