ci: use different git repo id for releases [backport 4.9]#17971
ci: use different git repo id for releases [backport 4.9]#17971dd-octo-sts[bot] wants to merge 1 commit into
Conversation
## Description When a flaky test is detected on a release branch (e.g. `4.8`, `4.10`), it is quarantined across **all** branches — including `main`. This causes problems in two directions: 1. **Release branches don't receive flaky-test-fix backports from `main`**, so a test that gets fixed and unquarantined on `main` may remain quarantined (or vice versa) on a release branch indefinitely. 2. **New flakes introduced during release activity** (cherry-picks, backports, release-specific patches) pollute the main pool and can suppress test failures on `main` and feature branches. This change uses a separate [Datadog Test Optimization](https://docs.datadoghq.com/tests/) Flaky Test Management pool for release branches by overriding `DD_GIT_REPOSITORY_URL` to `https://github.com/DataDog/dd-trace-py-release.git` in GitLab CI pipelines that run on release branches. The Datadog backend uses `git.repository_url` (surfaced in the UI as `@git.repository.id_v2`) as the partition key for Flaky Test Management. By giving release-branch pipelines a distinct URL, their quarantine list is fully independent from the one used by `main`, merge queues, and feature branches. ### How it works - **Release branches** (`X.Y` / `X.YZ` pattern, e.g. `4.8`, `4.10`): `DD_GIT_REPOSITORY_URL` is set to `https://github.com/DataDog/dd-trace-py-release.git` vian`workflow:rules:variables`, which overrides the global variable for the entire pipeline. - **All other pipelines** (`main`, merge queues, feature branches): continue using `https://github.com/DataDog/dd-trace-py.git` unchanged. ## Testing The change is CI-only. It can be validated by triggering a pipeline on a release branch (e.g. `4.8`) and confirming that test results appear under `@git.repository.id_v2:github.com/datadog/dd-trace-py-release` in the Datadog Test Optimization UI, while a pipeline on `main` still shows `@git.repository.id_v2:github.com/datadog/dd-trace-py`. ## Risks - **Quarantine list starts empty for release branches.** Any tests already quarantined on release branches under the old pool ID will not carry over — they will re-run and be re-detected as flaky. This is a one-time, bounded cost. - No application code is changed; no test behaviour is affected outside of the flaky-test quarantine logic. ## Additional Notes `workflow:rules:variables` in GitLab CI takes precedence over the global `variables:` block, so the override is not silently clobbered. `DD_GIT_REPOSITORY_URL` is not redefined in any included job template or project-level CI/CD variable, so the behaviour is deterministic. Co-authored-by: federico.mon <federico.mon@datadoghq.com> (cherry picked from commit e3bdccd) Co-authored-by: Federico Mon <federico.mon@datadoghq.com>
Codeowners resolved as |
Performance SLOsComparing candidate unknown (unknown) with baseline 4.9 (313049c) 📈 Performance Regressions (1 suite)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 500.197µs (SLO: <700.000µs 📉 -28.5%) vs baseline: 📈 +22.8% Memory: ✅ 43.871MB (SLO: <46.000MB -4.6%) vs baseline: +4.7% ✅ ospathbasename_noaspectTime: ✅ 421.163µs (SLO: <700.000µs 📉 -39.8%) vs baseline: +0.6% Memory: ✅ 43.921MB (SLO: <46.000MB -4.5%) vs baseline: +5.0% ✅ ospathjoin_aspectTime: ✅ 613.225µs (SLO: <700.000µs 📉 -12.4%) vs baseline: +0.2% Memory: ✅ 43.887MB (SLO: <46.000MB -4.6%) vs baseline: +4.8% ✅ ospathjoin_noaspectTime: ✅ 619.182µs (SLO: <700.000µs 📉 -11.5%) vs baseline: -0.3% Memory: ✅ 43.933MB (SLO: <46.000MB -4.5%) vs baseline: +5.2% ✅ ospathnormcase_aspectTime: ✅ 336.232µs (SLO: <700.000µs 📉 -52.0%) vs baseline: +0.2% Memory: ✅ 43.949MB (SLO: <46.000MB -4.5%) vs baseline: +4.9% ✅ ospathnormcase_noaspectTime: ✅ 344.806µs (SLO: <700.000µs 📉 -50.7%) vs baseline: ~same Memory: ✅ 43.889MB (SLO: <46.000MB -4.6%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 475.985µs (SLO: <700.000µs 📉 -32.0%) vs baseline: ~same Memory: ✅ 43.883MB (SLO: <46.000MB -4.6%) vs baseline: +4.8% ✅ ospathsplit_noaspectTime: ✅ 486.970µs (SLO: <700.000µs 📉 -30.4%) vs baseline: ~same Memory: ✅ 43.991MB (SLO: <46.000MB -4.4%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 356.313µs (SLO: <700.000µs 📉 -49.1%) vs baseline: +0.2% Memory: ✅ 43.872MB (SLO: <46.000MB -4.6%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 72.203µs (SLO: <700.000µs 📉 -89.7%) vs baseline: -0.4% Memory: ✅ 43.894MB (SLO: <46.000MB -4.6%) vs baseline: +4.7% ✅ ospathsplitext_aspectTime: ✅ 445.105µs (SLO: <700.000µs 📉 -36.4%) vs baseline: +0.8% Memory: ✅ 43.930MB (SLO: <46.000MB -4.5%) vs baseline: +4.9% ✅ ospathsplitext_noaspectTime: ✅ 448.968µs (SLO: <700.000µs 📉 -35.9%) vs baseline: -0.7% Memory: ✅ 43.903MB (SLO: <46.000MB -4.6%) vs baseline: +4.7% 🟡 Near SLO Breach (5 suites)🟡 djangosimple - 28/28✅ appsecTime: ✅ 19.747ms (SLO: <22.300ms 📉 -11.4%) vs baseline: +0.6% Memory: ✅ 71.290MB (SLO: <73.500MB -3.0%) vs baseline: +4.8% ✅ exception-replay-enabledTime: ✅ 1.368ms (SLO: <1.450ms -5.7%) vs baseline: -0.4% Memory: ✅ 69.540MB (SLO: <71.500MB -2.7%) vs baseline: +4.8% ✅ iastTime: ✅ 19.781ms (SLO: <22.250ms 📉 -11.1%) vs baseline: +0.6% Memory: ✅ 71.231MB (SLO: <75.000MB -5.0%) vs baseline: +4.8% ✅ profilerTime: ✅ 15.233ms (SLO: <16.550ms -8.0%) vs baseline: +0.7% Memory: ✅ 60.092MB (SLO: <61.000MB 🟡 -1.5%) vs baseline: +5.0% ✅ resource-renamingTime: ✅ 19.519ms (SLO: <21.750ms 📉 -10.3%) vs baseline: -0.2% Memory: ✅ 71.310MB (SLO: <73.500MB -3.0%) vs baseline: +4.9% ✅ span-code-originTime: ✅ 19.987ms (SLO: <28.200ms 📉 -29.1%) vs baseline: +0.2% Memory: ✅ 71.290MB (SLO: <75.000MB -4.9%) vs baseline: +4.9% ✅ tracerTime: ✅ 19.610ms (SLO: <21.750ms -9.8%) vs baseline: ~same Memory: ✅ 71.369MB (SLO: <75.000MB -4.8%) vs baseline: +5.0% ✅ tracer-and-profilerTime: ✅ 21.054ms (SLO: <23.500ms 📉 -10.4%) vs baseline: +0.3% Memory: ✅ 73.256MB (SLO: <75.000MB -2.3%) vs baseline: +5.0% ✅ tracer-dont-create-db-spansTime: ✅ 19.774ms (SLO: <21.500ms -8.0%) vs baseline: +0.1% Memory: ✅ 71.231MB (SLO: <75.000MB -5.0%) vs baseline: +4.7% ✅ tracer-minimalTime: ✅ 16.651ms (SLO: <18.500ms -10.0%) vs baseline: -0.4% Memory: ✅ 71.369MB (SLO: <75.000MB -4.8%) vs baseline: +5.0% ✅ tracer-no-cachesTime: ✅ 17.465ms (SLO: <19.650ms 📉 -11.1%) vs baseline: -0.4% Memory: ✅ 71.349MB (SLO: <75.000MB -4.9%) vs baseline: +4.9% ✅ tracer-no-databasesTime: ✅ 19.320ms (SLO: <21.100ms -8.4%) vs baseline: +0.1% Memory: ✅ 71.290MB (SLO: <75.000MB -4.9%) vs baseline: +4.9% ✅ tracer-no-middlewareTime: ✅ 19.434ms (SLO: <21.500ms -9.6%) vs baseline: -0.1% Memory: ✅ 71.290MB (SLO: <75.000MB -4.9%) vs baseline: +4.8% ✅ tracer-no-templatesTime: ✅ 19.554ms (SLO: <22.000ms 📉 -11.1%) vs baseline: +0.6% Memory: ✅ 71.310MB (SLO: <73.500MB -3.0%) vs baseline: +4.8% 🟡 forktime - 4/4✅ baselineTime: ✅ 1.944ms (SLO: <3.000ms 📉 -35.2%) vs baseline: +3.4% Memory: ✅ 29.196MB (SLO: <33.000MB 📉 -11.5%) vs baseline: +4.2% ✅ configuredTime: ✅ 9.391ms (SLO: <17.000ms 📉 -44.8%) vs baseline: +0.3% Memory: ✅ 58.969MB (SLO: <60.000MB 🟡 -1.7%) vs baseline: +4.8% 🟡 iastpropagation - 8/8✅ no-propagationTime: ✅ 48.882µs (SLO: <60.000µs 📉 -18.5%) vs baseline: +0.4% Memory: ✅ 41.071MB (SLO: <42.000MB -2.2%) vs baseline: +4.8% ✅ propagation_enabledTime: ✅ 135.653µs (SLO: <190.000µs 📉 -28.6%) vs baseline: -0.7% Memory: ✅ 41.130MB (SLO: <42.000MB -2.1%) vs baseline: +4.8% ✅ propagation_enabled_100Time: ✅ 1.555ms (SLO: <2.300ms 📉 -32.4%) vs baseline: -0.4% Memory: ✅ 41.209MB (SLO: <42.000MB 🟡 -1.9%) vs baseline: +5.1% ✅ propagation_enabled_1000Time: ✅ 29.004ms (SLO: <34.550ms 📉 -16.1%) vs baseline: +0.5% Memory: ✅ 41.091MB (SLO: <42.000MB -2.2%) vs baseline: +4.8% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 41.511ms (SLO: <47.150ms 📉 -12.0%) vs baseline: -0.7% Memory: ✅ 41.560MB (SLO: <47.000MB 📉 -11.6%) vs baseline: +4.9% ✅ add-metricsTime: ✅ 233.936ms (SLO: <344.800ms 📉 -32.2%) vs baseline: -1.1% Memory: ✅ 45.561MB (SLO: <47.500MB -4.1%) vs baseline: +4.9% ✅ add-tagsTime: ✅ 266.546ms (SLO: <330.000ms 📉 -19.2%) vs baseline: +0.6% Memory: ✅ 45.676MB (SLO: <47.500MB -3.8%) vs baseline: +4.9% ✅ get-contextTime: ✅ 81.209ms (SLO: <92.350ms 📉 -12.1%) vs baseline: +0.2% Memory: ✅ 41.366MB (SLO: <46.500MB 📉 -11.0%) vs baseline: +4.8% ✅ is-recordingTime: ✅ 38.011ms (SLO: <44.500ms 📉 -14.6%) vs baseline: +0.3% Memory: ✅ 41.119MB (SLO: <47.500MB 📉 -13.4%) vs baseline: +5.1% ✅ record-exceptionTime: ✅ 63.110ms (SLO: <67.650ms -6.7%) vs baseline: -0.5% Memory: ✅ 41.829MB (SLO: <47.000MB 📉 -11.0%) vs baseline: +4.6% ✅ set-statusTime: ✅ 43.767ms (SLO: <50.400ms 📉 -13.2%) vs baseline: +0.7% Memory: ✅ 41.092MB (SLO: <47.000MB 📉 -12.6%) vs baseline: +4.9% ✅ startTime: ✅ 38.933ms (SLO: <44.500ms 📉 -12.5%) vs baseline: +4.8% Memory: ✅ 41.087MB (SLO: <47.000MB 📉 -12.6%) vs baseline: +5.1% ✅ start-finishTime: ✅ 89.990ms (SLO: <92.000ms -2.2%) vs baseline: ~same Memory: ✅ 38.948MB (SLO: <46.500MB 📉 -16.2%) vs baseline: +4.5% ✅ start-finish-telemetryTime: ✅ 91.327ms (SLO: <93.000ms 🟡 -1.8%) vs baseline: ~same Memory: ✅ 39.066MB (SLO: <46.500MB 📉 -16.0%) vs baseline: +5.0% ✅ update-nameTime: ✅ 39.075ms (SLO: <45.150ms 📉 -13.5%) vs baseline: +0.2% Memory: ✅ 41.121MB (SLO: <47.000MB 📉 -12.5%) vs baseline: +4.6% 🟡 packagesupdateimporteddependencies - 24/24 (1 unstable)✅ import_manyTime: ✅ 169.375µs (SLO: <170.000µs 🟡 -0.4%) vs baseline: +0.3% Memory: ✅ 41.151MB (SLO: <46.000MB 📉 -10.5%) vs baseline: +4.3% ✅ import_many_cachedTime: ✅ 132.554µs (SLO: <170.000µs 📉 -22.0%) vs baseline: +0.4% Memory: ✅ 41.332MB (SLO: <46.000MB 📉 -10.1%) vs baseline: +4.7% ✅ import_many_stdlibTime: ✅ 1.261ms (SLO: <1.750ms 📉 -28.0%) vs baseline: ~same Memory: ✅ 41.412MB (SLO: <46.000MB -10.0%) vs baseline: +5.2%
|
Backport #17942 to 4.9