Create configure_tests convention plugin
#9859
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sarahchen6
added
type: enhancement
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 11 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1047218
Total [baseline] (10.729 s) : 0, 10728831
Agent [candidate] (1.053 s) : 0, 1052506
Total [candidate] (10.797 s) : 0, 10796779
section appsec
Agent [baseline] (1.221 s) : 0, 1220806
Total [baseline] (10.813 s) : 0, 10813134
Agent [candidate] (1.231 s) : 0, 1230802
Total [candidate] (10.89 s) : 0, 10890404
section iast
Agent [baseline] (1.189 s) : 0, 1188905
Total [baseline] (11.115 s) : 0, 11115196
Agent [candidate] (1.182 s) : 0, 1182350
Total [candidate] (11.22 s) : 0, 11219525
section profiling
Agent [baseline] (1.194 s) : 0, 1193732
Total [baseline] (10.96 s) : 0, 10959503
Agent [candidate] (1.202 s) : 0, 1201797
Total [candidate] (10.897 s) : 0, 10897218
gantt
title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.451 ms) : 0, 1451
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (705.214 ms) : 0, 705214
BytebuddyAgent [candidate] (709.74 ms) : 0, 709740
GlobalTracer [baseline] (246.164 ms) : 0, 246164
GlobalTracer [candidate] (246.599 ms) : 0, 246599
AppSec [baseline] (32.274 ms) : 0, 32274
AppSec [candidate] (32.343 ms) : 0, 32343
Debugger [baseline] (6.406 ms) : 0, 6406
Debugger [candidate] (6.367 ms) : 0, 6367
Remote Config [baseline] (723.365 µs) : 0, 723
Remote Config [candidate] (706.022 µs) : 0, 706
Telemetry [baseline] (15.235 ms) : 0, 15235
Telemetry [candidate] (12.331 ms) : 0, 12331
Flare Poller [baseline] (5.002 ms) : 0, 5002
Flare Poller [candidate] (8.172 ms) : 0, 8172
section appsec
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.469 ms) : 0, 1469
BytebuddyAgent [baseline] (728.61 ms) : 0, 728610
BytebuddyAgent [candidate] (734.962 ms) : 0, 734962
GlobalTracer [baseline] (237.141 ms) : 0, 237141
GlobalTracer [candidate] (239.158 ms) : 0, 239158
AppSec [baseline] (174.699 ms) : 0, 174699
AppSec [candidate] (175.775 ms) : 0, 175775
Debugger [baseline] (6.015 ms) : 0, 6015
Debugger [candidate] (6.028 ms) : 0, 6028
Remote Config [baseline] (652.566 µs) : 0, 653
Remote Config [candidate] (659.807 µs) : 0, 660
Telemetry [baseline] (8.564 ms) : 0, 8564
Telemetry [candidate] (8.57 ms) : 0, 8570
Flare Poller [baseline] (4.066 ms) : 0, 4066
Flare Poller [candidate] (3.993 ms) : 0, 3993
IAST [baseline] (24.697 ms) : 0, 24697
IAST [candidate] (25.164 ms) : 0, 25164
section iast
crashtracking [baseline] (1.483 ms) : 0, 1483
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (835.704 ms) : 0, 835704
BytebuddyAgent [candidate] (829.099 ms) : 0, 829099
GlobalTracer [baseline] (235.859 ms) : 0, 235859
GlobalTracer [candidate] (236.218 ms) : 0, 236218
AppSec [baseline] (27.087 ms) : 0, 27087
AppSec [candidate] (28.958 ms) : 0, 28958
Debugger [baseline] (6.013 ms) : 0, 6013
Debugger [candidate] (6.014 ms) : 0, 6014
Remote Config [baseline] (600.032 µs) : 0, 600
Remote Config [candidate] (614.286 µs) : 0, 614
Telemetry [baseline] (8.481 ms) : 0, 8481
Telemetry [candidate] (8.4 ms) : 0, 8400
Flare Poller [baseline] (4.169 ms) : 0, 4169
Flare Poller [candidate] (4.155 ms) : 0, 4155
IAST [baseline] (34.665 ms) : 0, 34665
IAST [candidate] (32.754 ms) : 0, 32754
section profiling
crashtracking [baseline] (1.435 ms) : 0, 1435
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (729.686 ms) : 0, 729686
BytebuddyAgent [candidate] (734.662 ms) : 0, 734662
GlobalTracer [baseline] (222.073 ms) : 0, 222073
GlobalTracer [candidate] (223.58 ms) : 0, 223580
AppSec [baseline] (32.197 ms) : 0, 32197
AppSec [candidate] (32.499 ms) : 0, 32499
Debugger [baseline] (9.023 ms) : 0, 9023
Debugger [candidate] (6.792 ms) : 0, 6792
Remote Config [baseline] (707.79 µs) : 0, 708
Remote Config [candidate] (1.423 ms) : 0, 1423
Telemetry [baseline] (14.05 ms) : 0, 14050
Telemetry [candidate] (15.565 ms) : 0, 15565
Flare Poller [baseline] (4.098 ms) : 0, 4098
Flare Poller [candidate] (4.16 ms) : 0, 4160
ProfilingAgent [baseline] (110.692 ms) : 0, 110692
ProfilingAgent [candidate] (111.359 ms) : 0, 111359
Profiling [baseline] (111.346 ms) : 0, 111346
Profiling [candidate] (112.048 ms) : 0, 112048
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063487
Total [baseline] (8.663 s) : 0, 8662916
Agent [candidate] (1.056 s) : 0, 1055585
Total [candidate] (8.633 s) : 0, 8632512
section iast
Agent [baseline] (1.179 s) : 0, 1178597
Total [baseline] (9.273 s) : 0, 9272870
Agent [candidate] (1.178 s) : 0, 1177760
Total [candidate] (9.25 s) : 0, 9250126
gantt
title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.476 ms) : 0, 1476
crashtracking [candidate] (1.483 ms) : 0, 1483
BytebuddyAgent [baseline] (716.152 ms) : 0, 716152
BytebuddyAgent [candidate] (710.734 ms) : 0, 710734
GlobalTracer [baseline] (249.147 ms) : 0, 249147
GlobalTracer [candidate] (247.694 ms) : 0, 247694
AppSec [baseline] (32.862 ms) : 0, 32862
AppSec [candidate] (32.791 ms) : 0, 32791
Debugger [baseline] (6.567 ms) : 0, 6567
Debugger [candidate] (6.429 ms) : 0, 6429
Remote Config [baseline] (723.937 µs) : 0, 724
Remote Config [candidate] (717.803 µs) : 0, 718
Telemetry [baseline] (13.177 ms) : 0, 13177
Telemetry [candidate] (13.582 ms) : 0, 13582
Flare Poller [baseline] (8.304 ms) : 0, 8304
Flare Poller [candidate] (7.345 ms) : 0, 7345
section iast
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (827.863 ms) : 0, 827863
BytebuddyAgent [candidate] (826.972 ms) : 0, 826972
GlobalTracer [baseline] (234.282 ms) : 0, 234282
GlobalTracer [candidate] (234.296 ms) : 0, 234296
AppSec [baseline] (27.89 ms) : 0, 27890
AppSec [candidate] (30.473 ms) : 0, 30473
Debugger [baseline] (6.0 ms) : 0, 6000
Debugger [candidate] (5.988 ms) : 0, 5988
Remote Config [baseline] (597.231 µs) : 0, 597
Remote Config [candidate] (596.588 µs) : 0, 597
Telemetry [baseline] (8.418 ms) : 0, 8418
Telemetry [candidate] (8.387 ms) : 0, 8387
Flare Poller [baseline] (4.091 ms) : 0, 4091
Flare Poller [candidate] (4.194 ms) : 0, 4194
IAST [baseline] (33.232 ms) : 0, 33232
IAST [candidate] (30.672 ms) : 0, 30672
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section baseline
no_agent (1.228 ms) : 1216, 1241
. : milestone, 1228,
iast (3.212 ms) : 3165, 3259
. : milestone, 3212,
iast_FULL (5.82 ms) : 5761, 5878
. : milestone, 5820,
iast_GLOBAL (3.706 ms) : 3637, 3775
. : milestone, 3706,
profiling (1.924 ms) : 1907, 1940
. : milestone, 1924,
tracing (1.766 ms) : 1752, 1779
. : milestone, 1766,
section candidate
no_agent (1.212 ms) : 1200, 1224
. : milestone, 1212,
iast (3.186 ms) : 3149, 3222
. : milestone, 3186,
iast_FULL (5.895 ms) : 5835, 5954
. : milestone, 5895,
iast_GLOBAL (3.633 ms) : 3578, 3688
. : milestone, 3633,
profiling (1.973 ms) : 1957, 1990
. : milestone, 1973,
tracing (1.804 ms) : 1789, 1820
. : milestone, 1804,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section baseline
no_agent (17.409 ms) : 17234, 17585
. : milestone, 17409,
appsec (18.653 ms) : 18464, 18842
. : milestone, 18653,
code_origins (17.88 ms) : 17705, 18055
. : milestone, 17880,
iast (17.911 ms) : 17730, 18092
. : milestone, 17911,
profiling (18.869 ms) : 18679, 19059
. : milestone, 18869,
tracing (18.825 ms) : 18637, 19014
. : milestone, 18825,
section candidate
no_agent (18.214 ms) : 18024, 18404
. : milestone, 18214,
appsec (18.665 ms) : 18477, 18852
. : milestone, 18665,
code_origins (18.011 ms) : 17833, 18190
. : milestone, 18011,
iast (17.88 ms) : 17699, 18062
. : milestone, 17880,
profiling (18.902 ms) : 18712, 19093
. : milestone, 18902,
tracing (17.788 ms) : 17610, 17967
. : milestone, 17788,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section baseline
no_agent (15.523 s) : 15523000, 15523000
. : milestone, 15523000,
appsec (14.721 s) : 14721000, 14721000
. : milestone, 14721000,
iast (18.582 s) : 18582000, 18582000
. : milestone, 18582000,
iast_GLOBAL (17.778 s) : 17778000, 17778000
. : milestone, 17778000,
profiling (15.13 s) : 15130000, 15130000
. : milestone, 15130000,
tracing (14.718 s) : 14718000, 14718000
. : milestone, 14718000,
section candidate
no_agent (14.851 s) : 14851000, 14851000
. : milestone, 14851000,
appsec (14.929 s) : 14929000, 14929000
. : milestone, 14929000,
iast (18.311 s) : 18311000, 18311000
. : milestone, 18311000,
iast_GLOBAL (18.0 s) : 18000000, 18000000
. : milestone, 18000000,
profiling (15.782 s) : 15782000, 15782000
. : milestone, 15782000,
tracing (14.841 s) : 14841000, 14841000
. : milestone, 14841000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~97348c5470, baseline=1.56.0-SNAPSHOT~5743cd5c9d
dateFormat X
axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
. : milestone, 1474,
appsec (3.698 ms) : 3481, 3916
. : milestone, 3698,
iast (2.224 ms) : 2160, 2288
. : milestone, 2224,
iast_GLOBAL (2.264 ms) : 2200, 2328
. : milestone, 2264,
profiling (2.077 ms) : 2024, 2131
. : milestone, 2077,
tracing (2.032 ms) : 1983, 2082
. : milestone, 2032,
section candidate
no_agent (1.476 ms) : 1465, 1488
. : milestone, 1476,
appsec (3.661 ms) : 3446, 3876
. : milestone, 3661,
iast (2.21 ms) : 2146, 2274
. : milestone, 2210,
iast_GLOBAL (2.258 ms) : 2195, 2322
. : milestone, 2258,
profiling (2.079 ms) : 2026, 2132
. : milestone, 2079,
tracing (2.042 ms) : 1992, 2092
. : milestone, 2042,
|
Contributor
|
🎯 Code Coverage 🔗 Commit SHA: 97348c5 | Docs | Datadog PR Page | Was this helpful? Give us feedback! |
Kafka / producer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
Kafka / consumer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
sarahchen6
commented
Oct 27, 2025
sarahchen6
commented
Oct 27, 2025
sarahchen6
force-pushed
the
sarahchen6/configure-tests-convention-plugin
branch
from
24e44ed to
be41722
Compare
AlexeyKuznetsov-DD
requested changes
Oct 27, 2025
bric3
reviewed
Oct 28, 2025
Comment on lines
51
to
57
| val activePartitionProvider = providers.provider { | ||
| project.extra.properties["activePartition"] as? Boolean ?: true | ||
| } | ||
|
|
||
| // Go through the Test tasks and configure them | ||
| tasks.withType(Test::class.java).configureEach { | ||
| enabled = activePartitionProvider.get() |
Contributor
There was a problem hiding this comment.
thought: I wonder if this "trait" or "feature" should be moved over the ci convention plugin ?
Contributor
Author
There was a problem hiding this comment.
Do you mean this entire tasks.withType<Test>() block in the ci-jobs convention plugin? I think it makes sense to keep this here in that case because this section is more just configuring the tests (configure-tests plugin) with some CI input... I guess if we want all things CI-related in the ci-jobs plugin though, we could move this there?
Contributor
There was a problem hiding this comment.
The idea of the active partition is a concept of the ci-jobs and actually the main purpose of ci-jobs. Using the partition here feels "non encapsulated".
Also it's perfectly normal that some plugin can affect tasks behavior. That said, we can do it in another PR. Because the ci-jobs plugin is applied to the root project, while the configure-tests is applied to each sub-projects.
Contributor
Author
There was a problem hiding this comment.
Ohhh okay I see. Agreed. For some reason I thought you were referring to this entire test block 😅
Comment on lines
+47
to
+53
| val skipTestsProvider = rootProject.providers.gradleProperty("skipTests") | ||
| val skipForkedTestsProvider = rootProject.providers.gradleProperty("skipForkedTests") | ||
| val skipFlakyTestsProvider = rootProject.providers.gradleProperty("skipFlakyTests") | ||
| val runFlakyTestsProvider = rootProject.providers.gradleProperty("runFlakyTests") | ||
| val activePartitionProvider = providers.provider { | ||
| project.extra.properties["activePartition"] as? Boolean ?: true | ||
| } |
Contributor
There was a problem hiding this comment.
note: Maybe, these could end-up in the extension I mentioned earlier. I'm kind if undecided there.
Contributor
|
Cool, I have been eying to convert this script. Looking further down the road, I believe this could be split in two, as some tests config might only make sense for integrations, not so much for other components. |
sarahchen6
requested review from
AlexeyKuznetsov-DD,
bric3 and
nikita-tkachenko-datadog
bric3
approved these changes
Nov 6, 2025
| import java.time.Duration | ||
| import java.time.temporal.ChronoUnit | ||
|
|
||
| val isTestingInstrumentation = project.findProperty("testingInstrumentation") as? Boolean ?: false |
Contributor
There was a problem hiding this comment.
note: This is a good trade-off today, but ultimately we should avoid extra properties. In the future, when we have convention plugins everywhere, it might be better to have a "simple" conventions for "testing frameworks instrumentation".
AlexeyKuznetsov-DD
approved these changes
Nov 6, 2025
This was referenced Nov 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
This PR turns the
configure_tests.gradlefile into a convention plugin. It is also introduces the testInstrumentation extension that is added to relevant instrumentationbuild.gradlefiles. This helps separate the build and instrumentation logic, adding to our efforts to modularize and organize the build system. The PR also makes a few lazy API improvements along the way.Motivation
Improve our build!
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: https://datadoghq.atlassian.net/browse/APMLP-623