Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update system-tests to 06cbfad5f9facbc6e258d9c8208c466cd46efeea and use Python 3.12 #7297

Merged
merged 7 commits into from
Jul 15, 2024
Merged

Update system-tests to 06cbfad5f9facbc6e258d9c8208c466cd46efeea and use Python 3.12 #7297

merged 7 commits into from
Jul 15, 2024

Conversation

smola
Copy link
Member

@smola smola commented Jul 9, 2024

What Does This Do

Motivation

Additional Notes

Based on DataDog/dd-trace-php#2748

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Jul 9, 2024
@pr-commenter
Copy link

pr-commenter bot commented Jul 9, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/update-system-tests-20240709
git_commit_date 1720794919 1720795605
git_commit_sha 0e5e274 ab7632a
release_version 1.38.0-SNAPSHOT~0e5e2749b0 1.38.0-SNAPSHOT~ab7632ac0d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1720798012 1720798012
ci_job_id 571000234 571000234
ci_pipeline_id 39044794 39044794
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 11 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:iast:AppSec better
[-6.290ms; -1.362ms] or [-12.316%; -2.666%]		 47.251ms 51.077ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063364
Total [baseline] (8.512 s) : 0, 8512199
Agent [candidate] (1.077 s) : 0, 1076680
Total [candidate] (8.595 s) : 0, 8594515
section iast
Agent [baseline] (1.178 s) : 0, 1178186
Total [baseline] (8.967 s) : 0, 8967013
Agent [candidate] (1.173 s) : 0, 1172847
Total [candidate] (8.975 s) : 0, 8974934
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.179 s) : 0, 1178981
Total [baseline] (8.961 s) : 0, 8961458
Agent [candidate] (1.175 s) : 0, 1175207
Total [candidate] (8.963 s) : 0, 8962563
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1168516
Total [baseline] (8.961 s) : 0, 8960883
Agent [candidate] (1.168 s) : 0, 1168009
Total [candidate] (8.968 s) : 0, 8968309
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.178 s 114.822 ms (10.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.179 s 115.617 ms (10.9%)
Agent iast_TELEMETRY_OFF 1.169 s 105.152 ms (9.9%)
Total tracing 8.512 s -
Total iast 8.967 s 454.814 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.961 s 449.26 ms (5.3%)
Total iast_TELEMETRY_OFF 8.961 s 448.684 ms (5.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.173 s 96.167 ms (8.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.175 s 98.527 ms (9.2%)
Agent iast_TELEMETRY_OFF 1.168 s 91.328 ms (8.5%)
Total tracing 8.595 s -
Total iast 8.975 s 380.42 ms (4.4%)
Total iast_HARDCODED_SECRET_DISABLED 8.963 s 368.048 ms (4.3%)
Total iast_TELEMETRY_OFF 8.968 s 373.794 ms (4.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.522 ms) : 0, 665522
BytebuddyAgent [candidate] (673.967 ms) : 0, 673967
GlobalTracer [baseline] (305.234 ms) : 0, 305234
GlobalTracer [candidate] (308.402 ms) : 0, 308402
AppSec [baseline] (49.827 ms) : 0, 49827
AppSec [candidate] (50.919 ms) : 0, 50919
Remote Config [baseline] (678.574 µs) : 0, 679
Remote Config [candidate] (690.741 µs) : 0, 691
Telemetry [baseline] (7.618 ms) : 0, 7618
Telemetry [candidate] (7.731 ms) : 0, 7731
section iast
BytebuddyAgent [baseline] (786.476 ms) : 0, 786476
BytebuddyAgent [candidate] (780.485 ms) : 0, 780485
GlobalTracer [baseline] (296.177 ms) : 0, 296177
GlobalTracer [candidate] (296.164 ms) : 0, 296164
AppSec [baseline] (48.932 ms) : 0, 48932
AppSec [candidate] (50.501 ms) : 0, 50501
Remote Config [baseline] (569.653 µs) : 0, 570
Remote Config [candidate] (566.622 µs) : 0, 567
Telemetry [baseline] (6.991 ms) : 0, 6991
Telemetry [candidate] (6.941 ms) : 0, 6941
IAST [baseline] (25.459 ms) : 0, 25459
IAST [candidate] (24.708 ms) : 0, 24708
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (786.191 ms) : 0, 786191
BytebuddyAgent [candidate] (782.056 ms) : 0, 782056
GlobalTracer [baseline] (296.859 ms) : 0, 296859
GlobalTracer [candidate] (295.899 ms) : 0, 295899
AppSec [baseline] (49.208 ms) : 0, 49208
AppSec [candidate] (48.162 ms) : 0, 48162
Remote Config [baseline] (567.183 µs) : 0, 567
Remote Config [candidate] (565.189 µs) : 0, 565
Telemetry [baseline] (7.016 ms) : 0, 7016
Telemetry [candidate] (7.776 ms) : 0, 7776
IAST [baseline] (25.487 ms) : 0, 25487
IAST [candidate] (27.173 ms) : 0, 27173
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (778.289 ms) : 0, 778289
BytebuddyAgent [candidate] (777.352 ms) : 0, 777352
GlobalTracer [baseline] (295.246 ms) : 0, 295246
GlobalTracer [candidate] (294.919 ms) : 0, 294919
AppSec [baseline] (47.281 ms) : 0, 47281
AppSec [candidate] (47.382 ms) : 0, 47382
Remote Config [baseline] (574.736 µs) : 0, 575
Remote Config [candidate] (594.673 µs) : 0, 595
Telemetry [baseline] (6.824 ms) : 0, 6824
Telemetry [candidate] (6.781 ms) : 0, 6781
IAST [baseline] (26.758 ms) : 0, 26758
IAST [candidate] (27.418 ms) : 0, 27418
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.073 s) : 0, 1073108
Total [baseline] (10.401 s) : 0, 10400635
Agent [candidate] (1.064 s) : 0, 1064104
Total [candidate] (10.347 s) : 0, 10347239
section appsec
Agent [baseline] (1.185 s) : 0, 1184587
Total [baseline] (10.491 s) : 0, 10490964
Agent [candidate] (1.189 s) : 0, 1189021
Total [candidate] (10.538 s) : 0, 10537806
section iast
Agent [baseline] (1.173 s) : 0, 1173408
Total [baseline] (10.676 s) : 0, 10676182
Agent [candidate] (1.169 s) : 0, 1168735
Total [candidate] (10.792 s) : 0, 10792160
section profiling
Agent [baseline] (1.266 s) : 0, 1266053
Total [baseline] (10.597 s) : 0, 10597431
Agent [candidate] (1.267 s) : 0, 1266578
Total [candidate] (10.58 s) : 0, 10580183
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.073 s -
Agent appsec 1.185 s 111.479 ms (10.4%)
Agent iast 1.173 s 100.299 ms (9.3%)
Agent profiling 1.266 s 192.944 ms (18.0%)
Total tracing 10.401 s -
Total appsec 10.491 s 90.329 ms (0.9%)
Total iast 10.676 s 275.547 ms (2.6%)
Total profiling 10.597 s 196.796 ms (1.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.189 s 124.917 ms (11.7%)
Agent iast 1.169 s 104.631 ms (9.8%)
Agent profiling 1.267 s 202.474 ms (19.0%)
Total tracing 10.347 s -
Total appsec 10.538 s 190.567 ms (1.8%)
Total iast 10.792 s 444.921 ms (4.3%)
Total profiling 10.58 s 232.945 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.377 ms) : 0, 671377
BytebuddyAgent [candidate] (665.768 ms) : 0, 665768
GlobalTracer [baseline] (308.258 ms) : 0, 308258
GlobalTracer [candidate] (305.379 ms) : 0, 305379
AppSec [baseline] (50.172 ms) : 0, 50172
AppSec [candidate] (50.086 ms) : 0, 50086
Remote Config [baseline] (681.477 µs) : 0, 681
Remote Config [candidate] (667.761 µs) : 0, 668
Telemetry [baseline] (7.677 ms) : 0, 7677
Telemetry [candidate] (7.552 ms) : 0, 7552
section appsec
BytebuddyAgent [baseline] (676.92 ms) : 0, 676920
BytebuddyAgent [candidate] (678.504 ms) : 0, 678504
GlobalTracer [baseline] (299.575 ms) : 0, 299575
GlobalTracer [candidate] (300.61 ms) : 0, 300610
AppSec [baseline] (153.665 ms) : 0, 153665
AppSec [candidate] (154.091 ms) : 0, 154091
Remote Config [baseline] (623.646 µs) : 0, 624
Remote Config [candidate] (623.701 µs) : 0, 624
Telemetry [baseline] (8.908 ms) : 0, 8908
Telemetry [candidate] (8.894 ms) : 0, 8894
IAST [baseline] (20.328 ms) : 0, 20328
IAST [candidate] (22.982 ms) : 0, 22982
section iast
BytebuddyAgent [baseline] (780.852 ms) : 0, 780852
BytebuddyAgent [candidate] (779.662 ms) : 0, 779662
GlobalTracer [baseline] (295.556 ms) : 0, 295556
GlobalTracer [candidate] (295.743 ms) : 0, 295743
AppSec [baseline] (51.077 ms) : 0, 51077
AppSec [candidate] (47.251 ms) : 0, 47251
Remote Config [baseline] (577.672 µs) : 0, 578
Remote Config [candidate] (576.372 µs) : 0, 576
Telemetry [baseline] (7.76 ms) : 0, 7760
Telemetry [candidate] (7.049 ms) : 0, 7049
IAST [baseline] (24.031 ms) : 0, 24031
IAST [candidate] (24.911 ms) : 0, 24911
section profiling
BytebuddyAgent [baseline] (662.936 ms) : 0, 662936
BytebuddyAgent [candidate] (663.861 ms) : 0, 663861
GlobalTracer [baseline] (390.086 ms) : 0, 390086
GlobalTracer [candidate] (389.129 ms) : 0, 389129
AppSec [baseline] (51.676 ms) : 0, 51676
AppSec [candidate] (51.756 ms) : 0, 51756
Remote Config [baseline] (664.828 µs) : 0, 665
Remote Config [candidate] (656.969 µs) : 0, 657
Telemetry [baseline] (7.336 ms) : 0, 7336
Telemetry [candidate] (7.395 ms) : 0, 7395
ProfilingAgent [baseline] (96.088 ms) : 0, 96088
ProfilingAgent [candidate] (96.509 ms) : 0, 96509
Profiling [baseline] (96.113 ms) : 0, 96113
Profiling [candidate] (96.534 ms) : 0, 96534
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-07-12T14:57:42 2024-07-12T15:04:31
git_branch master smola/update-system-tests-20240709
git_commit_date 1720794919 1720795605
git_commit_sha 0e5e274 ab7632a
release_version 1.38.0-SNAPSHOT~0e5e2749b0 1.38.0-SNAPSHOT~ab7632ac0d
start_time 2024-07-12T14:57:29 2024-07-12T15:04:18
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1720797018 1720797018
ci_job_id 571000235 571000235
ci_pipeline_id 39044794 39044794
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.438 µs) : 349, 390
.   : milestone, 369,
iast (478.47 µs) : 458, 499
.   : milestone, 478,
iast_FULL (549.799 µs) : 529, 571
.   : milestone, 550,
iast_GLOBAL (510.095 µs) : 488, 532
.   : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (481.659 µs) : 461, 503
.   : milestone, 482,
iast_INACTIVE (455.475 µs) : 434, 477
.   : milestone, 455,
iast_TELEMETRY_OFF (473.531 µs) : 452, 495
.   : milestone, 474,
tracing (450.732 µs) : 430, 471
.   : milestone, 451,
section candidate
no_agent (370.564 µs) : 351, 390
.   : milestone, 371,
iast (481.613 µs) : 460, 503
.   : milestone, 482,
iast_FULL (559.854 µs) : 539, 581
.   : milestone, 560,
iast_GLOBAL (510.229 µs) : 488, 532
.   : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (487.854 µs) : 467, 509
.   : milestone, 488,
iast_INACTIVE (461.576 µs) : 440, 483
.   : milestone, 462,
iast_TELEMETRY_OFF (473.245 µs) : 452, 494
.   : milestone, 473,
tracing (447.484 µs) : 426, 469
.   : milestone, 447,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.438 µs [348.743 µs, 390.133 µs] -
iast 478.47 µs [457.681 µs, 499.259 µs] 109.032 µs (29.5%)
iast_FULL 549.799 µs [528.799 µs, 570.799 µs] 180.361 µs (48.8%)
iast_GLOBAL 510.095 µs [488.157 µs, 532.032 µs] 140.657 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED 481.659 µs [460.545 µs, 502.773 µs] 112.221 µs (30.4%)
iast_INACTIVE 455.475 µs [434.256 µs, 476.694 µs] 86.037 µs (23.3%)
iast_TELEMETRY_OFF 473.531 µs [452.247 µs, 494.814 µs] 104.093 µs (28.2%)
tracing 450.732 µs [429.999 µs, 471.465 µs] 81.294 µs (22.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.564 µs [351.22 µs, 389.909 µs] -
iast 481.613 µs [460.427 µs, 502.799 µs] 111.048 µs (30.0%)
iast_FULL 559.854 µs [538.792 µs, 580.916 µs] 189.29 µs (51.1%)
iast_GLOBAL 510.229 µs [488.164 µs, 532.295 µs] 139.665 µs (37.7%)
iast_HARDCODED_SECRET_DISABLED 487.854 µs [466.889 µs, 508.818 µs] 117.289 µs (31.7%)
iast_INACTIVE 461.576 µs [439.859 µs, 483.292 µs] 91.011 µs (24.6%)
iast_TELEMETRY_OFF 473.245 µs [452.171 µs, 494.32 µs] 102.681 µs (27.7%)
tracing 447.484 µs [426.31 µs, 468.658 µs] 76.919 µs (20.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1361
.   : milestone, 1341,
appsec (1.715 ms) : 1691, 1738
.   : milestone, 1715,
appsec_no_iast (1.704 ms) : 1679, 1729
.   : milestone, 1704,
iast (1.482 ms) : 1459, 1504
.   : milestone, 1482,
profiling (1.503 ms) : 1477, 1530
.   : milestone, 1503,
tracing (1.469 ms) : 1444, 1493
.   : milestone, 1469,
section candidate
no_agent (1.339 ms) : 1320, 1358
.   : milestone, 1339,
appsec (1.708 ms) : 1684, 1732
.   : milestone, 1708,
appsec_no_iast (1.729 ms) : 1705, 1753
.   : milestone, 1729,
iast (1.478 ms) : 1455, 1501
.   : milestone, 1478,
profiling (1.543 ms) : 1517, 1569
.   : milestone, 1543,
tracing (1.474 ms) : 1451, 1497
.   : milestone, 1474,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.322 ms, 1.361 ms] -
appsec 1.715 ms [1.691 ms, 1.738 ms] 373.401 µs (27.8%)
appsec_no_iast 1.704 ms [1.679 ms, 1.729 ms] 362.987 µs (27.1%)
iast 1.482 ms [1.459 ms, 1.504 ms] 140.461 µs (10.5%)
profiling 1.503 ms [1.477 ms, 1.53 ms] 162.254 µs (12.1%)
tracing 1.469 ms [1.444 ms, 1.493 ms] 127.375 µs (9.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.339 ms [1.32 ms, 1.358 ms] -
appsec 1.708 ms [1.684 ms, 1.732 ms] 369.426 µs (27.6%)
appsec_no_iast 1.729 ms [1.705 ms, 1.753 ms] 390.26 µs (29.2%)
iast 1.478 ms [1.455 ms, 1.501 ms] 138.927 µs (10.4%)
profiling 1.543 ms [1.517 ms, 1.569 ms] 203.988 µs (15.2%)
tracing 1.474 ms [1.451 ms, 1.497 ms] 135.364 µs (10.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/update-system-tests-20240709
git_commit_date 1720794919 1720795605
git_commit_sha 0e5e274 ab7632a
release_version 1.38.0-SNAPSHOT~0e5e2749b0 1.38.0-SNAPSHOT~ab7632ac0d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1720797527 1720797527
ci_job_id 571000236 571000236
ci_pipeline_id 39044794 39044794
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.45 ms) : 1439, 1462
.   : milestone, 1450,
appsec (2.213 ms) : 2178, 2248
.   : milestone, 2213,
iast (1.952 ms) : 1911, 1994
.   : milestone, 1952,
iast_GLOBAL (2.002 ms) : 1959, 2045
.   : milestone, 2002,
profiling (2.333 ms) : 2145, 2521
.   : milestone, 2333,
tracing (1.829 ms) : 1796, 1862
.   : milestone, 1829,
section candidate
no_agent (1.455 ms) : 1444, 1466
.   : milestone, 1455,
appsec (2.199 ms) : 2165, 2234
.   : milestone, 2199,
iast (1.966 ms) : 1925, 2008
.   : milestone, 1966,
iast_GLOBAL (2.005 ms) : 1962, 2047
.   : milestone, 2005,
profiling (1.854 ms) : 1818, 1889
.   : milestone, 1854,
tracing (1.829 ms) : 1796, 1862
.   : milestone, 1829,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.45 ms [1.439 ms, 1.462 ms] -
appsec 2.213 ms [2.178 ms, 2.248 ms] 762.896 µs (52.6%)
iast 1.952 ms [1.911 ms, 1.994 ms] 502.207 µs (34.6%)
iast_GLOBAL 2.002 ms [1.959 ms, 2.045 ms] 551.489 µs (38.0%)
profiling 2.333 ms [2.145 ms, 2.521 ms] 882.824 µs (60.9%)
tracing 1.829 ms [1.796 ms, 1.862 ms] 378.882 µs (26.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.455 ms [1.444 ms, 1.466 ms] -
appsec 2.199 ms [2.165 ms, 2.234 ms] 744.28 µs (51.2%)
iast 1.966 ms [1.925 ms, 2.008 ms] 511.275 µs (35.1%)
iast_GLOBAL 2.005 ms [1.962 ms, 2.047 ms] 549.609 µs (37.8%)
profiling 1.854 ms [1.818 ms, 1.889 ms] 398.793 µs (27.4%)
tracing 1.829 ms [1.796 ms, 1.862 ms] 374.393 µs (25.7%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.01 s) : 15010000, 15010000
.   : milestone, 15010000,
appsec (15.005 s) : 15005000, 15005000
.   : milestone, 15005000,
iast (18.67 s) : 18670000, 18670000
.   : milestone, 18670000,
iast_GLOBAL (18.01 s) : 18010000, 18010000
.   : milestone, 18010000,
profiling (15.352 s) : 15352000, 15352000
.   : milestone, 15352000,
tracing (15.095 s) : 15095000, 15095000
.   : milestone, 15095000,
section candidate
no_agent (15.067 s) : 15067000, 15067000
.   : milestone, 15067000,
appsec (15.173 s) : 15173000, 15173000
.   : milestone, 15173000,
iast (18.662 s) : 18662000, 18662000
.   : milestone, 18662000,
iast_GLOBAL (17.835 s) : 17835000, 17835000
.   : milestone, 17835000,
profiling (15.944 s) : 15944000, 15944000
.   : milestone, 15944000,
tracing (14.871 s) : 14871000, 14871000
.   : milestone, 14871000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.01 s [15.01 s, 15.01 s] -
appsec 15.005 s [15.005 s, 15.005 s] -5.0 ms (-0.0%)
iast 18.67 s [18.67 s, 18.67 s] 3.66 s (24.4%)
iast_GLOBAL 18.01 s [18.01 s, 18.01 s] 3.0 s (20.0%)
profiling 15.352 s [15.352 s, 15.352 s] 342.0 ms (2.3%)
tracing 15.095 s [15.095 s, 15.095 s] 85.0 ms (0.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.067 s [15.067 s, 15.067 s] -
appsec 15.173 s [15.173 s, 15.173 s] 106.0 ms (0.7%)
iast 18.662 s [18.662 s, 18.662 s] 3.595 s (23.9%)
iast_GLOBAL 17.835 s [17.835 s, 17.835 s] 2.768 s (18.4%)
profiling 15.944 s [15.944 s, 15.944 s] 877.0 ms (5.8%)
tracing 14.871 s [14.871 s, 14.871 s] -196.0 ms (-1.3%)

@smola smola force-pushed the smola/update-system-tests-20240709 branch 2 times, most recently from 450de52 to ed43051 Compare July 11, 2024 12:14
@smola smola changed the title Update system-tests to 2640b90ad5641717b6012a840646e85bad3a24fc and use Python 3.12 Update system-tests to 5330ad86a59bd0f055d88287f5e64af994e22908 and use Python 3.12 Jul 11, 2024
@smola smola force-pushed the smola/update-system-tests-20240709 branch from 6bb9f74 to ab7632a Compare July 12, 2024 14:46
@smola smola changed the title Update system-tests to 5330ad86a59bd0f055d88287f5e64af994e22908 and use Python 3.12 Update system-tests to 06cbfad5f9facbc6e258d9c8208c466cd46efeea and use Python 3.12 Jul 12, 2024
@smola smola marked this pull request as ready for review July 15, 2024 15:18
@smola smola requested a review from a team as a code owner July 15, 2024 15:18
@smola smola requested review from ygree and nayeem-kamal July 15, 2024 15:18
@smola smola merged commit 9184826 into master Jul 15, 2024
82 checks passed
@smola smola deleted the smola/update-system-tests-20240709 branch July 15, 2024 15:18
@github-actions github-actions bot added this to the 1.38.0 milestone Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@mcculls mcculls mcculls approved these changes

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.38.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@smola @mcculls @PerfectSlayer