Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[POC] Flaky test retries #8113

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

[POC] Flaky test retries #8113

wants to merge 4 commits into from

Conversation

smola
Copy link
Member

@smola smola commented Dec 18, 2024

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Dec 18, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/retry-flaky-tests
git_commit_date 1734694675 1734710544
git_commit_sha a3e9bda bb572fc
release_version 1.45.0-SNAPSHOT~a3e9bda406 1.45.0-SNAPSHOT~bb572fc4ad
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1734712921 1734712921
ci_job_id 747200698 747200698
ci_pipeline_id 51663864 51663864
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 57 metrics, 5 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:tracing:AppSec worse
[+1.741ms; +6.635ms] or [+3.084%; +11.756%]		 60.625ms 56.437ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1052230
Total [baseline] (10.353 s) : 0, 10352988
Agent [candidate] (1.054 s) : 0, 1053632
Total [candidate] (10.389 s) : 0, 10388586
section appsec
Agent [baseline] (1.19 s) : 0, 1189765
Total [baseline] (10.727 s) : 0, 10727243
Agent [candidate] (1.185 s) : 0, 1185492
Total [candidate] (10.808 s) : 0, 10808011
section iast
Agent [baseline] (1.179 s) : 0, 1178899
Total [baseline] (10.97 s) : 0, 10970325
Agent [candidate] (1.177 s) : 0, 1176765
Total [candidate] (10.94 s) : 0, 10940316
section profiling
Agent [baseline] (1.283 s) : 0, 1283005
Total [baseline] (10.874 s) : 0, 10873943
Agent [candidate] (1.275 s) : 0, 1275460
Total [candidate] (10.784 s) : 0, 10784147
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.052 s -
Agent appsec 1.19 s 137.535 ms (13.1%)
Agent iast 1.179 s 126.67 ms (12.0%)
Agent profiling 1.283 s 230.775 ms (21.9%)
Total tracing 10.353 s -
Total appsec 10.727 s 374.255 ms (3.6%)
Total iast 10.97 s 617.337 ms (6.0%)
Total profiling 10.874 s 520.955 ms (5.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent appsec 1.185 s 131.859 ms (12.5%)
Agent iast 1.177 s 123.133 ms (11.7%)
Agent profiling 1.275 s 221.827 ms (21.1%)
Total tracing 10.389 s -
Total appsec 10.808 s 419.425 ms (4.0%)
Total iast 10.94 s 551.73 ms (5.3%)
Total profiling 10.784 s 395.561 ms (3.8%) 
gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (710.897 ms) : 0, 710897
BytebuddyAgent [candidate] (711.603 ms) : 0, 711603
GlobalTracer [baseline] (255.454 ms) : 0, 255454
GlobalTracer [candidate] (256.373 ms) : 0, 256373
AppSec [baseline] (60.591 ms) : 0, 60591
AppSec [candidate] (57.954 ms) : 0, 57954
Remote Config [baseline] (678.475 µs) : 0, 678
Remote Config [candidate] (692.202 µs) : 0, 692
Telemetry [baseline] (9.639 ms) : 0, 9639
Telemetry [candidate] (12.002 ms) : 0, 12002
section appsec
BytebuddyAgent [baseline] (731.092 ms) : 0, 731092
BytebuddyAgent [candidate] (728.567 ms) : 0, 728567
GlobalTracer [baseline] (253.547 ms) : 0, 253547
GlobalTracer [candidate] (252.719 ms) : 0, 252719
AppSec [baseline] (171.253 ms) : 0, 171253
AppSec [candidate] (170.873 ms) : 0, 170873
IAST [baseline] (19.62 ms) : 0, 19620
IAST [candidate] (19.397 ms) : 0, 19397
Remote Config [baseline] (651.221 µs) : 0, 651
Remote Config [candidate] (655.595 µs) : 0, 656
Telemetry [baseline] (8.418 ms) : 0, 8418
Telemetry [candidate] (7.999 ms) : 0, 7999
section iast
BytebuddyAgent [baseline] (829.096 ms) : 0, 829096
BytebuddyAgent [candidate] (827.789 ms) : 0, 827789
GlobalTracer [baseline] (246.721 ms) : 0, 246721
GlobalTracer [candidate] (246.202 ms) : 0, 246202
AppSec [baseline] (57.761 ms) : 0, 57761
AppSec [candidate] (57.531 ms) : 0, 57531
IAST [baseline] (21.06 ms) : 0, 21060
IAST [candidate] (21.214 ms) : 0, 21214
Remote Config [baseline] (663.888 µs) : 0, 664
Remote Config [candidate] (654.222 µs) : 0, 654
Telemetry [baseline] (8.499 ms) : 0, 8499
Telemetry [candidate] (8.422 ms) : 0, 8422
section profiling
BytebuddyAgent [baseline] (707.385 ms) : 0, 707385
BytebuddyAgent [candidate] (702.885 ms) : 0, 702885
GlobalTracer [baseline] (373.573 ms) : 0, 373573
GlobalTracer [candidate] (372.442 ms) : 0, 372442
AppSec [baseline] (54.397 ms) : 0, 54397
AppSec [candidate] (54.021 ms) : 0, 54021
Remote Config [baseline] (673.291 µs) : 0, 673
Remote Config [candidate] (662.504 µs) : 0, 663
Telemetry [baseline] (7.992 ms) : 0, 7992
Telemetry [candidate] (7.9 ms) : 0, 7900
ProfilingAgent [baseline] (96.876 ms) : 0, 96876
ProfilingAgent [candidate] (95.825 ms) : 0, 95825
Profiling [baseline] (96.9 ms) : 0, 96900
Profiling [candidate] (95.848 ms) : 0, 95848
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1054775
Total [baseline] (8.602 s) : 0, 8601949
Agent [candidate] (1.054 s) : 0, 1053563
Total [candidate] (8.594 s) : 0, 8594112
section iast
Agent [baseline] (1.177 s) : 0, 1176510
Total [baseline] (9.169 s) : 0, 9168929
Agent [candidate] (1.178 s) : 0, 1178005
Total [candidate] (9.179 s) : 0, 9178704
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.18 s) : 0, 1179997
Total [baseline] (9.161 s) : 0, 9160627
Agent [candidate] (1.18 s) : 0, 1179738
Total [candidate] (9.129 s) : 0, 9129062
section iast_TELEMETRY_OFF
Agent [baseline] (1.175 s) : 0, 1174854
Total [baseline] (9.197 s) : 0, 9196837
Agent [candidate] (1.173 s) : 0, 1173002
Total [candidate] (9.153 s) : 0, 9153439
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent iast 1.177 s 121.735 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.18 s 125.222 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.175 s 120.078 ms (11.4%)
Total tracing 8.602 s -
Total iast 9.169 s 566.98 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.161 s 558.678 ms (6.5%)
Total iast_TELEMETRY_OFF 9.197 s 594.888 ms (6.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.178 s 124.442 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.18 s 126.175 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.173 s 119.439 ms (11.3%)
Total tracing 8.594 s -
Total iast 9.179 s 584.592 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.129 s 534.95 ms (6.2%)
Total iast_TELEMETRY_OFF 9.153 s 559.327 ms (6.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (712.303 ms) : 0, 712303
BytebuddyAgent [candidate] (711.231 ms) : 0, 711231
GlobalTracer [baseline] (256.339 ms) : 0, 256339
GlobalTracer [candidate] (255.687 ms) : 0, 255687
AppSec [baseline] (56.437 ms) : 0, 56437
AppSec [candidate] (60.625 ms) : 0, 60625
Remote Config [baseline] (696.361 µs) : 0, 696
Remote Config [candidate] (691.731 µs) : 0, 692
Telemetry [baseline] (14.071 ms) : 0, 14071
Telemetry [candidate] (10.348 ms) : 0, 10348
section iast
BytebuddyAgent [baseline] (827.916 ms) : 0, 827916
BytebuddyAgent [candidate] (828.538 ms) : 0, 828538
GlobalTracer [baseline] (245.958 ms) : 0, 245958
GlobalTracer [candidate] (246.404 ms) : 0, 246404
AppSec [baseline] (57.336 ms) : 0, 57336
AppSec [candidate] (57.771 ms) : 0, 57771
IAST [baseline] (21.238 ms) : 0, 21238
IAST [candidate] (21.096 ms) : 0, 21096
Remote Config [baseline] (662.257 µs) : 0, 662
Remote Config [candidate] (683.891 µs) : 0, 684
Telemetry [baseline] (8.474 ms) : 0, 8474
Telemetry [candidate] (8.509 ms) : 0, 8509
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (828.782 ms) : 0, 828782
BytebuddyAgent [candidate] (829.503 ms) : 0, 829503
GlobalTracer [baseline] (247.241 ms) : 0, 247241
GlobalTracer [candidate] (246.779 ms) : 0, 246779
AppSec [baseline] (58.318 ms) : 0, 58318
AppSec [candidate] (58.099 ms) : 0, 58099
IAST [baseline] (21.453 ms) : 0, 21453
IAST [candidate] (21.154 ms) : 0, 21154
Remote Config [baseline] (663.809 µs) : 0, 664
Remote Config [candidate] (653.797 µs) : 0, 654
Telemetry [baseline] (8.551 ms) : 0, 8551
Telemetry [candidate] (8.479 ms) : 0, 8479
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (825.466 ms) : 0, 825466
BytebuddyAgent [candidate] (824.086 ms) : 0, 824086
GlobalTracer [baseline] (246.446 ms) : 0, 246446
GlobalTracer [candidate] (246.579 ms) : 0, 246579
AppSec [baseline] (58.071 ms) : 0, 58071
AppSec [candidate] (57.648 ms) : 0, 57648
IAST [baseline] (20.914 ms) : 0, 20914
IAST [candidate] (20.86 ms) : 0, 20860
Remote Config [baseline] (658.461 µs) : 0, 658
Remote Config [candidate] (640.859 µs) : 0, 641
Telemetry [baseline] (8.396 ms) : 0, 8396
Telemetry [candidate] (8.279 ms) : 0, 8279
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-12-20T16:14:21 2024-12-20T16:21:20
git_branch master smola/retry-flaky-tests
git_commit_date 1734694675 1734710544
git_commit_sha a3e9bda bb572fc
release_version 1.45.0-SNAPSHOT~a3e9bda406 1.45.0-SNAPSHOT~bb572fc4ad
start_time 2024-12-20T16:14:07 2024-12-20T16:21:07
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1734712033 1734712033
ci_job_id 747200699 747200699
ci_pipeline_id 51663864 51663864
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (382.587 µs) : 362, 403
.   : milestone, 383,
iast (494.596 µs) : 473, 517
.   : milestone, 495,
iast_FULL (656.275 µs) : 635, 678
.   : milestone, 656,
iast_GLOBAL (523.508 µs) : 501, 546
.   : milestone, 524,
iast_HARDCODED_SECRET_DISABLED (490.394 µs) : 469, 512
.   : milestone, 490,
iast_INACTIVE (455.872 µs) : 434, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (485.544 µs) : 464, 507
.   : milestone, 486,
tracing (451.228 µs) : 431, 472
.   : milestone, 451,
section candidate
no_agent (372.463 µs) : 353, 392
.   : milestone, 372,
iast (491.299 µs) : 470, 513
.   : milestone, 491,
iast_FULL (652.645 µs) : 631, 674
.   : milestone, 653,
iast_GLOBAL (522.378 µs) : 501, 544
.   : milestone, 522,
iast_HARDCODED_SECRET_DISABLED (499.123 µs) : 477, 521
.   : milestone, 499,
iast_INACTIVE (461.391 µs) : 439, 483
.   : milestone, 461,
iast_TELEMETRY_OFF (482.787 µs) : 461, 504
.   : milestone, 483,
tracing (457.962 µs) : 437, 479
.   : milestone, 458,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 382.587 µs [361.87 µs, 403.303 µs] -
iast 494.596 µs [472.619 µs, 516.573 µs] 112.009 µs (29.3%)
iast_FULL 656.275 µs [634.525 µs, 678.026 µs] 273.689 µs (71.5%)
iast_GLOBAL 523.508 µs [501.241 µs, 545.774 µs] 140.921 µs (36.8%)
iast_HARDCODED_SECRET_DISABLED 490.394 µs [469.098 µs, 511.69 µs] 107.808 µs (28.2%)
iast_INACTIVE 455.872 µs [434.475 µs, 477.269 µs] 73.286 µs (19.2%)
iast_TELEMETRY_OFF 485.544 µs [463.689 µs, 507.4 µs] 102.958 µs (26.9%)
tracing 451.228 µs [430.54 µs, 471.915 µs] 68.641 µs (17.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.463 µs [352.642 µs, 392.285 µs] -
iast 491.299 µs [469.93 µs, 512.668 µs] 118.836 µs (31.9%)
iast_FULL 652.645 µs [631.067 µs, 674.223 µs] 280.182 µs (75.2%)
iast_GLOBAL 522.378 µs [500.797 µs, 543.959 µs] 149.915 µs (40.2%)
iast_HARDCODED_SECRET_DISABLED 499.123 µs [477.028 µs, 521.218 µs] 126.66 µs (34.0%)
iast_INACTIVE 461.391 µs [439.377 µs, 483.406 µs] 88.928 µs (23.9%)
iast_TELEMETRY_OFF 482.787 µs [461.232 µs, 504.341 µs] 110.323 µs (29.6%)
tracing 457.962 µs [436.833 µs, 479.09 µs] 85.499 µs (23.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.363 ms) : 1343, 1383
.   : milestone, 1363,
appsec (1.761 ms) : 1736, 1785
.   : milestone, 1761,
appsec_no_iast (1.756 ms) : 1731, 1781
.   : milestone, 1756,
iast (1.502 ms) : 1479, 1526
.   : milestone, 1502,
profiling (1.53 ms) : 1506, 1555
.   : milestone, 1530,
tracing (1.5 ms) : 1476, 1524
.   : milestone, 1500,
section candidate
no_agent (1.347 ms) : 1327, 1366
.   : milestone, 1347,
appsec (1.752 ms) : 1728, 1776
.   : milestone, 1752,
appsec_no_iast (1.776 ms) : 1752, 1801
.   : milestone, 1776,
iast (1.509 ms) : 1486, 1533
.   : milestone, 1509,
profiling (1.514 ms) : 1490, 1539
.   : milestone, 1514,
tracing (1.486 ms) : 1461, 1511
.   : milestone, 1486,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.343 ms, 1.383 ms] -
appsec 1.761 ms [1.736 ms, 1.785 ms] 397.651 µs (29.2%)
appsec_no_iast 1.756 ms [1.731 ms, 1.781 ms] 393.254 µs (28.9%)
iast 1.502 ms [1.479 ms, 1.526 ms] 139.539 µs (10.2%)
profiling 1.53 ms [1.506 ms, 1.555 ms] 167.326 µs (12.3%)
tracing 1.5 ms [1.476 ms, 1.524 ms] 137.175 µs (10.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.347 ms [1.327 ms, 1.366 ms] -
appsec 1.752 ms [1.728 ms, 1.776 ms] 405.71 µs (30.1%)
appsec_no_iast 1.776 ms [1.752 ms, 1.801 ms] 429.868 µs (31.9%)
iast 1.509 ms [1.486 ms, 1.533 ms] 162.967 µs (12.1%)
profiling 1.514 ms [1.49 ms, 1.539 ms] 167.771 µs (12.5%)
tracing 1.486 ms [1.461 ms, 1.511 ms] 139.717 µs (10.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/retry-flaky-tests
git_commit_date 1734694675 1734710544
git_commit_sha a3e9bda bb572fc
release_version 1.45.0-SNAPSHOT~a3e9bda406 1.45.0-SNAPSHOT~bb572fc4ad
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1734712631 1734712631
ci_job_id 747200700 747200700
ci_pipeline_id 51663864 51663864
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.94 s) : 14940000, 14940000
.   : milestone, 14940000,
appsec (15.175 s) : 15175000, 15175000
.   : milestone, 15175000,
iast (18.946 s) : 18946000, 18946000
.   : milestone, 18946000,
iast_GLOBAL (18.081 s) : 18081000, 18081000
.   : milestone, 18081000,
profiling (15.018 s) : 15018000, 15018000
.   : milestone, 15018000,
tracing (15.232 s) : 15232000, 15232000
.   : milestone, 15232000,
section candidate
no_agent (15.285 s) : 15285000, 15285000
.   : milestone, 15285000,
appsec (14.697 s) : 14697000, 14697000
.   : milestone, 14697000,
iast (18.99 s) : 18990000, 18990000
.   : milestone, 18990000,
iast_GLOBAL (17.875 s) : 17875000, 17875000
.   : milestone, 17875000,
profiling (15.034 s) : 15034000, 15034000
.   : milestone, 15034000,
tracing (14.972 s) : 14972000, 14972000
.   : milestone, 14972000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.94 s [14.94 s, 14.94 s] -
appsec 15.175 s [15.175 s, 15.175 s] 235.0 ms (1.6%)
iast 18.946 s [18.946 s, 18.946 s] 4.006 s (26.8%)
iast_GLOBAL 18.081 s [18.081 s, 18.081 s] 3.141 s (21.0%)
profiling 15.018 s [15.018 s, 15.018 s] 78.0 ms (0.5%)
tracing 15.232 s [15.232 s, 15.232 s] 292.0 ms (2.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.285 s [15.285 s, 15.285 s] -
appsec 14.697 s [14.697 s, 14.697 s] -588.0 ms (-3.8%)
iast 18.99 s [18.99 s, 18.99 s] 3.705 s (24.2%)
iast_GLOBAL 17.875 s [17.875 s, 17.875 s] 2.59 s (16.9%)
profiling 15.034 s [15.034 s, 15.034 s] -251.0 ms (-1.6%)
tracing 14.972 s [14.972 s, 14.972 s] -313.0 ms (-2.0%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~bb572fc4ad, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.356 ms) : 2314, 2397
.   : milestone, 2356,
iast (2.102 ms) : 2049, 2154
.   : milestone, 2102,
iast_GLOBAL (2.145 ms) : 2092, 2198
.   : milestone, 2145,
profiling (1.964 ms) : 1922, 2006
.   : milestone, 1964,
tracing (1.931 ms) : 1890, 1971
.   : milestone, 1931,
section candidate
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (2.36 ms) : 2318, 2402
.   : milestone, 2360,
iast (2.087 ms) : 2034, 2139
.   : milestone, 2087,
iast_GLOBAL (2.143 ms) : 2090, 2196
.   : milestone, 2143,
profiling (1.985 ms) : 1942, 2029
.   : milestone, 1985,
tracing (1.946 ms) : 1905, 1987
.   : milestone, 1946,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.356 ms [2.314 ms, 2.397 ms] 880.291 µs (59.7%)
iast 2.102 ms [2.049 ms, 2.154 ms] 626.264 µs (42.5%)
iast_GLOBAL 2.145 ms [2.092 ms, 2.198 ms] 669.592 µs (45.4%)
profiling 1.964 ms [1.922 ms, 2.006 ms] 489.029 µs (33.1%)
tracing 1.931 ms [1.89 ms, 1.971 ms] 455.294 µs (30.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.47 ms, 1.493 ms] -
appsec 2.36 ms [2.318 ms, 2.402 ms] 878.871 µs (59.3%)
iast 2.087 ms [2.034 ms, 2.139 ms] 605.338 µs (40.9%)
iast_GLOBAL 2.143 ms [2.09 ms, 2.196 ms] 662.021 µs (44.7%)
profiling 1.985 ms [1.942 ms, 2.029 ms] 504.019 µs (34.0%)
tracing 1.946 ms [1.905 ms, 1.987 ms] 464.539 µs (31.4%)

@smola smola force-pushed the smola/retry-flaky-tests branch 2 times, most recently from e8da49a to f17f444 Compare December 19, 2024 15:22
@smola smola force-pushed the smola/retry-flaky-tests branch from b55b57c to 17f6dc7 Compare December 20, 2024 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@smola