Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rename sampling tests #6121

Merged
merged 1 commit into from
Nov 2, 2023
Merged

Rename sampling tests #6121

merged 1 commit into from
Nov 2, 2023

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Oct 31, 2023 •
edited
Loading

add line probe condition and ensure call sampling

What Does This Do

Motivation

Additional Notes

@jpbempel
Rename sampling tests
a58a244 
add line probe condition and ensure call sampling
@jpbempel jpbempel added comp: testing Testing tag: no release notes Changes to exclude from release notes comp: debugger Dynamic Instrumentation labels Oct 31, 2023
@jpbempel jpbempel requested a review from a team as a code owner October 31, 2023 09:52
@jpbempel jpbempel requested review from shatzi and removed request for a team October 31, 2023 09:52
@pr-commenter
Copy link

pr-commenter bot commented Oct 31, 2023

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~34c69558d4 1.23.0-SNAPSHOT~a58a244b32
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~a58a244b32, baseline=1.23.0-SNAPSHOT~34c69558d4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.034 s) : 0, 1033977
Total [baseline] (8.754 s) : 0, 8754318
Agent [candidate] (1.036 s) : 0, 1035558
Total [candidate] (8.785 s) : 0, 8784985
section iast
Agent [baseline] (1.163 s) : 0, 1162886
Total [baseline] (9.308 s) : 0, 9307688
Agent [candidate] (1.172 s) : 0, 1171640
Total [candidate] (9.307 s) : 0, 9307277
section iast_TELEMETRY_OFF
Agent [baseline] (1.162 s) : 0, 1162301
Total [baseline] (9.32 s) : 0, 9320011
Agent [candidate] (1.146 s) : 0, 1145509
Total [candidate] (9.25 s) : 0, 9250018
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.034 s -
Agent iast 1.163 s 128.908 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.162 s 128.324 ms (12.4%)
Total tracing 8.754 s -
Total iast 9.308 s 553.37 ms (6.3%)
Total iast_TELEMETRY_OFF 9.32 s 565.694 ms (6.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent iast 1.172 s 136.083 ms (13.1%)
Agent iast_TELEMETRY_OFF 1.146 s 109.952 ms (10.6%)
Total tracing 8.785 s -
Total iast 9.307 s 522.292 ms (5.9%)
Total iast_TELEMETRY_OFF 9.25 s 465.033 ms (5.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~a58a244b32, baseline=1.23.0-SNAPSHOT~34c69558d4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (647.125 ms) : 0, 647125
BytebuddyAgent [candidate] (647.391 ms) : 0, 647391
GlobalTracer [baseline] (296.016 ms) : 0, 296016
GlobalTracer [candidate] (297.272 ms) : 0, 297272
AppSec [baseline] (49.324 ms) : 0, 49324
AppSec [candidate] (49.222 ms) : 0, 49222
Remote Config [baseline] (690.811 µs) : 0, 691
Remote Config [candidate] (688.44 µs) : 0, 688
Telemetry [baseline] (6.146 ms) : 0, 6146
Telemetry [candidate] (6.213 ms) : 0, 6213
section iast
BytebuddyAgent [baseline] (778.561 ms) : 0, 778561
BytebuddyAgent [candidate] (784.76 ms) : 0, 784760
GlobalTracer [baseline] (275.583 ms) : 0, 275583
GlobalTracer [candidate] (278.125 ms) : 0, 278125
AppSec [baseline] (48.622 ms) : 0, 48622
AppSec [candidate] (48.347 ms) : 0, 48347
IAST [baseline] (16.494 ms) : 0, 16494
IAST [candidate] (17.202 ms) : 0, 17202
Remote Config [baseline] (583.785 µs) : 0, 584
Remote Config [candidate] (581.898 µs) : 0, 582
Telemetry [baseline] (8.359 ms) : 0, 8359
Telemetry [candidate] (7.644 ms) : 0, 7644
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (776.962 ms) : 0, 776962
BytebuddyAgent [candidate] (764.77 ms) : 0, 764770
GlobalTracer [baseline] (277.649 ms) : 0, 277649
GlobalTracer [candidate] (274.08 ms) : 0, 274080
AppSec [baseline] (47.172 ms) : 0, 47172
AppSec [candidate] (46.564 ms) : 0, 46564
IAST [baseline] (18.041 ms) : 0, 18041
IAST [candidate] (17.331 ms) : 0, 17331
Remote Config [baseline] (651.883 µs) : 0, 652
Remote Config [candidate] (645.952 µs) : 0, 646
Telemetry [baseline] (6.958 ms) : 0, 6958
Telemetry [candidate] (7.617 ms) : 0, 7617
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~a58a244b32, baseline=1.23.0-SNAPSHOT~34c69558d4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.035 s) : 0, 1034531
Total [baseline] (9.382 s) : 0, 9382156
Agent [candidate] (1.036 s) : 0, 1036365
Total [candidate] (9.39 s) : 0, 9390138
section appsec
Agent [baseline] (1.123 s) : 0, 1122636
Total [baseline] (9.419 s) : 0, 9419197
Agent [candidate] (1.123 s) : 0, 1123345
Total [candidate] (9.38 s) : 0, 9380215
section iast
Agent [baseline] (1.159 s) : 0, 1158862
Total [baseline] (9.501 s) : 0, 9500825
Agent [candidate] (1.155 s) : 0, 1154551
Total [candidate] (9.509 s) : 0, 9508941
section profiling
Agent [baseline] (1.204 s) : 0, 1203816
Total [baseline] (9.547 s) : 0, 9546929
Agent [candidate] (1.215 s) : 0, 1215212
Total [candidate] (9.565 s) : 0, 9564786
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.035 s -
Agent appsec 1.123 s 88.105 ms (8.5%)
Agent iast 1.159 s 124.331 ms (12.0%)
Agent profiling 1.204 s 169.285 ms (16.4%)
Total tracing 9.382 s -
Total appsec 9.419 s 37.041 ms (0.4%)
Total iast 9.501 s 118.669 ms (1.3%)
Total profiling 9.547 s 164.773 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent appsec 1.123 s 86.98 ms (8.4%)
Agent iast 1.155 s 118.186 ms (11.4%)
Agent profiling 1.215 s 178.846 ms (17.3%)
Total tracing 9.39 s -
Total appsec 9.38 s -9.923 ms (-0.1%)
Total iast 9.509 s 118.803 ms (1.3%)
Total profiling 9.565 s 174.648 ms (1.9%) 
gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~a58a244b32, baseline=1.23.0-SNAPSHOT~34c69558d4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.646 ms) : 0, 646646
BytebuddyAgent [candidate] (648.391 ms) : 0, 648391
GlobalTracer [baseline] (296.883 ms) : 0, 296883
GlobalTracer [candidate] (296.855 ms) : 0, 296855
AppSec [baseline] (49.441 ms) : 0, 49441
AppSec [candidate] (49.476 ms) : 0, 49476
Remote Config [baseline] (692.34 µs) : 0, 692
Remote Config [candidate] (689.643 µs) : 0, 690
Telemetry [baseline] (6.149 ms) : 0, 6149
Telemetry [candidate] (6.187 ms) : 0, 6187
section appsec
BytebuddyAgent [baseline] (646.718 ms) : 0, 646718
BytebuddyAgent [candidate] (647.086 ms) : 0, 647086
GlobalTracer [baseline] (295.447 ms) : 0, 295447
GlobalTracer [candidate] (296.057 ms) : 0, 296057
AppSec [baseline] (139.386 ms) : 0, 139386
AppSec [candidate] (139.233 ms) : 0, 139233
Remote Config [baseline] (647.448 µs) : 0, 647
Remote Config [candidate] (645.994 µs) : 0, 646
Telemetry [baseline] (5.761 ms) : 0, 5761
Telemetry [candidate] (5.755 ms) : 0, 5755
section iast
BytebuddyAgent [baseline] (776.848 ms) : 0, 776848
BytebuddyAgent [candidate] (774.158 ms) : 0, 774158
GlobalTracer [baseline] (275.143 ms) : 0, 275143
GlobalTracer [candidate] (274.989 ms) : 0, 274989
AppSec [baseline] (46.759 ms) : 0, 46759
AppSec [candidate] (46.312 ms) : 0, 46312
Remote Config [baseline] (601.316 µs) : 0, 601
Remote Config [candidate] (593.978 µs) : 0, 594
Telemetry [baseline] (8.997 ms) : 0, 8997
Telemetry [candidate] (7.595 ms) : 0, 7595
IAST [baseline] (15.872 ms) : 0, 15872
IAST [candidate] (16.474 ms) : 0, 16474
section profiling
ProfilingAgent [baseline] (80.983 ms) : 0, 80983
ProfilingAgent [candidate] (81.608 ms) : 0, 81608
BytebuddyAgent [baseline] (653.535 ms) : 0, 653535
BytebuddyAgent [candidate] (660.442 ms) : 0, 660442
GlobalTracer [baseline] (359.707 ms) : 0, 359707
GlobalTracer [candidate] (362.354 ms) : 0, 362354
AppSec [baseline] (49.541 ms) : 0, 49541
AppSec [candidate] (50.086 ms) : 0, 50086
Remote Config [baseline] (650.46 µs) : 0, 650
Remote Config [candidate] (652.394 µs) : 0, 652
Telemetry [baseline] (6.023 ms) : 0, 6023
Telemetry [candidate] (6.118 ms) : 0, 6118
Profiling [baseline] (81.008 ms) : 0, 81008
Profiling [candidate] (81.633 ms) : 0, 81633
Loading

Load

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~34c69558d4 1.23.0-SNAPSHOT~a58a244b32
config baseline candidate
end_time 2023-10-31T10:09:48 2023-10-31T10:26:12
start_time 2023-10-31T10:09:35 2023-10-31T10:25:59
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~a58a244b32, baseline=1.23.0-SNAPSHOT~34c69558d4
    dateFormat X
    axisFormat %s
section baseline
no_agent (368.344 µs) : 348, 389
.   : milestone, 368,
iast (465.673 µs) : 444, 487
.   : milestone, 466,
iast_FULL (524.98 µs) : 504, 546
.   : milestone, 525,
iast_INACTIVE (444.202 µs) : 422, 466
.   : milestone, 444,
iast_TELEMETRY_OFF (455.752 µs) : 434, 477
.   : milestone, 456,
tracing (441.518 µs) : 420, 463
.   : milestone, 442,
section candidate
no_agent (357.184 µs) : 337, 377
.   : milestone, 357,
iast (467.52 µs) : 447, 488
.   : milestone, 468,
iast_FULL (523.755 µs) : 503, 544
.   : milestone, 524,
iast_INACTIVE (437.573 µs) : 417, 458
.   : milestone, 438,
iast_TELEMETRY_OFF (458.384 µs) : 437, 479
.   : milestone, 458,
tracing (436.958 µs) : 416, 458
.   : milestone, 437,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 368.344 µs [347.613 µs, 389.076 µs] -
iast 465.673 µs [444.459 µs, 486.886 µs] 97.328 µs (26.4%)
iast_FULL 524.98 µs [504.262 µs, 545.698 µs] 156.636 µs (42.5%)
iast_INACTIVE 444.202 µs [422.267 µs, 466.138 µs] 75.858 µs (20.6%)
iast_TELEMETRY_OFF 455.752 µs [434.419 µs, 477.084 µs] 87.407 µs (23.7%)
tracing 441.518 µs [419.807 µs, 463.23 µs] 73.174 µs (19.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 357.184 µs [337.477 µs, 376.891 µs] -
iast 467.52 µs [446.919 µs, 488.121 µs] 110.336 µs (30.9%)
iast_FULL 523.755 µs [503.178 µs, 544.332 µs] 166.571 µs (46.6%)
iast_INACTIVE 437.573 µs [416.865 µs, 458.281 µs] 80.389 µs (22.5%)
iast_TELEMETRY_OFF 458.384 µs [437.405 µs, 479.363 µs] 101.2 µs (28.3%)
tracing 436.958 µs [415.554 µs, 458.361 µs] 79.774 µs (22.3%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~a58a244b32, baseline=1.23.0-SNAPSHOT~34c69558d4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.346 ms) : 1327, 1365
.   : milestone, 1346,
appsec (1.728 ms) : 1704, 1752
.   : milestone, 1728,
iast (1.491 ms) : 1467, 1515
.   : milestone, 1491,
profiling (1.492 ms) : 1467, 1517
.   : milestone, 1492,
tracing (1.457 ms) : 1433, 1481
.   : milestone, 1457,
section candidate
no_agent (1.348 ms) : 1329, 1368
.   : milestone, 1348,
appsec (1.717 ms) : 1693, 1742
.   : milestone, 1717,
iast (1.477 ms) : 1452, 1501
.   : milestone, 1477,
profiling (1.479 ms) : 1454, 1504
.   : milestone, 1479,
tracing (1.467 ms) : 1443, 1492
.   : milestone, 1467,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.327 ms, 1.365 ms] -
appsec 1.728 ms [1.704 ms, 1.752 ms] 381.872 µs (28.4%)
iast 1.491 ms [1.467 ms, 1.515 ms] 144.958 µs (10.8%)
profiling 1.492 ms [1.467 ms, 1.517 ms] 145.625 µs (10.8%)
tracing 1.457 ms [1.433 ms, 1.481 ms] 111.052 µs (8.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.329 ms, 1.368 ms] -
appsec 1.717 ms [1.693 ms, 1.742 ms] 369.109 µs (27.4%)
iast 1.477 ms [1.452 ms, 1.501 ms] 128.664 µs (9.5%)
profiling 1.479 ms [1.454 ms, 1.504 ms] 130.996 µs (9.7%)
tracing 1.467 ms [1.443 ms, 1.492 ms] 119.128 µs (8.8%)

@jpbempel jpbempel merged commit 23bbe14 into master Nov 2, 2023
69 of 71 checks passed
@jpbempel jpbempel deleted the jpbempel/rename-sampling-tests branch November 2, 2023 21:46
@github-actions github-actions bot added this to the 1.23.0 milestone Nov 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shatzi shatzi shatzi approved these changes

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.23.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @shatzi