Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move snapshot UUID generation at serialization #7280

Merged
merged 1 commit into from
Jul 5, 2024
Merged

Move snapshot UUID generation at serialization #7280

merged 1 commit into from
Jul 5, 2024

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Jul 4, 2024 •
edited by jira bot
Loading

What Does This Do

UUID generation is not cheap, and therefore should be performed only when the snapshot is about to be sent.
Moving UUID generation at seriazliation time makes it out of the critical path.

Motivation

Performance

Additional Notes

Jira ticket: DEBUG-2346

@jpbempel jpbempel requested a review from a team as a code owner July 4, 2024 14:49
@jpbempel jpbempel requested review from cimi and removed request for a team July 4, 2024 14:49
cimi
cimi approved these changes Jul 4, 2024
View reviewed changes
Copy link
Contributor

@cimi cimi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Good idea, nice improvement!

Curious to see the benchmark results after this.

@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Jul 4, 2024
@jpbempel jpbempel force-pushed the jpbempel/move-snapshot-id-generation branch from a2e2282 to 14cf31b Compare July 5, 2024 10:33
@jpbempel jpbempel requested a review from a team as a code owner July 5, 2024 10:33
@jpbempel
Move snapshot UUID generation at serialization
692236f 
UUID generation is not cheap, and therefore should be performed only
when the snapshot is about to be sent.
Moving UUID generation at serialization time makes it out of the
critical path.
@jpbempel jpbempel force-pushed the jpbempel/move-snapshot-id-generation branch from 14cf31b to 692236f Compare July 5, 2024 14:22
@pr-commenter
Copy link

pr-commenter bot commented Jul 5, 2024

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1720190073 1720190490
end_time 2024-07-05T14:35:47 2024-07-05T14:42:43
git_branch master jpbempel/move-snapshot-id-generation
git_commit_sha 23a8164 692236f
start_time 2024-07-05T14:34:34 2024-07-05T14:41:31
See matching parameters
Baseline Candidate
ci_job_id 563600590 563600590
ci_pipeline_id 38448695 38448695
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1720189354 1720189354

Summary

Found 5 performance improvements and 0 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop better
[-920.367µs; -902.930µs] or [-7.554%; -7.411%]		 better
[-972.905µs; -889.722µs] or [-7.857%; -7.186%]		 better
[-1028.879µs; -882.028µs] or [-8.226%; -7.052%]		 better
[-1.357ms; -0.757ms] or [-10.293%; -5.745%]		 better
[+5.818op/s; +7.234op/s] or [+7.301%; +9.078%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-29.606µs; +22.271µs] or [-11.340%; +8.530%]		 unstable
[-41.900µs; +37.546µs] or [-14.094%; +12.629%]		 unstable
[-54.162µs; +51.988µs] or [-17.388%; +16.690%]		 unstable
[-90.203µs; +122.176µs] or [-14.197%; +19.230%]		 same
scenario:basic same same same unstable
[-72.977µs; +1.180µs] or [-11.829%; +0.191%]		 unstable
[-141.725op/s; +141.725op/s] or [-5.527%; +5.527%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (297.295 µs) : 262, 333
.   : milestone, 297,
basic (302.617 µs) : 293, 312
.   : milestone, 303,
loop (12.382 ms) : 12341, 12423
.   : milestone, 12382,
section candidate
noprobe (295.118 µs) : 255, 335
.   : milestone, 295,
basic (304.307 µs) : 295, 313
.   : milestone, 304,
loop (11.451 ms) : 11414, 11487
.   : milestone, 11451,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 297.295 µs [261.59 µs, 333.0 µs]
basic 302.617 µs [293.075 µs, 312.159 µs]
loop 12.382 ms [12.341 ms, 12.423 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 295.118 µs [255.121 µs, 335.115 µs]
basic 304.307 µs [295.186 µs, 313.428 µs]
loop 11.451 ms [11.414 ms, 11.487 ms]

@pr-commenter
Copy link

pr-commenter bot commented Jul 5, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/move-snapshot-id-generation
git_commit_date 1720178709 1720189354
git_commit_sha 23a8164 692236f
release_version 1.38.0-SNAPSHOT~23a8164e90 1.38.0-SNAPSHOT~692236f00e
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1720191898 1720191898
ci_job_id 563600584 563600584
ci_pipeline_id 38448695 38448695
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 11 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.073 s) : 0, 1073123
Total [baseline] (10.361 s) : 0, 10361064
Agent [candidate] (1.077 s) : 0, 1076856
Total [candidate] (10.379 s) : 0, 10378638
section appsec
Agent [baseline] (1.189 s) : 0, 1188797
Total [baseline] (10.518 s) : 0, 10518468
Agent [candidate] (1.189 s) : 0, 1188787
Total [candidate] (10.495 s) : 0, 10494702
section iast
Agent [baseline] (1.173 s) : 0, 1172557
Total [baseline] (10.678 s) : 0, 10677765
Agent [candidate] (1.17 s) : 0, 1170026
Total [candidate] (10.736 s) : 0, 10736407
section profiling
Agent [baseline] (1.281 s) : 0, 1280673
Total [baseline] (10.64 s) : 0, 10639585
Agent [candidate] (1.265 s) : 0, 1265145
Total [candidate] (10.609 s) : 0, 10609308
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.073 s -
Agent appsec 1.189 s 115.674 ms (10.8%)
Agent iast 1.173 s 99.434 ms (9.3%)
Agent profiling 1.281 s 207.55 ms (19.3%)
Total tracing 10.361 s -
Total appsec 10.518 s 157.404 ms (1.5%)
Total iast 10.678 s 316.701 ms (3.1%)
Total profiling 10.64 s 278.521 ms (2.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent appsec 1.189 s 111.932 ms (10.4%)
Agent iast 1.17 s 93.17 ms (8.7%)
Agent profiling 1.265 s 188.289 ms (17.5%)
Total tracing 10.379 s -
Total appsec 10.495 s 116.063 ms (1.1%)
Total iast 10.736 s 357.769 ms (3.4%)
Total profiling 10.609 s 230.67 ms (2.2%) 
gantt
    title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.057 ms) : 0, 671057
BytebuddyAgent [candidate] (673.203 ms) : 0, 673203
GlobalTracer [baseline] (308.189 ms) : 0, 308189
GlobalTracer [candidate] (309.742 ms) : 0, 309742
AppSec [baseline] (50.49 ms) : 0, 50490
AppSec [candidate] (50.444 ms) : 0, 50444
Remote Config [baseline] (686.585 µs) : 0, 687
Remote Config [candidate] (738.677 µs) : 0, 739
Telemetry [baseline] (7.772 ms) : 0, 7772
Telemetry [candidate] (7.749 ms) : 0, 7749
section appsec
BytebuddyAgent [baseline] (681.043 ms) : 0, 681043
BytebuddyAgent [candidate] (678.817 ms) : 0, 678817
GlobalTracer [baseline] (300.423 ms) : 0, 300423
GlobalTracer [candidate] (301.59 ms) : 0, 301590
AppSec [baseline] (154.024 ms) : 0, 154024
AppSec [candidate] (154.318 ms) : 0, 154318
Remote Config [baseline] (623.718 µs) : 0, 624
Remote Config [candidate] (636.453 µs) : 0, 636
Telemetry [baseline] (7.695 ms) : 0, 7695
Telemetry [candidate] (8.727 ms) : 0, 8727
IAST [baseline] (20.298 ms) : 0, 20298
IAST [candidate] (19.876 ms) : 0, 19876
section iast
BytebuddyAgent [baseline] (781.079 ms) : 0, 781079
BytebuddyAgent [candidate] (779.048 ms) : 0, 779048
GlobalTracer [baseline] (295.437 ms) : 0, 295437
GlobalTracer [candidate] (295.084 ms) : 0, 295084
AppSec [baseline] (47.406 ms) : 0, 47406
AppSec [candidate] (46.945 ms) : 0, 46945
Remote Config [baseline] (561.969 µs) : 0, 562
Remote Config [candidate] (581.17 µs) : 0, 581
Telemetry [baseline] (6.944 ms) : 0, 6944
Telemetry [candidate] (7.684 ms) : 0, 7684
IAST [baseline] (27.606 ms) : 0, 27606
IAST [candidate] (27.151 ms) : 0, 27151
section profiling
BytebuddyAgent [baseline] (672.685 ms) : 0, 672685
BytebuddyAgent [candidate] (663.015 ms) : 0, 663015
GlobalTracer [baseline] (393.243 ms) : 0, 393243
GlobalTracer [candidate] (389.666 ms) : 0, 389666
AppSec [baseline] (51.984 ms) : 0, 51984
AppSec [candidate] (51.297 ms) : 0, 51297
Remote Config [baseline] (666.363 µs) : 0, 666
Remote Config [candidate] (676.669 µs) : 0, 677
Telemetry [baseline] (7.465 ms) : 0, 7465
Telemetry [candidate] (7.32 ms) : 0, 7320
ProfilingAgent [baseline] (96.537 ms) : 0, 96537
ProfilingAgent [candidate] (95.984 ms) : 0, 95984
Profiling [baseline] (96.561 ms) : 0, 96561
Profiling [candidate] (96.008 ms) : 0, 96008
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066402
Total [baseline] (8.51 s) : 0, 8510460
Agent [candidate] (1.071 s) : 0, 1070718
Total [candidate] (8.506 s) : 0, 8506064
section iast
Agent [baseline] (1.181 s) : 0, 1180561
Total [baseline] (9.042 s) : 0, 9042200
Agent [candidate] (1.169 s) : 0, 1169332
Total [candidate] (8.971 s) : 0, 8970872
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.171 s) : 0, 1171490
Total [baseline] (8.91 s) : 0, 8910246
Agent [candidate] (1.18 s) : 0, 1179777
Total [candidate] (8.938 s) : 0, 8938319
section iast_TELEMETRY_OFF
Agent [baseline] (1.179 s) : 0, 1178544
Total [baseline] (8.994 s) : 0, 8993560
Agent [candidate] (1.171 s) : 0, 1170836
Total [candidate] (8.936 s) : 0, 8935602
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.181 s 114.159 ms (10.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 105.088 ms (9.9%)
Agent iast_TELEMETRY_OFF 1.179 s 112.142 ms (10.5%)
Total tracing 8.51 s -
Total iast 9.042 s 531.74 ms (6.2%)
Total iast_HARDCODED_SECRET_DISABLED 8.91 s 399.786 ms (4.7%)
Total iast_TELEMETRY_OFF 8.994 s 483.1 ms (5.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.071 s -
Agent iast 1.169 s 98.614 ms (9.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.18 s 109.059 ms (10.2%)
Agent iast_TELEMETRY_OFF 1.171 s 100.118 ms (9.4%)
Total tracing 8.506 s -
Total iast 8.971 s 464.809 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 8.938 s 432.255 ms (5.1%)
Total iast_TELEMETRY_OFF 8.936 s 429.539 ms (5.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (667.143 ms) : 0, 667143
BytebuddyAgent [candidate] (670.646 ms) : 0, 670646
GlobalTracer [baseline] (306.44 ms) : 0, 306440
GlobalTracer [candidate] (307.005 ms) : 0, 307005
AppSec [baseline] (49.805 ms) : 0, 49805
AppSec [candidate] (49.969 ms) : 0, 49969
Remote Config [baseline] (666.772 µs) : 0, 667
Remote Config [candidate] (767.475 µs) : 0, 767
Telemetry [baseline] (7.59 ms) : 0, 7590
Telemetry [candidate] (7.541 ms) : 0, 7541
section iast
BytebuddyAgent [baseline] (786.576 ms) : 0, 786576
BytebuddyAgent [candidate] (779.117 ms) : 0, 779117
GlobalTracer [baseline] (298.78 ms) : 0, 298780
GlobalTracer [candidate] (295.042 ms) : 0, 295042
AppSec [baseline] (47.726 ms) : 0, 47726
AppSec [candidate] (48.115 ms) : 0, 48115
Remote Config [baseline] (599.398 µs) : 0, 599
Remote Config [candidate] (572.482 µs) : 0, 572
Telemetry [baseline] (7.202 ms) : 0, 7202
Telemetry [candidate] (6.953 ms) : 0, 6953
IAST [baseline] (26.12 ms) : 0, 26120
IAST [candidate] (26.069 ms) : 0, 26069
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (780.125 ms) : 0, 780125
BytebuddyAgent [candidate] (785.818 ms) : 0, 785818
GlobalTracer [baseline] (295.596 ms) : 0, 295596
GlobalTracer [candidate] (297.526 ms) : 0, 297526
AppSec [baseline] (47.115 ms) : 0, 47115
AppSec [candidate] (47.352 ms) : 0, 47352
Remote Config [baseline] (555.954 µs) : 0, 556
Remote Config [candidate] (571.885 µs) : 0, 572
Telemetry [baseline] (6.93 ms) : 0, 6930
Telemetry [candidate] (7.015 ms) : 0, 7015
IAST [baseline] (27.66 ms) : 0, 27660
IAST [candidate] (27.797 ms) : 0, 27797
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (784.142 ms) : 0, 784142
BytebuddyAgent [candidate] (781.449 ms) : 0, 781449
GlobalTracer [baseline] (297.459 ms) : 0, 297459
GlobalTracer [candidate] (294.719 ms) : 0, 294719
AppSec [baseline] (47.354 ms) : 0, 47354
AppSec [candidate] (46.471 ms) : 0, 46471
Remote Config [baseline] (597.572 µs) : 0, 598
Remote Config [candidate] (571.98 µs) : 0, 572
Telemetry [baseline] (8.322 ms) : 0, 8322
Telemetry [candidate] (6.766 ms) : 0, 6766
IAST [baseline] (27.06 ms) : 0, 27060
IAST [candidate] (27.27 ms) : 0, 27270
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-07-05T14:35:45 2024-07-05T14:42:33
git_branch master jpbempel/move-snapshot-id-generation
git_commit_date 1720178709 1720189354
git_commit_sha 23a8164 692236f
release_version 1.38.0-SNAPSHOT~23a8164e90 1.38.0-SNAPSHOT~692236f00e
start_time 2024-07-05T14:35:32 2024-07-05T14:42:20
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1720190897 1720190897
ci_job_id 563600585 563600585
ci_pipeline_id 38448695 38448695
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.663 µs) : 345, 384
.   : milestone, 365,
iast (478.888 µs) : 458, 500
.   : milestone, 479,
iast_FULL (545.263 µs) : 524, 566
.   : milestone, 545,
iast_GLOBAL (498.842 µs) : 478, 520
.   : milestone, 499,
iast_HARDCODED_SECRET_DISABLED (478.704 µs) : 458, 500
.   : milestone, 479,
iast_INACTIVE (452.879 µs) : 431, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (464.808 µs) : 444, 486
.   : milestone, 465,
tracing (443.386 µs) : 422, 465
.   : milestone, 443,
section candidate
no_agent (364.38 µs) : 345, 384
.   : milestone, 364,
iast (474.197 µs) : 453, 495
.   : milestone, 474,
iast_FULL (540.69 µs) : 520, 562
.   : milestone, 541,
iast_GLOBAL (515.956 µs) : 493, 539
.   : milestone, 516,
iast_HARDCODED_SECRET_DISABLED (474.093 µs) : 453, 495
.   : milestone, 474,
iast_INACTIVE (452.564 µs) : 431, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (474.483 µs) : 453, 496
.   : milestone, 474,
tracing (438.631 µs) : 418, 459
.   : milestone, 439,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.663 µs [345.196 µs, 384.13 µs] -
iast 478.888 µs [457.533 µs, 500.243 µs] 114.225 µs (31.3%)
iast_FULL 545.263 µs [524.338 µs, 566.188 µs] 180.6 µs (49.5%)
iast_GLOBAL 498.842 µs [477.654 µs, 520.029 µs] 134.179 µs (36.8%)
iast_HARDCODED_SECRET_DISABLED 478.704 µs [457.533 µs, 499.874 µs] 114.041 µs (31.3%)
iast_INACTIVE 452.879 µs [431.306 µs, 474.453 µs] 88.216 µs (24.2%)
iast_TELEMETRY_OFF 464.808 µs [443.983 µs, 485.632 µs] 100.145 µs (27.5%)
tracing 443.386 µs [422.172 µs, 464.601 µs] 78.723 µs (21.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.38 µs [345.118 µs, 383.642 µs] -
iast 474.197 µs [452.905 µs, 495.49 µs] 109.818 µs (30.1%)
iast_FULL 540.69 µs [519.689 µs, 561.692 µs] 176.31 µs (48.4%)
iast_GLOBAL 515.956 µs [492.641 µs, 539.27 µs] 151.576 µs (41.6%)
iast_HARDCODED_SECRET_DISABLED 474.093 µs [453.238 µs, 494.947 µs] 109.713 µs (30.1%)
iast_INACTIVE 452.564 µs [430.925 µs, 474.204 µs] 88.184 µs (24.2%)
iast_TELEMETRY_OFF 474.483 µs [453.037 µs, 495.93 µs] 110.103 µs (30.2%)
tracing 438.631 µs [417.841 µs, 459.42 µs] 74.251 µs (20.4%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.349 ms) : 1329, 1369
.   : milestone, 1349,
appsec (1.719 ms) : 1695, 1743
.   : milestone, 1719,
appsec_no_iast (1.704 ms) : 1679, 1728
.   : milestone, 1704,
iast (1.48 ms) : 1457, 1502
.   : milestone, 1480,
profiling (1.495 ms) : 1471, 1518
.   : milestone, 1495,
tracing (1.449 ms) : 1425, 1474
.   : milestone, 1449,
section candidate
no_agent (1.317 ms) : 1298, 1336
.   : milestone, 1317,
appsec (1.718 ms) : 1694, 1742
.   : milestone, 1718,
appsec_no_iast (1.717 ms) : 1693, 1742
.   : milestone, 1717,
iast (1.473 ms) : 1451, 1495
.   : milestone, 1473,
profiling (1.512 ms) : 1487, 1538
.   : milestone, 1512,
tracing (1.477 ms) : 1453, 1500
.   : milestone, 1477,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.329 ms, 1.369 ms] -
appsec 1.719 ms [1.695 ms, 1.743 ms] 370.203 µs (27.4%)
appsec_no_iast 1.704 ms [1.679 ms, 1.728 ms] 354.69 µs (26.3%)
iast 1.48 ms [1.457 ms, 1.502 ms] 131.041 µs (9.7%)
profiling 1.495 ms [1.471 ms, 1.518 ms] 145.868 µs (10.8%)
tracing 1.449 ms [1.425 ms, 1.474 ms] 100.433 µs (7.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.317 ms [1.298 ms, 1.336 ms] -
appsec 1.718 ms [1.694 ms, 1.742 ms] 401.2 µs (30.5%)
appsec_no_iast 1.717 ms [1.693 ms, 1.742 ms] 400.463 µs (30.4%)
iast 1.473 ms [1.451 ms, 1.495 ms] 156.056 µs (11.8%)
profiling 1.512 ms [1.487 ms, 1.538 ms] 195.461 µs (14.8%)
tracing 1.477 ms [1.453 ms, 1.5 ms] 159.786 µs (12.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/move-snapshot-id-generation
git_commit_date 1720178709 1720189354
git_commit_sha 23a8164 692236f
release_version 1.38.0-SNAPSHOT~23a8164e90 1.38.0-SNAPSHOT~692236f00e
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1720191416 1720191416
ci_job_id 563600586 563600586
ci_pipeline_id 38448695 38448695
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1451, 1475
.   : milestone, 1463,
appsec (2.21 ms) : 2175, 2245
.   : milestone, 2210,
iast (1.978 ms) : 1936, 2019
.   : milestone, 1978,
iast_GLOBAL (2.015 ms) : 1973, 2058
.   : milestone, 2015,
profiling (1.863 ms) : 1829, 1898
.   : milestone, 1863,
tracing (1.835 ms) : 1803, 1867
.   : milestone, 1835,
section candidate
no_agent (1.459 ms) : 1447, 1470
.   : milestone, 1459,
appsec (2.222 ms) : 2187, 2257
.   : milestone, 2222,
iast (1.96 ms) : 1919, 2002
.   : milestone, 1960,
iast_GLOBAL (2.016 ms) : 1974, 2058
.   : milestone, 2016,
profiling (1.857 ms) : 1823, 1890
.   : milestone, 1857,
tracing (1.835 ms) : 1802, 1868
.   : milestone, 1835,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.463 ms [1.451 ms, 1.475 ms] -
appsec 2.21 ms [2.175 ms, 2.245 ms] 747.232 µs (51.1%)
iast 1.978 ms [1.936 ms, 2.019 ms] 514.662 µs (35.2%)
iast_GLOBAL 2.015 ms [1.973 ms, 2.058 ms] 552.181 µs (37.7%)
profiling 1.863 ms [1.829 ms, 1.898 ms] 400.023 µs (27.3%)
tracing 1.835 ms [1.803 ms, 1.867 ms] 372.01 µs (25.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.459 ms [1.447 ms, 1.47 ms] -
appsec 2.222 ms [2.187 ms, 2.257 ms] 763.297 µs (52.3%)
iast 1.96 ms [1.919 ms, 2.002 ms] 501.821 µs (34.4%)
iast_GLOBAL 2.016 ms [1.974 ms, 2.058 ms] 557.023 µs (38.2%)
profiling 1.857 ms [1.823 ms, 1.89 ms] 398.267 µs (27.3%)
tracing 1.835 ms [1.802 ms, 1.868 ms] 376.037 µs (25.8%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~692236f00e, baseline=1.38.0-SNAPSHOT~23a8164e90
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.369 s) : 15369000, 15369000
.   : milestone, 15369000,
appsec (14.823 s) : 14823000, 14823000
.   : milestone, 14823000,
iast (18.794 s) : 18794000, 18794000
.   : milestone, 18794000,
iast_GLOBAL (17.926 s) : 17926000, 17926000
.   : milestone, 17926000,
profiling (15.287 s) : 15287000, 15287000
.   : milestone, 15287000,
tracing (14.86 s) : 14860000, 14860000
.   : milestone, 14860000,
section candidate
no_agent (15.449 s) : 15449000, 15449000
.   : milestone, 15449000,
appsec (14.812 s) : 14812000, 14812000
.   : milestone, 14812000,
iast (18.864 s) : 18864000, 18864000
.   : milestone, 18864000,
iast_GLOBAL (17.986 s) : 17986000, 17986000
.   : milestone, 17986000,
profiling (15.421 s) : 15421000, 15421000
.   : milestone, 15421000,
tracing (15.181 s) : 15181000, 15181000
.   : milestone, 15181000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.369 s [15.369 s, 15.369 s] -
appsec 14.823 s [14.823 s, 14.823 s] -546.0 ms (-3.6%)
iast 18.794 s [18.794 s, 18.794 s] 3.425 s (22.3%)
iast_GLOBAL 17.926 s [17.926 s, 17.926 s] 2.557 s (16.6%)
profiling 15.287 s [15.287 s, 15.287 s] -82.0 ms (-0.5%)
tracing 14.86 s [14.86 s, 14.86 s] -509.0 ms (-3.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.449 s [15.449 s, 15.449 s] -
appsec 14.812 s [14.812 s, 14.812 s] -637.0 ms (-4.1%)
iast 18.864 s [18.864 s, 18.864 s] 3.415 s (22.1%)
iast_GLOBAL 17.986 s [17.986 s, 17.986 s] 2.537 s (16.4%)
profiling 15.421 s [15.421 s, 15.421 s] -28.0 ms (-0.2%)
tracing 15.181 s [15.181 s, 15.181 s] -268.0 ms (-1.7%)

@jpbempel jpbempel merged commit 3aea13c into master Jul 5, 2024
83 checks passed
@jpbempel jpbempel deleted the jpbempel/move-snapshot-id-generation branch July 5, 2024 15:40
@github-actions github-actions bot added this to the 1.38.0 milestone Jul 5, 2024
@jpbempel jpbempel added the tag: performance Performance related changes label Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cimi cimi cimi approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation tag: performance Performance related changes
Projects
None yet
Milestone
1.38.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@jpbempel @cimi