Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Report telemetry metrics for Exploit Prevention #7314

Merged
merged 7 commits into from
Jul 18, 2024
Merged

Report telemetry metrics for Exploit Prevention #7314

merged 7 commits into from
Jul 18, 2024

Conversation

ValentinZakharov
Copy link
Contributor

@ValentinZakharov ValentinZakharov commented Jul 12, 2024 •
edited by jira bot
Loading

What Does This Do

Introduced telemetry metrics for Exploit Prevention.

  • appsec.rasp.rule.eval - counts the number of times a rule type is evaluated
  • appsec.rasp.rule.match - counts the number of times a rule type has a match
  • appsec.rasp.timeout - counts the number of times a timeout was hit when evaluating a specific rule type

Motivation

This is part of Exploit prevention to let collect and report telemetry metrics

Additional Notes

Jira ticket: APPSEC-47228

@smola smola added the comp: asm waf Application Security Management (WAF) label Jul 12, 2024
@ValentinZakharov ValentinZakharov force-pushed the vzakharov/rasp_telemetry_metrics branch from d38aefd to 2cbdc6c Compare July 12, 2024 15:42
@pr-commenter
Copy link

pr-commenter bot commented Jul 12, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master vzakharov/rasp_telemetry_metrics
git_commit_date 1721227585 1721296010
git_commit_sha 2e9ba7a 289f2ac
release_version 1.38.0-SNAPSHOT~2e9ba7a643 1.38.0-SNAPSHOT~289f2ac83d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1721298526 1721298526
ci_job_id 576705048 576705048
ci_pipeline_id 39518811 39518811
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1063975
Total [baseline] (8.55 s) : 0, 8549641
Agent [candidate] (1.064 s) : 0, 1064332
Total [candidate] (8.557 s) : 0, 8556589
section iast
Agent [baseline] (1.187 s) : 0, 1186794
Total [baseline] (9.055 s) : 0, 9054979
Agent [candidate] (1.174 s) : 0, 1174455
Total [candidate] (9.072 s) : 0, 9071698
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.171 s) : 0, 1171312
Total [baseline] (8.93 s) : 0, 8930432
Agent [candidate] (1.175 s) : 0, 1174870
Total [candidate] (8.951 s) : 0, 8950521
section iast_TELEMETRY_OFF
Agent [baseline] (1.17 s) : 0, 1170469
Total [baseline] (8.97 s) : 0, 8969944
Agent [candidate] (1.179 s) : 0, 1178989
Total [candidate] (9.009 s) : 0, 9009431
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.187 s 122.818 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 107.337 ms (10.1%)
Agent iast_TELEMETRY_OFF 1.17 s 106.494 ms (10.0%)
Total tracing 8.55 s -
Total iast 9.055 s 505.337 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 8.93 s 380.79 ms (4.5%)
Total iast_TELEMETRY_OFF 8.97 s 420.303 ms (4.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.174 s 110.123 ms (10.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.175 s 110.538 ms (10.4%)
Agent iast_TELEMETRY_OFF 1.179 s 114.657 ms (10.8%)
Total tracing 8.557 s -
Total iast 9.072 s 515.109 ms (6.0%)
Total iast_HARDCODED_SECRET_DISABLED 8.951 s 393.932 ms (4.6%)
Total iast_TELEMETRY_OFF 9.009 s 452.842 ms (5.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.844 ms) : 0, 665844
BytebuddyAgent [candidate] (666.231 ms) : 0, 666231
GlobalTracer [baseline] (305.067 ms) : 0, 305067
GlobalTracer [candidate] (304.938 ms) : 0, 304938
AppSec [baseline] (50.188 ms) : 0, 50188
AppSec [candidate] (50.199 ms) : 0, 50199
Remote Config [baseline] (675.071 µs) : 0, 675
Remote Config [candidate] (667.122 µs) : 0, 667
Telemetry [baseline] (7.688 ms) : 0, 7688
Telemetry [candidate] (7.67 ms) : 0, 7670
section iast
BytebuddyAgent [baseline] (791.634 ms) : 0, 791634
BytebuddyAgent [candidate] (781.974 ms) : 0, 781974
GlobalTracer [baseline] (298.582 ms) : 0, 298582
GlobalTracer [candidate] (295.719 ms) : 0, 295719
AppSec [baseline] (48.829 ms) : 0, 48829
AppSec [candidate] (48.313 ms) : 0, 48313
IAST [baseline] (25.698 ms) : 0, 25698
IAST [candidate] (27.308 ms) : 0, 27308
Remote Config [baseline] (589.207 µs) : 0, 589
Remote Config [candidate] (592.536 µs) : 0, 593
Telemetry [baseline] (7.816 ms) : 0, 7816
Telemetry [candidate] (7.07 ms) : 0, 7070
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (780.017 ms) : 0, 780017
BytebuddyAgent [candidate] (782.22 ms) : 0, 782220
GlobalTracer [baseline] (295.267 ms) : 0, 295267
GlobalTracer [candidate] (295.544 ms) : 0, 295544
AppSec [baseline] (50.52 ms) : 0, 50520
AppSec [candidate] (50.594 ms) : 0, 50594
IAST [baseline] (23.74 ms) : 0, 23740
IAST [candidate] (25.366 ms) : 0, 25366
Remote Config [baseline] (563.639 µs) : 0, 564
Remote Config [candidate] (561.407 µs) : 0, 561
Telemetry [baseline] (7.682 ms) : 0, 7682
Telemetry [candidate] (6.984 ms) : 0, 6984
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.155 ms) : 0, 779155
BytebuddyAgent [candidate] (783.862 ms) : 0, 783862
GlobalTracer [baseline] (295.513 ms) : 0, 295513
GlobalTracer [candidate] (297.174 ms) : 0, 297174
AppSec [baseline] (47.591 ms) : 0, 47591
AppSec [candidate] (47.761 ms) : 0, 47761
IAST [baseline] (26.381 ms) : 0, 26381
IAST [candidate] (28.296 ms) : 0, 28296
Remote Config [baseline] (581.331 µs) : 0, 581
Remote Config [candidate] (587.904 µs) : 0, 588
Telemetry [baseline] (7.645 ms) : 0, 7645
Telemetry [candidate] (7.665 ms) : 0, 7665
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063229
Total [baseline] (10.376 s) : 0, 10376154
Agent [candidate] (1.065 s) : 0, 1064973
Total [candidate] (10.443 s) : 0, 10442803
section appsec
Agent [baseline] (1.183 s) : 0, 1183494
Total [baseline] (10.541 s) : 0, 10540850
Agent [candidate] (1.19 s) : 0, 1190138
Total [candidate] (10.555 s) : 0, 10554823
section iast
Agent [baseline] (1.172 s) : 0, 1171977
Total [baseline] (10.759 s) : 0, 10759104
Agent [candidate] (1.173 s) : 0, 1173188
Total [candidate] (10.778 s) : 0, 10777699
section profiling
Agent [baseline] (1.271 s) : 0, 1271446
Total [baseline] (10.626 s) : 0, 10626496
Agent [candidate] (1.264 s) : 0, 1263664
Total [candidate] (10.603 s) : 0, 10602926
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.183 s 120.265 ms (11.3%)
Agent iast 1.172 s 108.748 ms (10.2%)
Agent profiling 1.271 s 208.217 ms (19.6%)
Total tracing 10.376 s -
Total appsec 10.541 s 164.695 ms (1.6%)
Total iast 10.759 s 382.95 ms (3.7%)
Total profiling 10.626 s 250.342 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent appsec 1.19 s 125.164 ms (11.8%)
Agent iast 1.173 s 108.215 ms (10.2%)
Agent profiling 1.264 s 198.691 ms (18.7%)
Total tracing 10.443 s -
Total appsec 10.555 s 112.021 ms (1.1%)
Total iast 10.778 s 334.896 ms (3.2%)
Total profiling 10.603 s 160.123 ms (1.5%) 
gantt
    title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.615 ms) : 0, 665615
BytebuddyAgent [candidate] (666.532 ms) : 0, 666532
GlobalTracer [baseline] (304.723 ms) : 0, 304723
GlobalTracer [candidate] (305.308 ms) : 0, 305308
AppSec [baseline] (49.964 ms) : 0, 49964
AppSec [candidate] (50.12 ms) : 0, 50120
Remote Config [baseline] (660.076 µs) : 0, 660
Remote Config [candidate] (674.906 µs) : 0, 675
Telemetry [baseline] (7.657 ms) : 0, 7657
Telemetry [candidate] (7.627 ms) : 0, 7627
section appsec
BytebuddyAgent [baseline] (676.764 ms) : 0, 676764
BytebuddyAgent [candidate] (681.443 ms) : 0, 681443
GlobalTracer [baseline] (299.148 ms) : 0, 299148
GlobalTracer [candidate] (301.112 ms) : 0, 301112
AppSec [baseline] (153.588 ms) : 0, 153588
AppSec [candidate] (154.404 ms) : 0, 154404
Remote Config [baseline] (623.366 µs) : 0, 623
Remote Config [candidate] (628.705 µs) : 0, 629
Telemetry [baseline] (8.251 ms) : 0, 8251
Telemetry [candidate] (7.773 ms) : 0, 7773
IAST [baseline] (20.844 ms) : 0, 20844
IAST [candidate] (19.838 ms) : 0, 19838
section iast
BytebuddyAgent [baseline] (781.062 ms) : 0, 781062
BytebuddyAgent [candidate] (780.755 ms) : 0, 780755
GlobalTracer [baseline] (296.003 ms) : 0, 296003
GlobalTracer [candidate] (296.053 ms) : 0, 296053
AppSec [baseline] (47.386 ms) : 0, 47386
AppSec [candidate] (47.307 ms) : 0, 47307
Remote Config [baseline] (579.966 µs) : 0, 580
Remote Config [candidate] (576.106 µs) : 0, 576
Telemetry [baseline] (6.986 ms) : 0, 6986
Telemetry [candidate] (7.647 ms) : 0, 7647
IAST [baseline] (26.411 ms) : 0, 26411
IAST [candidate] (27.314 ms) : 0, 27314
section profiling
BytebuddyAgent [baseline] (667.064 ms) : 0, 667064
BytebuddyAgent [candidate] (662.804 ms) : 0, 662804
GlobalTracer [baseline] (389.986 ms) : 0, 389986
GlobalTracer [candidate] (387.774 ms) : 0, 387774
AppSec [baseline] (51.773 ms) : 0, 51773
AppSec [candidate] (51.663 ms) : 0, 51663
Remote Config [baseline] (653.496 µs) : 0, 653
Remote Config [candidate] (655.495 µs) : 0, 655
Telemetry [baseline] (7.408 ms) : 0, 7408
Telemetry [candidate] (7.421 ms) : 0, 7421
ProfilingAgent [baseline] (96.953 ms) : 0, 96953
ProfilingAgent [candidate] (96.239 ms) : 0, 96239
Profiling [baseline] (96.978 ms) : 0, 96978
Profiling [candidate] (96.264 ms) : 0, 96264
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-07-18T09:59:31 2024-07-18T10:06:21
git_branch master vzakharov/rasp_telemetry_metrics
git_commit_date 1721227585 1721296010
git_commit_sha 2e9ba7a 289f2ac
release_version 1.38.0-SNAPSHOT~2e9ba7a643 1.38.0-SNAPSHOT~289f2ac83d
start_time 2024-07-18T09:59:18 2024-07-18T10:06:08
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1721297526 1721297526
ci_job_id 576705049 576705049
ci_pipeline_id 39518811 39518811
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.924 µs) : 345, 386
.   : milestone, 366,
iast (478.179 µs) : 457, 499
.   : milestone, 478,
iast_FULL (550.337 µs) : 529, 571
.   : milestone, 550,
iast_GLOBAL (505.253 µs) : 483, 527
.   : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (482.669 µs) : 461, 504
.   : milestone, 483,
iast_INACTIVE (461.514 µs) : 440, 483
.   : milestone, 462,
iast_TELEMETRY_OFF (468.12 µs) : 447, 490
.   : milestone, 468,
tracing (445.002 µs) : 424, 466
.   : milestone, 445,
section candidate
no_agent (367.064 µs) : 347, 387
.   : milestone, 367,
iast (476.601 µs) : 455, 498
.   : milestone, 477,
iast_FULL (552.55 µs) : 531, 574
.   : milestone, 553,
iast_GLOBAL (513.779 µs) : 491, 537
.   : milestone, 514,
iast_HARDCODED_SECRET_DISABLED (481.79 µs) : 461, 503
.   : milestone, 482,
iast_INACTIVE (452.228 µs) : 431, 473
.   : milestone, 452,
iast_TELEMETRY_OFF (478.414 µs) : 456, 500
.   : milestone, 478,
tracing (438.922 µs) : 419, 459
.   : milestone, 439,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.924 µs [345.364 µs, 386.484 µs] -
iast 478.179 µs [457.046 µs, 499.312 µs] 112.255 µs (30.7%)
iast_FULL 550.337 µs [529.235 µs, 571.44 µs] 184.414 µs (50.4%)
iast_GLOBAL 505.253 µs [483.415 µs, 527.091 µs] 139.329 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED 482.669 µs [461.318 µs, 504.021 µs] 116.745 µs (31.9%)
iast_INACTIVE 461.514 µs [439.955 µs, 483.072 µs] 95.59 µs (26.1%)
iast_TELEMETRY_OFF 468.12 µs [446.622 µs, 489.619 µs] 102.196 µs (27.9%)
tracing 445.002 µs [423.715 µs, 466.288 µs] 79.078 µs (21.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.064 µs [346.749 µs, 387.378 µs] -
iast 476.601 µs [455.112 µs, 498.089 µs] 109.537 µs (29.8%)
iast_FULL 552.55 µs [531.062 µs, 574.038 µs] 185.486 µs (50.5%)
iast_GLOBAL 513.779 µs [490.786 µs, 536.772 µs] 146.716 µs (40.0%)
iast_HARDCODED_SECRET_DISABLED 481.79 µs [460.824 µs, 502.755 µs] 114.726 µs (31.3%)
iast_INACTIVE 452.228 µs [431.183 µs, 473.273 µs] 85.164 µs (23.2%)
iast_TELEMETRY_OFF 478.414 µs [456.432 µs, 500.396 µs] 111.35 µs (30.3%)
tracing 438.922 µs [418.593 µs, 459.252 µs] 71.859 µs (19.6%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.746 ms) : 1722, 1769
.   : milestone, 1746,
appsec_no_iast (1.745 ms) : 1721, 1769
.   : milestone, 1745,
iast (1.472 ms) : 1450, 1495
.   : milestone, 1472,
profiling (1.509 ms) : 1483, 1535
.   : milestone, 1509,
tracing (1.474 ms) : 1450, 1498
.   : milestone, 1474,
section candidate
no_agent (1.343 ms) : 1324, 1363
.   : milestone, 1343,
appsec (1.699 ms) : 1674, 1723
.   : milestone, 1699,
appsec_no_iast (1.718 ms) : 1694, 1742
.   : milestone, 1718,
iast (1.461 ms) : 1439, 1483
.   : milestone, 1461,
profiling (1.489 ms) : 1463, 1516
.   : milestone, 1489,
tracing (1.439 ms) : 1414, 1463
.   : milestone, 1439,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.746 ms [1.722 ms, 1.769 ms] 400.56 µs (29.8%)
appsec_no_iast 1.745 ms [1.721 ms, 1.769 ms] 400.206 µs (29.8%)
iast 1.472 ms [1.45 ms, 1.495 ms] 127.407 µs (9.5%)
profiling 1.509 ms [1.483 ms, 1.535 ms] 163.786 µs (12.2%)
tracing 1.474 ms [1.45 ms, 1.498 ms] 128.872 µs (9.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.324 ms, 1.363 ms] -
appsec 1.699 ms [1.674 ms, 1.723 ms] 355.251 µs (26.4%)
appsec_no_iast 1.718 ms [1.694 ms, 1.742 ms] 374.604 µs (27.9%)
iast 1.461 ms [1.439 ms, 1.483 ms] 117.655 µs (8.8%)
profiling 1.489 ms [1.463 ms, 1.516 ms] 146.03 µs (10.9%)
tracing 1.439 ms [1.414 ms, 1.463 ms] 95.152 µs (7.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master vzakharov/rasp_telemetry_metrics
git_commit_date 1721227585 1721296010
git_commit_sha 2e9ba7a 289f2ac
release_version 1.38.0-SNAPSHOT~2e9ba7a643 1.38.0-SNAPSHOT~289f2ac83d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1721298035 1721298035
ci_job_id 576705051 576705051
ci_pipeline_id 39518811 39518811
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.248 s) : 15248000, 15248000
.   : milestone, 15248000,
appsec (15.153 s) : 15153000, 15153000
.   : milestone, 15153000,
iast (18.918 s) : 18918000, 18918000
.   : milestone, 18918000,
iast_GLOBAL (17.844 s) : 17844000, 17844000
.   : milestone, 17844000,
profiling (15.806 s) : 15806000, 15806000
.   : milestone, 15806000,
tracing (14.914 s) : 14914000, 14914000
.   : milestone, 14914000,
section candidate
no_agent (15.494 s) : 15494000, 15494000
.   : milestone, 15494000,
appsec (15.057 s) : 15057000, 15057000
.   : milestone, 15057000,
iast (18.847 s) : 18847000, 18847000
.   : milestone, 18847000,
iast_GLOBAL (17.627 s) : 17627000, 17627000
.   : milestone, 17627000,
profiling (15.207 s) : 15207000, 15207000
.   : milestone, 15207000,
tracing (14.907 s) : 14907000, 14907000
.   : milestone, 14907000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.248 s [15.248 s, 15.248 s] -
appsec 15.153 s [15.153 s, 15.153 s] -95.0 ms (-0.6%)
iast 18.918 s [18.918 s, 18.918 s] 3.67 s (24.1%)
iast_GLOBAL 17.844 s [17.844 s, 17.844 s] 2.596 s (17.0%)
profiling 15.806 s [15.806 s, 15.806 s] 558.0 ms (3.7%)
tracing 14.914 s [14.914 s, 14.914 s] -334.0 ms (-2.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.494 s [15.494 s, 15.494 s] -
appsec 15.057 s [15.057 s, 15.057 s] -437.0 ms (-2.8%)
iast 18.847 s [18.847 s, 18.847 s] 3.353 s (21.6%)
iast_GLOBAL 17.627 s [17.627 s, 17.627 s] 2.133 s (13.8%)
profiling 15.207 s [15.207 s, 15.207 s] -287.0 ms (-1.9%)
tracing 14.907 s [14.907 s, 14.907 s] -587.0 ms (-3.8%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~289f2ac83d, baseline=1.38.0-SNAPSHOT~2e9ba7a643
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.453 ms) : 1441, 1464
.   : milestone, 1453,
appsec (2.217 ms) : 2182, 2252
.   : milestone, 2217,
iast (1.949 ms) : 1908, 1990
.   : milestone, 1949,
iast_GLOBAL (2.011 ms) : 1967, 2054
.   : milestone, 2011,
profiling (1.851 ms) : 1817, 1886
.   : milestone, 1851,
tracing (1.828 ms) : 1795, 1861
.   : milestone, 1828,
section candidate
no_agent (1.451 ms) : 1440, 1462
.   : milestone, 1451,
appsec (2.201 ms) : 2166, 2236
.   : milestone, 2201,
iast (1.956 ms) : 1915, 1998
.   : milestone, 1956,
iast_GLOBAL (2.009 ms) : 1966, 2052
.   : milestone, 2009,
profiling (1.84 ms) : 1807, 1874
.   : milestone, 1840,
tracing (1.828 ms) : 1795, 1861
.   : milestone, 1828,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.453 ms [1.441 ms, 1.464 ms] -
appsec 2.217 ms [2.182 ms, 2.252 ms] 764.147 µs (52.6%)
iast 1.949 ms [1.908 ms, 1.99 ms] 496.336 µs (34.2%)
iast_GLOBAL 2.011 ms [1.967 ms, 2.054 ms] 558.136 µs (38.4%)
profiling 1.851 ms [1.817 ms, 1.886 ms] 398.862 µs (27.5%)
tracing 1.828 ms [1.795 ms, 1.861 ms] 375.558 µs (25.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.451 ms [1.44 ms, 1.462 ms] -
appsec 2.201 ms [2.166 ms, 2.236 ms] 749.752 µs (51.7%)
iast 1.956 ms [1.915 ms, 1.998 ms] 505.169 µs (34.8%)
iast_GLOBAL 2.009 ms [1.966 ms, 2.052 ms] 557.794 µs (38.4%)
profiling 1.84 ms [1.807 ms, 1.874 ms] 389.151 µs (26.8%)
tracing 1.828 ms [1.795 ms, 1.861 ms] 376.543 µs (25.9%)

@ValentinZakharov ValentinZakharov marked this pull request as ready for review July 15, 2024 15:44
@ValentinZakharov ValentinZakharov requested review from a team as code owners July 15, 2024 15:44
@ValentinZakharov ValentinZakharov force-pushed the vzakharov/rasp_telemetry_metrics branch from 986c98e to 9e945d9 Compare July 16, 2024 20:47
@ValentinZakharov ValentinZakharov force-pushed the vzakharov/rasp_telemetry_metrics branch from b4e02c2 to c0d353f Compare July 17, 2024 12:25
jandro996
jandro996 approved these changes Jul 17, 2024
View reviewed changes
Copy link
Member

@jandro996 jandro996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related with this PR

IMHO, we need to think in refactor the GatewayBridge in some way, as we will implement new rasp features seems that this class will contain a lot of logic for different purpose

mcculls
mcculls reviewed Jul 18, 2024
View reviewed changes
internal-api/src/main/java/datadog/trace/api/telemetry/WafMetricCollector.java Outdated
new EnumMap<>(RuleType.class);
private static final Map<RuleType, AtomicLong> raspRuleMatchCounter =
new EnumMap<>(RuleType.class);
private static final Map<RuleType, AtomicLong> respTimeoutCounter = new EnumMap<>(RuleType.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since RuleType is an enum and there aren't many entries you could replace each of these counter maps with an AtomicLongArray with a size of RuleType.values().length (you can add a method to the enum to make it more readable, like in https://github.com/DataDog/dd-trace-java/blob/master/dd-trace-core/src/main/java/datadog/trace/core/propagation/PropagationTags.java#L42)

Then just use the ordinal from the enum to access the appropriate element in each AtomicLongArray.

mcculls
mcculls requested changes Jul 18, 2024
View reviewed changes
Copy link
Contributor

@mcculls mcculls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, but I'd like to see each EnumMap of AtomicLong replaced by an AtomicLongArray - this simplifies the storage, especially since EnumMap is not thread-safe and would require synchronization around computeIfAbsent

@ValentinZakharov ValentinZakharov merged commit 144efa8 into master Jul 18, 2024
82 checks passed
@ValentinZakharov ValentinZakharov deleted the vzakharov/rasp_telemetry_metrics branch July 18, 2024 11:53
@github-actions github-actions bot added this to the 1.38.0 milestone Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@mcculls mcculls mcculls approved these changes

@jandro996 jandro996 jandro996 approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: asm waf Application Security Management (WAF)
Projects
None yet
Milestone
1.38.0
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@ValentinZakharov @smola @mcculls @jandro996