Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refine static analysis rules #7316

Merged
merged 3 commits into from
Jul 16, 2024
Merged

Refine static analysis rules #7316

merged 3 commits into from
Jul 16, 2024

Conversation

smola
Copy link
Member

@smola smola commented Jul 12, 2024

What Does This Do

Refine static analysis rules configuration, with a few things that seem to be irrelevant or noisy in our project.

I'm not excluding some noisy rules in cases where I reported false positives to be fixed in the rules themselves.

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Jul 12, 2024
@smola smola requested a review from a team as a code owner July 12, 2024 10:29
@pr-commenter
Copy link

pr-commenter bot commented Jul 12, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/dd-static-rules-refine
git_commit_date 1721056709 1721056870
git_commit_sha 9184826 bb0f2cc
release_version 1.38.0-SNAPSHOT~9184826036 1.38.0-SNAPSHOT~bb0f2cc81c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1721059325 1721059325
ci_job_id 572543883 572543883
ci_pipeline_id 39167992 39167992
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 10 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~bb0f2cc81c, baseline=1.38.0-SNAPSHOT~9184826036

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1066625
Total [baseline] (10.342 s) : 0, 10342247
Agent [candidate] (1.065 s) : 0, 1065252
Total [candidate] (10.318 s) : 0, 10317551
section appsec
Agent [baseline] (1.186 s) : 0, 1186453
Total [baseline] (10.598 s) : 0, 10598092
Agent [candidate] (1.185 s) : 0, 1184794
Total [candidate] (10.509 s) : 0, 10509315
section iast
Agent [baseline] (1.172 s) : 0, 1172471
Total [baseline] (10.84 s) : 0, 10839524
Agent [candidate] (1.18 s) : 0, 1179846
Total [candidate] (10.713 s) : 0, 10713126
section profiling
Agent [baseline] (1.267 s) : 0, 1266930
Total [baseline] (10.611 s) : 0, 10610901
Agent [candidate] (1.268 s) : 0, 1267503
Total [candidate] (10.635 s) : 0, 10635311
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent appsec 1.186 s 119.828 ms (11.2%)
Agent iast 1.172 s 105.846 ms (9.9%)
Agent profiling 1.267 s 200.306 ms (18.8%)
Total tracing 10.342 s -
Total appsec 10.598 s 255.845 ms (2.5%)
Total iast 10.84 s 497.277 ms (4.8%)
Total profiling 10.611 s 268.655 ms (2.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent appsec 1.185 s 119.541 ms (11.2%)
Agent iast 1.18 s 114.594 ms (10.8%)
Agent profiling 1.268 s 202.25 ms (19.0%)
Total tracing 10.318 s -
Total appsec 10.509 s 191.764 ms (1.9%)
Total iast 10.713 s 395.575 ms (3.8%)
Total profiling 10.635 s 317.76 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~bb0f2cc81c, baseline=1.38.0-SNAPSHOT~9184826036

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (667.706 ms) : 0, 667706
BytebuddyAgent [candidate] (666.703 ms) : 0, 666703
GlobalTracer [baseline] (305.963 ms) : 0, 305963
GlobalTracer [candidate] (305.488 ms) : 0, 305488
AppSec [baseline] (49.976 ms) : 0, 49976
AppSec [candidate] (49.984 ms) : 0, 49984
Remote Config [baseline] (675.612 µs) : 0, 676
Remote Config [candidate] (670.746 µs) : 0, 671
Telemetry [baseline] (7.643 ms) : 0, 7643
Telemetry [candidate] (7.611 ms) : 0, 7611
section appsec
BytebuddyAgent [baseline] (679.563 ms) : 0, 679563
BytebuddyAgent [candidate] (677.562 ms) : 0, 677562
GlobalTracer [baseline] (299.399 ms) : 0, 299399
GlobalTracer [candidate] (299.856 ms) : 0, 299856
AppSec [baseline] (153.856 ms) : 0, 153856
AppSec [candidate] (153.817 ms) : 0, 153817
Remote Config [baseline] (621.059 µs) : 0, 621
Remote Config [candidate] (622.826 µs) : 0, 623
Telemetry [baseline] (8.338 ms) : 0, 8338
Telemetry [candidate] (8.349 ms) : 0, 8349
IAST [baseline] (19.627 ms) : 0, 19627
IAST [candidate] (19.676 ms) : 0, 19676
section iast
BytebuddyAgent [baseline] (781.432 ms) : 0, 781432
BytebuddyAgent [candidate] (785.533 ms) : 0, 785533
GlobalTracer [baseline] (296.242 ms) : 0, 296242
GlobalTracer [candidate] (297.375 ms) : 0, 297375
AppSec [baseline] (48.942 ms) : 0, 48942
AppSec [candidate] (48.255 ms) : 0, 48255
Remote Config [baseline] (568.699 µs) : 0, 569
Remote Config [candidate] (586.784 µs) : 0, 587
Telemetry [baseline] (6.945 ms) : 0, 6945
Telemetry [candidate] (7.005 ms) : 0, 7005
IAST [baseline] (24.767 ms) : 0, 24767
IAST [candidate] (27.461 ms) : 0, 27461
section profiling
BytebuddyAgent [baseline] (663.841 ms) : 0, 663841
BytebuddyAgent [candidate] (664.417 ms) : 0, 664417
GlobalTracer [baseline] (388.814 ms) : 0, 388814
GlobalTracer [candidate] (389.393 ms) : 0, 389393
AppSec [baseline] (52.07 ms) : 0, 52070
AppSec [candidate] (51.985 ms) : 0, 51985
Remote Config [baseline] (652.972 µs) : 0, 653
Remote Config [candidate] (653.373 µs) : 0, 653
Telemetry [baseline] (7.324 ms) : 0, 7324
Telemetry [candidate] (7.305 ms) : 0, 7305
ProfilingAgent [baseline] (97.004 ms) : 0, 97004
ProfilingAgent [candidate] (96.295 ms) : 0, 96295
Profiling [baseline] (97.03 ms) : 0, 97030
Profiling [candidate] (96.32 ms) : 0, 96320
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~bb0f2cc81c, baseline=1.38.0-SNAPSHOT~9184826036

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1065416
Total [baseline] (8.513 s) : 0, 8513026
Agent [candidate] (1.079 s) : 0, 1079016
Total [candidate] (8.578 s) : 0, 8578399
section iast
Agent [baseline] (1.187 s) : 0, 1186522
Total [baseline] (9.008 s) : 0, 9007699
Agent [candidate] (1.183 s) : 0, 1182759
Total [candidate] (8.978 s) : 0, 8977687
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.18 s) : 0, 1179885
Total [baseline] (8.924 s) : 0, 8924421
Agent [candidate] (1.172 s) : 0, 1172042
Total [candidate] (8.934 s) : 0, 8934155
section iast_TELEMETRY_OFF
Agent [baseline] (1.181 s) : 0, 1180725
Total [baseline] (8.985 s) : 0, 8985264
Agent [candidate] (1.169 s) : 0, 1168679
Total [candidate] (8.964 s) : 0, 8964050
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent iast 1.187 s 121.106 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.18 s 114.469 ms (10.7%)
Agent iast_TELEMETRY_OFF 1.181 s 115.309 ms (10.8%)
Total tracing 8.513 s -
Total iast 9.008 s 494.673 ms (5.8%)
Total iast_HARDCODED_SECRET_DISABLED 8.924 s 411.395 ms (4.8%)
Total iast_TELEMETRY_OFF 8.985 s 472.237 ms (5.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.079 s -
Agent iast 1.183 s 103.743 ms (9.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 93.026 ms (8.6%)
Agent iast_TELEMETRY_OFF 1.169 s 89.663 ms (8.3%)
Total tracing 8.578 s -
Total iast 8.978 s 399.288 ms (4.7%)
Total iast_HARDCODED_SECRET_DISABLED 8.934 s 355.756 ms (4.1%)
Total iast_TELEMETRY_OFF 8.964 s 385.651 ms (4.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~bb0f2cc81c, baseline=1.38.0-SNAPSHOT~9184826036

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (667.021 ms) : 0, 667021
BytebuddyAgent [candidate] (675.826 ms) : 0, 675826
GlobalTracer [baseline] (305.5 ms) : 0, 305500
GlobalTracer [candidate] (309.175 ms) : 0, 309175
AppSec [baseline] (50.04 ms) : 0, 50040
AppSec [candidate] (50.479 ms) : 0, 50479
Remote Config [baseline] (685.339 µs) : 0, 685
Remote Config [candidate] (687.863 µs) : 0, 688
Telemetry [baseline] (7.629 ms) : 0, 7629
Telemetry [candidate] (7.813 ms) : 0, 7813
section iast
BytebuddyAgent [baseline] (792.813 ms) : 0, 792813
BytebuddyAgent [candidate] (789.568 ms) : 0, 789568
GlobalTracer [baseline] (297.584 ms) : 0, 297584
GlobalTracer [candidate] (297.767 ms) : 0, 297767
AppSec [baseline] (50.182 ms) : 0, 50182
AppSec [candidate] (49.426 ms) : 0, 49426
Remote Config [baseline] (583.958 µs) : 0, 584
Remote Config [candidate] (583.695 µs) : 0, 584
Telemetry [baseline] (6.968 ms) : 0, 6968
Telemetry [candidate] (7.003 ms) : 0, 7003
IAST [baseline] (24.716 ms) : 0, 24716
IAST [candidate] (24.772 ms) : 0, 24772
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (784.873 ms) : 0, 784873
BytebuddyAgent [candidate] (780.987 ms) : 0, 780987
GlobalTracer [baseline] (297.206 ms) : 0, 297206
GlobalTracer [candidate] (295.647 ms) : 0, 295647
AppSec [baseline] (48.559 ms) : 0, 48559
AppSec [candidate] (49.71 ms) : 0, 49710
Remote Config [baseline] (579.006 µs) : 0, 579
Remote Config [candidate] (565.744 µs) : 0, 566
Telemetry [baseline] (7.016 ms) : 0, 7016
Telemetry [candidate] (6.977 ms) : 0, 6977
IAST [baseline] (28.071 ms) : 0, 28071
IAST [candidate] (24.59 ms) : 0, 24590
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (785.934 ms) : 0, 785934
BytebuddyAgent [candidate] (778.076 ms) : 0, 778076
GlobalTracer [baseline] (298.359 ms) : 0, 298359
GlobalTracer [candidate] (294.998 ms) : 0, 294998
AppSec [baseline] (47.77 ms) : 0, 47770
AppSec [candidate] (47.195 ms) : 0, 47195
Remote Config [baseline] (576.078 µs) : 0, 576
Remote Config [candidate] (576.941 µs) : 0, 577
Telemetry [baseline] (7.613 ms) : 0, 7613
Telemetry [candidate] (7.648 ms) : 0, 7648
IAST [baseline] (26.797 ms) : 0, 26797
IAST [candidate] (26.619 ms) : 0, 26619
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-07-15T15:32:48 2024-07-15T15:39:36
git_branch master smola/dd-static-rules-refine
git_commit_date 1721056709 1721056870
git_commit_sha 9184826 bb0f2cc
release_version 1.38.0-SNAPSHOT~9184826036 1.38.0-SNAPSHOT~bb0f2cc81c
start_time 2024-07-15T15:32:35 2024-07-15T15:39:23
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1721058322 1721058322
ci_job_id 572543885 572543885
ci_pipeline_id 39167992 39167992
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~bb0f2cc81c, baseline=1.38.0-SNAPSHOT~9184826036
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.98 µs) : 353, 393
.   : milestone, 373,
iast (486.674 µs) : 465, 508
.   : milestone, 487,
iast_FULL (548.295 µs) : 527, 570
.   : milestone, 548,
iast_GLOBAL (505.455 µs) : 484, 527
.   : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (486.036 µs) : 464, 508
.   : milestone, 486,
iast_INACTIVE (448.09 µs) : 427, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (469.345 µs) : 448, 490
.   : milestone, 469,
tracing (444.753 µs) : 423, 466
.   : milestone, 445,
section candidate
no_agent (367.71 µs) : 348, 387
.   : milestone, 368,
iast (476.159 µs) : 455, 497
.   : milestone, 476,
iast_FULL (547.649 µs) : 527, 569
.   : milestone, 548,
iast_GLOBAL (517.052 µs) : 495, 540
.   : milestone, 517,
iast_HARDCODED_SECRET_DISABLED (479.661 µs) : 459, 501
.   : milestone, 480,
iast_INACTIVE (455.801 µs) : 434, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (474.365 µs) : 453, 495
.   : milestone, 474,
tracing (444.825 µs) : 424, 465
.   : milestone, 445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.98 µs [352.794 µs, 393.165 µs] -
iast 486.674 µs [465.33 µs, 508.017 µs] 113.694 µs (30.5%)
iast_FULL 548.295 µs [527.074 µs, 569.516 µs] 175.316 µs (47.0%)
iast_GLOBAL 505.455 µs [483.529 µs, 527.382 µs] 132.476 µs (35.5%)
iast_HARDCODED_SECRET_DISABLED 486.036 µs [464.489 µs, 507.584 µs] 113.057 µs (30.3%)
iast_INACTIVE 448.09 µs [427.364 µs, 468.816 µs] 75.11 µs (20.1%)
iast_TELEMETRY_OFF 469.345 µs [448.356 µs, 490.334 µs] 96.366 µs (25.8%)
tracing 444.753 µs [423.452 µs, 466.054 µs] 71.774 µs (19.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.71 µs [348.477 µs, 386.944 µs] -
iast 476.159 µs [455.16 µs, 497.157 µs] 108.448 µs (29.5%)
iast_FULL 547.649 µs [526.643 µs, 568.654 µs] 179.938 µs (48.9%)
iast_GLOBAL 517.052 µs [494.587 µs, 539.516 µs] 149.341 µs (40.6%)
iast_HARDCODED_SECRET_DISABLED 479.661 µs [458.713 µs, 500.609 µs] 111.95 µs (30.4%)
iast_INACTIVE 455.801 µs [434.347 µs, 477.255 µs] 88.09 µs (24.0%)
iast_TELEMETRY_OFF 474.365 µs [453.327 µs, 495.404 µs] 106.655 µs (29.0%)
tracing 444.825 µs [424.273 µs, 465.378 µs] 77.115 µs (21.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~bb0f2cc81c, baseline=1.38.0-SNAPSHOT~9184826036
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.356 ms) : 1336, 1376
.   : milestone, 1356,
appsec (1.711 ms) : 1687, 1735
.   : milestone, 1711,
appsec_no_iast (1.71 ms) : 1685, 1736
.   : milestone, 1710,
iast (1.465 ms) : 1443, 1488
.   : milestone, 1465,
profiling (1.528 ms) : 1503, 1554
.   : milestone, 1528,
tracing (1.456 ms) : 1432, 1481
.   : milestone, 1456,
section candidate
no_agent (1.348 ms) : 1328, 1368
.   : milestone, 1348,
appsec (1.731 ms) : 1707, 1754
.   : milestone, 1731,
appsec_no_iast (1.712 ms) : 1687, 1737
.   : milestone, 1712,
iast (1.486 ms) : 1463, 1508
.   : milestone, 1486,
profiling (1.492 ms) : 1468, 1517
.   : milestone, 1492,
tracing (1.477 ms) : 1453, 1501
.   : milestone, 1477,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.356 ms [1.336 ms, 1.376 ms] -
appsec 1.711 ms [1.687 ms, 1.735 ms] 354.883 µs (26.2%)
appsec_no_iast 1.71 ms [1.685 ms, 1.736 ms] 354.322 µs (26.1%)
iast 1.465 ms [1.443 ms, 1.488 ms] 109.195 µs (8.1%)
profiling 1.528 ms [1.503 ms, 1.554 ms] 172.314 µs (12.7%)
tracing 1.456 ms [1.432 ms, 1.481 ms] 100.319 µs (7.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.328 ms, 1.368 ms] -
appsec 1.731 ms [1.707 ms, 1.754 ms] 382.895 µs (28.4%)
appsec_no_iast 1.712 ms [1.687 ms, 1.737 ms] 364.314 µs (27.0%)
iast 1.486 ms [1.463 ms, 1.508 ms] 137.774 µs (10.2%)
profiling 1.492 ms [1.468 ms, 1.517 ms] 144.739 µs (10.7%)
tracing 1.477 ms [1.453 ms, 1.501 ms] 129.202 µs (9.6%)

Dacapo

@smola smola merged commit 2e11acc into master Jul 16, 2024
82 checks passed
@smola smola deleted the smola/dd-static-rules-refine branch July 16, 2024 12:12
@github-actions github-actions bot added this to the 1.38.0 milestone Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.38.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@smola @PerfectSlayer