Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add NullAway to IAST module and fix errors #6106

Merged
merged 1 commit into from
Nov 2, 2023
Merged

Add NullAway to IAST module and fix errors #6106

merged 1 commit into from
Nov 2, 2023

Conversation

manuel-alvarez-alvarez
Copy link
Contributor

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Oct 26, 2023 •
edited by smola
Loading

What Does This Do

  • Add NullAway as build time tool to prevent NPE in IAST code
  • Force compilation with JDK11, targeting Java 8.

Motivation

Additional Notes

Jira ticket: APPSEC-11860

@manuel-alvarez-alvarez manuel-alvarez-alvarez added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling comp: asm iast Application Security Management (IAST) labels Oct 26, 2023
@pr-commenter
Copy link

pr-commenter bot commented Oct 26, 2023 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~047c47ca69 1.23.0-SNAPSHOT~07ca0e595a
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042973
Total [baseline] (8.781 s) : 0, 8781123
Agent [candidate] (1.036 s) : 0, 1035544
Total [candidate] (8.79 s) : 0, 8789523
section iast
Agent [baseline] (1.149 s) : 0, 1149404
Total [baseline] (9.358 s) : 0, 9357537
Agent [candidate] (1.151 s) : 0, 1150621
Total [candidate] (9.321 s) : 0, 9320750
section iast_TELEMETRY_OFF
Agent [baseline] (1.152 s) : 0, 1151633
Total [baseline] (9.279 s) : 0, 9279233
Agent [candidate] (1.154 s) : 0, 1153737
Total [candidate] (9.351 s) : 0, 9350716
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent iast 1.149 s 106.431 ms (10.2%)
Agent iast_TELEMETRY_OFF 1.152 s 108.66 ms (10.4%)
Total tracing 8.781 s -
Total iast 9.358 s 576.413 ms (6.6%)
Total iast_TELEMETRY_OFF 9.279 s 498.11 ms (5.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent iast 1.151 s 115.077 ms (11.1%)
Agent iast_TELEMETRY_OFF 1.154 s 118.193 ms (11.4%)
Total tracing 8.79 s -
Total iast 9.321 s 531.227 ms (6.0%)
Total iast_TELEMETRY_OFF 9.351 s 561.193 ms (6.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (650.5 ms) : 0, 650500
BytebuddyAgent [candidate] (646.031 ms) : 0, 646031
GlobalTracer [baseline] (296.379 ms) : 0, 296379
GlobalTracer [candidate] (294.349 ms) : 0, 294349
AppSec [baseline] (49.289 ms) : 0, 49289
AppSec [candidate] (48.777 ms) : 0, 48777
Remote Config [baseline] (709.841 µs) : 0, 710
Remote Config [candidate] (700.168 µs) : 0, 700
Telemetry [baseline] (11.383 ms) : 0, 11383
Telemetry [candidate] (11.171 ms) : 0, 11171
section iast
BytebuddyAgent [baseline] (765.11 ms) : 0, 765110
BytebuddyAgent [candidate] (764.932 ms) : 0, 764932
GlobalTracer [baseline] (273.741 ms) : 0, 273741
GlobalTracer [candidate] (273.754 ms) : 0, 273754
AppSec [baseline] (46.788 ms) : 0, 46788
AppSec [candidate] (46.747 ms) : 0, 46747
IAST [baseline] (17.266 ms) : 0, 17266
IAST [candidate] (19.229 ms) : 0, 19229
Remote Config [baseline] (573.161 µs) : 0, 573
Remote Config [candidate] (567.181 µs) : 0, 567
Telemetry [baseline] (11.365 ms) : 0, 11365
Telemetry [candidate] (11.074 ms) : 0, 11074
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (763.991 ms) : 0, 763991
BytebuddyAgent [candidate] (765.551 ms) : 0, 765551
GlobalTracer [baseline] (274.727 ms) : 0, 274727
GlobalTracer [candidate] (276.677 ms) : 0, 276677
AppSec [baseline] (46.695 ms) : 0, 46695
AppSec [candidate] (47.102 ms) : 0, 47102
IAST [baseline] (18.265 ms) : 0, 18265
IAST [candidate] (17.926 ms) : 0, 17926
Remote Config [baseline] (549.912 µs) : 0, 550
Remote Config [candidate] (569.572 µs) : 0, 570
Telemetry [baseline] (12.608 ms) : 0, 12608
Telemetry [candidate] (11.266 ms) : 0, 11266
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.051 s) : 0, 1051238
Total [baseline] (9.368 s) : 0, 9367934
Agent [candidate] (1.046 s) : 0, 1046098
Total [candidate] (9.323 s) : 0, 9323487
section appsec
Agent [baseline] (1.122 s) : 0, 1121872
Total [baseline] (9.389 s) : 0, 9389143
Agent [candidate] (1.123 s) : 0, 1122537
Total [candidate] (9.472 s) : 0, 9472111
section iast
Agent [baseline] (1.149 s) : 0, 1149151
Total [baseline] (9.475 s) : 0, 9475029
Agent [candidate] (1.153 s) : 0, 1152721
Total [candidate] (9.555 s) : 0, 9555047
section profiling
Agent [baseline] (1.222 s) : 0, 1222055
Total [baseline] (9.571 s) : 0, 9570906
Agent [candidate] (1.215 s) : 0, 1214698
Total [candidate] (9.527 s) : 0, 9526925
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.051 s -
Agent appsec 1.122 s 70.634 ms (6.7%)
Agent iast 1.149 s 97.912 ms (9.3%)
Agent profiling 1.222 s 170.816 ms (16.2%)
Total tracing 9.368 s -
Total appsec 9.389 s 21.209 ms (0.2%)
Total iast 9.475 s 107.095 ms (1.1%)
Total profiling 9.571 s 202.972 ms (2.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent appsec 1.123 s 76.439 ms (7.3%)
Agent iast 1.153 s 106.622 ms (10.2%)
Agent profiling 1.215 s 168.599 ms (16.1%)
Total tracing 9.323 s -
Total appsec 9.472 s 148.624 ms (1.6%)
Total iast 9.555 s 231.56 ms (2.5%)
Total profiling 9.527 s 203.437 ms (2.2%) 
gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (655.294 ms) : 0, 655294
BytebuddyAgent [candidate] (652.644 ms) : 0, 652644
GlobalTracer [baseline] (298.603 ms) : 0, 298603
GlobalTracer [candidate] (297.207 ms) : 0, 297207
AppSec [baseline] (49.996 ms) : 0, 49996
AppSec [candidate] (49.317 ms) : 0, 49317
Remote Config [baseline] (713.839 µs) : 0, 714
Remote Config [candidate] (705.948 µs) : 0, 706
Telemetry [baseline] (11.582 ms) : 0, 11582
Telemetry [candidate] (11.413 ms) : 0, 11413
section appsec
BytebuddyAgent [baseline] (646.972 ms) : 0, 646972
BytebuddyAgent [candidate] (646.698 ms) : 0, 646698
GlobalTracer [baseline] (294.776 ms) : 0, 294776
GlobalTracer [candidate] (295.102 ms) : 0, 295102
AppSec [baseline] (138.103 ms) : 0, 138103
AppSec [candidate] (138.705 ms) : 0, 138705
Remote Config [baseline] (646.203 µs) : 0, 646
Remote Config [candidate] (649.257 µs) : 0, 649
Telemetry [baseline] (6.889 ms) : 0, 6889
Telemetry [candidate] (6.911 ms) : 0, 6911
section iast
BytebuddyAgent [baseline] (765.109 ms) : 0, 765109
BytebuddyAgent [candidate] (766.795 ms) : 0, 766795
GlobalTracer [baseline] (273.78 ms) : 0, 273780
GlobalTracer [candidate] (274.822 ms) : 0, 274822
AppSec [baseline] (46.761 ms) : 0, 46761
AppSec [candidate] (46.753 ms) : 0, 46753
Remote Config [baseline] (564.278 µs) : 0, 564
Remote Config [candidate] (566.616 µs) : 0, 567
Telemetry [baseline] (11.285 ms) : 0, 11285
Telemetry [candidate] (11.841 ms) : 0, 11841
IAST [baseline] (17.131 ms) : 0, 17131
IAST [candidate] (17.371 ms) : 0, 17371
section profiling
ProfilingAgent [baseline] (81.816 ms) : 0, 81816
ProfilingAgent [candidate] (81.487 ms) : 0, 81487
BytebuddyAgent [baseline] (661.19 ms) : 0, 661190
BytebuddyAgent [candidate] (657.968 ms) : 0, 657968
GlobalTracer [baseline] (362.146 ms) : 0, 362146
GlobalTracer [candidate] (359.613 ms) : 0, 359613
AppSec [baseline] (50.147 ms) : 0, 50147
AppSec [candidate] (49.232 ms) : 0, 49232
Remote Config [baseline] (646.27 µs) : 0, 646
Remote Config [candidate] (663.015 µs) : 0, 663
Telemetry [baseline] (11.412 ms) : 0, 11412
Telemetry [candidate] (11.425 ms) : 0, 11425
Profiling [baseline] (81.84 ms) : 0, 81840
Profiling [candidate] (81.512 ms) : 0, 81512
Loading

Load

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~047c47ca69 1.23.0-SNAPSHOT~07ca0e595a
config baseline candidate
end_time 2023-11-02T13:17:16 2023-11-02T13:33:44
start_time 2023-11-02T13:17:03 2023-11-02T13:33:31
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69
    dateFormat X
    axisFormat %s
section baseline
no_agent (361.823 µs) : 342, 382
.   : milestone, 362,
iast (455.312 µs) : 434, 476
.   : milestone, 455,
iast_FULL (524.125 µs) : 504, 545
.   : milestone, 524,
iast_INACTIVE (430.62 µs) : 410, 452
.   : milestone, 431,
iast_TELEMETRY_OFF (459.251 µs) : 438, 481
.   : milestone, 459,
tracing (433.577 µs) : 412, 455
.   : milestone, 434,
section candidate
no_agent (358.763 µs) : 339, 379
.   : milestone, 359,
iast (459.897 µs) : 439, 481
.   : milestone, 460,
iast_FULL (521.807 µs) : 501, 542
.   : milestone, 522,
iast_INACTIVE (433.802 µs) : 413, 455
.   : milestone, 434,
iast_TELEMETRY_OFF (462.026 µs) : 441, 483
.   : milestone, 462,
tracing (432.261 µs) : 411, 453
.   : milestone, 432,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.823 µs [341.977 µs, 381.669 µs] -
iast 455.312 µs [434.46 µs, 476.164 µs] 93.489 µs (25.8%)
iast_FULL 524.125 µs [503.572 µs, 544.677 µs] 162.302 µs (44.9%)
iast_INACTIVE 430.62 µs [409.514 µs, 451.726 µs] 68.797 µs (19.0%)
iast_TELEMETRY_OFF 459.251 µs [437.527 µs, 480.976 µs] 97.428 µs (26.9%)
tracing 433.577 µs [412.121 µs, 455.033 µs] 71.754 µs (19.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 358.763 µs [338.565 µs, 378.962 µs] -
iast 459.897 µs [439.018 µs, 480.775 µs] 101.133 µs (28.2%)
iast_FULL 521.807 µs [501.386 µs, 542.229 µs] 163.044 µs (45.4%)
iast_INACTIVE 433.802 µs [413.087 µs, 454.517 µs] 75.038 µs (20.9%)
iast_TELEMETRY_OFF 462.026 µs [440.94 µs, 483.112 µs] 103.263 µs (28.8%)
tracing 432.261 µs [411.169 µs, 453.353 µs] 73.498 µs (20.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.359 ms) : 1339, 1379
.   : milestone, 1359,
appsec (1.68 ms) : 1655, 1705
.   : milestone, 1680,
iast (1.456 ms) : 1433, 1480
.   : milestone, 1456,
profiling (1.48 ms) : 1455, 1505
.   : milestone, 1480,
tracing (1.459 ms) : 1435, 1483
.   : milestone, 1459,
section candidate
no_agent (1.342 ms) : 1322, 1361
.   : milestone, 1342,
appsec (1.692 ms) : 1667, 1717
.   : milestone, 1692,
iast (1.469 ms) : 1445, 1493
.   : milestone, 1469,
profiling (1.487 ms) : 1462, 1513
.   : milestone, 1487,
tracing (1.455 ms) : 1431, 1480
.   : milestone, 1455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.359 ms [1.339 ms, 1.379 ms] -
appsec 1.68 ms [1.655 ms, 1.705 ms] 320.753 µs (23.6%)
iast 1.456 ms [1.433 ms, 1.48 ms] 97.593 µs (7.2%)
profiling 1.48 ms [1.455 ms, 1.505 ms] 121.301 µs (8.9%)
tracing 1.459 ms [1.435 ms, 1.483 ms] 100.254 µs (7.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.322 ms, 1.361 ms] -
appsec 1.692 ms [1.667 ms, 1.717 ms] 350.226 µs (26.1%)
iast 1.469 ms [1.445 ms, 1.493 ms] 127.604 µs (9.5%)
profiling 1.487 ms [1.462 ms, 1.513 ms] 145.449 µs (10.8%)
tracing 1.455 ms [1.431 ms, 1.48 ms] 113.517 µs (8.5%)

@manuel-alvarez-alvarez manuel-alvarez-alvarez marked this pull request as ready for review October 30, 2023 09:26
@manuel-alvarez-alvarez manuel-alvarez-alvarez requested a review from a team as a code owner October 30, 2023 09:26
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-enable-nullaway branch 3 times, most recently from c5e0d75 to 12aec57 Compare November 1, 2023 10:17
@manuel-alvarez-alvarez manuel-alvarez-alvarez merged commit ee3049b into master Nov 2, 2023
67 of 69 checks passed
@manuel-alvarez-alvarez manuel-alvarez-alvarez deleted the malvarez/iast-enable-nullaway branch November 2, 2023 14:01
@github-actions github-actions bot added this to the 1.23.0 milestone Nov 2, 2023
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@anderruiz anderruiz Awaiting requested review from anderruiz anderruiz is a code owner automatically assigned from DataDog/asm-java

Assignees
No one assigned
Labels
comp: asm iast Application Security Management (IAST) comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.23.0
Development

Successfully merging this pull request may close these issues.
2 participants
@manuel-alvarez-alvarez @smola