Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor of the PropagationModule and removal of WebModule #6033

Merged
merged 1 commit into from
Oct 27, 2023
Merged

Refactor of the PropagationModule and removal of WebModule #6033

merged 1 commit into from
Oct 27, 2023

Conversation

manuel-alvarez-alvarez
Copy link
Contributor

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Oct 11, 2023 •
edited
Loading

What Does This Do

Refactoring of the tainting API by moving all tainting operations to the PropagationModule with methods according to:

  1. taint methods are used to taint values : taint(context?, target, source_attributes+)
  2. taintIfTainted methods are used to taint values only if the input is tainted : taintIfTainted(context?, target, input, source_attributes+)
  3. taintIfAnyTainted methods are used to taint values only if any of the inputs is tainted : taintIfAnyTainted(context?, target, input[], source_attributes+)

Motivation

Both modules WebModule and PropagationModule have grown organically bringing a lot of duplication, this PR tries to merge all tainting operations providing a cohesive API.

Additional Notes

This PR is quite big but it does not introduce any new functionality, just a reorganization of the tainting API.

Jira ticket: APPSEC-10454

@manuel-alvarez-alvarez manuel-alvarez-alvarez added type: refactoring comp: asm iast Application Security Management (IAST) labels Oct 11, 2023
@pr-commenter
Copy link

pr-commenter bot commented Oct 11, 2023 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~a012761e16 1.23.0-SNAPSHOT~072e68a521
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.032 s) : 0, 1031748
Total [baseline] (8.759 s) : 0, 8759255
Agent [candidate] (1.027 s) : 0, 1026874
Total [candidate] (8.764 s) : 0, 8763927
section iast
Agent [baseline] (1.171 s) : 0, 1171310
Total [baseline] (9.349 s) : 0, 9348814
Agent [candidate] (1.155 s) : 0, 1154731
Total [candidate] (9.271 s) : 0, 9270759
section iast_TELEMETRY_OFF
Agent [baseline] (1.143 s) : 0, 1143493
Total [baseline] (9.312 s) : 0, 9312482
Agent [candidate] (1.147 s) : 0, 1147284
Total [candidate] (9.246 s) : 0, 9245937
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.032 s -
Agent iast 1.171 s 139.562 ms (13.5%)
Agent iast_TELEMETRY_OFF 1.143 s 111.745 ms (10.8%)
Total tracing 8.759 s -
Total iast 9.349 s 589.559 ms (6.7%)
Total iast_TELEMETRY_OFF 9.312 s 553.227 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.027 s -
Agent iast 1.155 s 127.857 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.147 s 120.41 ms (11.7%)
Total tracing 8.764 s -
Total iast 9.271 s 506.832 ms (5.8%)
Total iast_TELEMETRY_OFF 9.246 s 482.01 ms (5.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.19 ms) : 0, 645190
BytebuddyAgent [candidate] (642.159 ms) : 0, 642159
GlobalTracer [baseline] (295.556 ms) : 0, 295556
GlobalTracer [candidate] (294.505 ms) : 0, 294505
AppSec [baseline] (49.647 ms) : 0, 49647
AppSec [candidate] (48.964 ms) : 0, 48964
Remote Config [baseline] (683.009 µs) : 0, 683
Remote Config [candidate] (677.767 µs) : 0, 678
Telemetry [baseline] (6.146 ms) : 0, 6146
Telemetry [candidate] (6.058 ms) : 0, 6058
section iast
BytebuddyAgent [baseline] (785.25 ms) : 0, 785250
BytebuddyAgent [candidate] (773.117 ms) : 0, 773117
GlobalTracer [baseline] (277.418 ms) : 0, 277418
GlobalTracer [candidate] (274.07 ms) : 0, 274070
AppSec [baseline] (48.892 ms) : 0, 48892
AppSec [candidate] (46.311 ms) : 0, 46311
Remote Config [baseline] (1.247 ms) : 0, 1247
Remote Config [candidate] (578.151 µs) : 0, 578
Telemetry [baseline] (8.238 ms) : 0, 8238
Telemetry [candidate] (11.272 ms) : 0, 11272
IAST [baseline] (15.406 ms) : 0, 15406
IAST [candidate] (14.954 ms) : 0, 14954
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (763.725 ms) : 0, 763725
BytebuddyAgent [candidate] (766.779 ms) : 0, 766779
GlobalTracer [baseline] (273.667 ms) : 0, 273667
GlobalTracer [candidate] (274.258 ms) : 0, 274258
AppSec [baseline] (46.74 ms) : 0, 46740
AppSec [candidate] (46.287 ms) : 0, 46287
Remote Config [baseline] (582.785 µs) : 0, 583
Remote Config [candidate] (607.005 µs) : 0, 607
Telemetry [baseline] (6.796 ms) : 0, 6796
Telemetry [candidate] (9.01 ms) : 0, 9010
IAST [baseline] (17.649 ms) : 0, 17649
IAST [candidate] (15.876 ms) : 0, 15876
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1027447
Total [baseline] (9.348 s) : 0, 9347892
Agent [candidate] (1.033 s) : 0, 1032660
Total [candidate] (9.326 s) : 0, 9326114
section appsec
Agent [baseline] (1.121 s) : 0, 1121300
Total [baseline] (9.389 s) : 0, 9388998
Agent [candidate] (1.114 s) : 0, 1114184
Total [candidate] (9.355 s) : 0, 9354766
section iast
Agent [baseline] (1.159 s) : 0, 1158952
Total [baseline] (9.517 s) : 0, 9516657
Agent [candidate] (1.152 s) : 0, 1151679
Total [candidate] (9.511 s) : 0, 9510660
section profiling
Agent [baseline] (1.209 s) : 0, 1208580
Total [baseline] (9.589 s) : 0, 9588982
Agent [candidate] (1.217 s) : 0, 1216876
Total [candidate] (9.576 s) : 0, 9575790
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.027 s -
Agent appsec 1.121 s 93.854 ms (9.1%)
Agent iast 1.159 s 131.505 ms (12.8%)
Agent profiling 1.209 s 181.133 ms (17.6%)
Total tracing 9.348 s -
Total appsec 9.389 s 41.107 ms (0.4%)
Total iast 9.517 s 168.765 ms (1.8%)
Total profiling 9.589 s 241.09 ms (2.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.033 s -
Agent appsec 1.114 s 81.524 ms (7.9%)
Agent iast 1.152 s 119.019 ms (11.5%)
Agent profiling 1.217 s 184.215 ms (17.8%)
Total tracing 9.326 s -
Total appsec 9.355 s 28.652 ms (0.3%)
Total iast 9.511 s 184.546 ms (2.0%)
Total profiling 9.576 s 249.676 ms (2.7%) 
gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (642.296 ms) : 0, 642296
BytebuddyAgent [candidate] (646.206 ms) : 0, 646206
GlobalTracer [baseline] (294.646 ms) : 0, 294646
GlobalTracer [candidate] (295.76 ms) : 0, 295760
AppSec [baseline] (49.208 ms) : 0, 49208
AppSec [candidate] (49.209 ms) : 0, 49209
Remote Config [baseline] (680.815 µs) : 0, 681
Remote Config [candidate] (697.338 µs) : 0, 697
Telemetry [baseline] (6.105 ms) : 0, 6105
Telemetry [candidate] (6.161 ms) : 0, 6161
section appsec
BytebuddyAgent [baseline] (645.984 ms) : 0, 645984
BytebuddyAgent [candidate] (642.272 ms) : 0, 642272
GlobalTracer [baseline] (295.352 ms) : 0, 295352
GlobalTracer [candidate] (293.272 ms) : 0, 293272
AppSec [baseline] (139.032 ms) : 0, 139032
AppSec [candidate] (138.026 ms) : 0, 138026
Remote Config [baseline] (643.199 µs) : 0, 643
Remote Config [candidate] (638.492 µs) : 0, 638
Telemetry [baseline] (5.751 ms) : 0, 5751
Telemetry [candidate] (5.704 ms) : 0, 5704
section iast
BytebuddyAgent [baseline] (777.439 ms) : 0, 777439
BytebuddyAgent [candidate] (771.787 ms) : 0, 771787
GlobalTracer [baseline] (275.153 ms) : 0, 275153
GlobalTracer [candidate] (273.491 ms) : 0, 273491
AppSec [baseline] (46.694 ms) : 0, 46694
AppSec [candidate] (47.134 ms) : 0, 47134
Remote Config [baseline] (1.244 ms) : 0, 1244
Remote Config [candidate] (579.982 µs) : 0, 580
Telemetry [baseline] (6.946 ms) : 0, 6946
Telemetry [candidate] (7.524 ms) : 0, 7524
IAST [baseline] (16.794 ms) : 0, 16794
IAST [candidate] (16.831 ms) : 0, 16831
section profiling
BytebuddyAgent [baseline] (657.163 ms) : 0, 657163
BytebuddyAgent [candidate] (662.71 ms) : 0, 662710
GlobalTracer [baseline] (359.635 ms) : 0, 359635
GlobalTracer [candidate] (362.227 ms) : 0, 362227
AppSec [baseline] (50.321 ms) : 0, 50321
AppSec [candidate] (49.8 ms) : 0, 49800
Remote Config [baseline] (647.036 µs) : 0, 647
Remote Config [candidate] (659.882 µs) : 0, 660
Telemetry [baseline] (6.042 ms) : 0, 6042
Telemetry [candidate] (6.168 ms) : 0, 6168
ProfilingAgent [baseline] (81.015 ms) : 0, 81015
ProfilingAgent [candidate] (81.204 ms) : 0, 81204
Profiling [baseline] (81.041 ms) : 0, 81041
Profiling [candidate] (81.23 ms) : 0, 81230
Loading

Load

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~a012761e16 1.23.0-SNAPSHOT~072e68a521
config baseline candidate
end_time 2023-10-27T09:17:42 2023-10-27T09:34:04
start_time 2023-10-27T09:17:29 2023-10-27T09:33:51
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.147 µs) : 339, 381
.   : milestone, 360,
iast (471.176 µs) : 450, 492
.   : milestone, 471,
iast_FULL (528.003 µs) : 507, 549
.   : milestone, 528,
iast_INACTIVE (430.327 µs) : 409, 451
.   : milestone, 430,
iast_TELEMETRY_OFF (461.486 µs) : 440, 483
.   : milestone, 461,
tracing (430.152 µs) : 409, 451
.   : milestone, 430,
section candidate
no_agent (363.093 µs) : 342, 384
.   : milestone, 363,
iast (461.19 µs) : 441, 482
.   : milestone, 461,
iast_FULL (526.994 µs) : 506, 548
.   : milestone, 527,
iast_INACTIVE (431.855 µs) : 410, 454
.   : milestone, 432,
iast_TELEMETRY_OFF (463.163 µs) : 442, 485
.   : milestone, 463,
tracing (431.538 µs) : 411, 452
.   : milestone, 432,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.147 µs [339.297 µs, 380.996 µs] -
iast 471.176 µs [449.963 µs, 492.388 µs] 111.029 µs (30.8%)
iast_FULL 528.003 µs [507.274 µs, 548.732 µs] 167.857 µs (46.6%)
iast_INACTIVE 430.327 µs [409.372 µs, 451.282 µs] 70.18 µs (19.5%)
iast_TELEMETRY_OFF 461.486 µs [440.386 µs, 482.587 µs] 101.34 µs (28.1%)
tracing 430.152 µs [408.982 µs, 451.322 µs] 70.006 µs (19.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.093 µs [341.895 µs, 384.292 µs] -
iast 461.19 µs [440.545 µs, 481.835 µs] 98.097 µs (27.0%)
iast_FULL 526.994 µs [506.101 µs, 547.886 µs] 163.9 µs (45.1%)
iast_INACTIVE 431.855 µs [410.184 µs, 453.526 µs] 68.762 µs (18.9%)
iast_TELEMETRY_OFF 463.163 µs [441.569 µs, 484.758 µs] 100.07 µs (27.6%)
tracing 431.538 µs [410.945 µs, 452.131 µs] 68.445 µs (18.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.348 ms) : 1329, 1367
.   : milestone, 1348,
appsec (1.713 ms) : 1689, 1738
.   : milestone, 1713,
iast (1.448 ms) : 1424, 1472
.   : milestone, 1448,
profiling (1.451 ms) : 1426, 1475
.   : milestone, 1451,
tracing (1.461 ms) : 1436, 1486
.   : milestone, 1461,
section candidate
no_agent (1.369 ms) : 1349, 1388
.   : milestone, 1369,
appsec (1.685 ms) : 1661, 1710
.   : milestone, 1685,
iast (1.457 ms) : 1433, 1481
.   : milestone, 1457,
profiling (1.497 ms) : 1472, 1523
.   : milestone, 1497,
tracing (1.457 ms) : 1432, 1482
.   : milestone, 1457,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.329 ms, 1.367 ms] -
appsec 1.713 ms [1.689 ms, 1.738 ms] 365.532 µs (27.1%)
iast 1.448 ms [1.424 ms, 1.472 ms] 100.156 µs (7.4%)
profiling 1.451 ms [1.426 ms, 1.475 ms] 102.938 µs (7.6%)
tracing 1.461 ms [1.436 ms, 1.486 ms] 113.483 µs (8.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.369 ms [1.349 ms, 1.388 ms] -
appsec 1.685 ms [1.661 ms, 1.71 ms] 316.519 µs (23.1%)
iast 1.457 ms [1.433 ms, 1.481 ms] 88.442 µs (6.5%)
profiling 1.497 ms [1.472 ms, 1.523 ms] 128.675 µs (9.4%)
tracing 1.457 ms [1.432 ms, 1.482 ms] 88.262 µs (6.4%)

@manuel-alvarez-alvarez manuel-alvarez-alvarez mentioned this pull request Oct 16, 2023
Closed
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 3 times, most recently from 0efacb4 to 995f5c4 Compare October 22, 2023 20:27
@manuel-alvarez-alvarez manuel-alvarez-alvarez marked this pull request as ready for review October 22, 2023 21:23
@manuel-alvarez-alvarez manuel-alvarez-alvarez added run-tests: all Run all tests tag: no release notes Changes to exclude from release notes labels Oct 23, 2023
@manuel-alvarez-alvarez manuel-alvarez-alvarez mentioned this pull request Oct 24, 2023
Closed
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 4 times, most recently from 2116662 to 469769d Compare October 25, 2023 15:24
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 2 times, most recently from 547364a to 3a6030d Compare October 27, 2023 08:32
@manuel-alvarez-alvarez manuel-alvarez-alvarez merged commit 00a81bb into master Oct 27, 2023
119 checks passed
@manuel-alvarez-alvarez manuel-alvarez-alvarez deleted the malvarez/iast-taint-api-refactor branch October 27, 2023 10:00
@github-actions github-actions bot added this to the 1.23.0 milestone Oct 27, 2023
jandro996 pushed a commit that referenced this pull request Oct 27, 2023
@manuel-alvarez-alvarez @jandro996
Refactor of the PropagationModule and removal of WebModule (#6033)
b68b8f0
jandro996 pushed a commit that referenced this pull request Oct 30, 2023
@manuel-alvarez-alvarez @jandro996
Refactor of the PropagationModule and removal of WebModule (#6033)
ae66378
@manuel-alvarez-alvarez manuel-alvarez-alvarez mentioned this pull request Oct 30, 2023
Merged
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@jandro996 jandro996 jandro996 approved these changes

@DDJavierSantos DDJavierSantos DDJavierSantos approved these changes

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees

@manuel-alvarez-alvarez manuel-alvarez-alvarez

Labels
comp: asm iast Application Security Management (IAST) run-tests: all Run all tests tag: no release notes Changes to exclude from release notes type: refactoring
Projects
None yet
Milestone
1.23.0
Development

Successfully merging this pull request may close these issues.
4 participants
@manuel-alvarez-alvarez @smola @jandro996 @DDJavierSantos