-
Notifications
You must be signed in to change notification settings - Fork 288
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement debug context propagation to enable live debugging of java applications #7286
Conversation
...ugger/debugger-bootstrap/src/main/java/datadog/trace/bootstrap/debugger/DebuggerContext.java
Outdated
Show resolved
Hide resolved
BenchmarksStartupParameters
See matching parameters
SummaryFound 2 performance improvements and 3 performance regressions! Performance is the same for 49 metrics, 9 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1046099
Total [baseline] (10.314 s) : 0, 10313630
Agent [candidate] (1.049 s) : 0, 1049483
Total [candidate] (10.348 s) : 0, 10347632
section appsec
Agent [baseline] (1.168 s) : 0, 1167860
Total [baseline] (10.552 s) : 0, 10551528
Agent [candidate] (1.161 s) : 0, 1161174
Total [candidate] (10.488 s) : 0, 10487787
section iast
Agent [baseline] (1.173 s) : 0, 1173250
Total [baseline] (10.782 s) : 0, 10782351
Agent [candidate] (1.173 s) : 0, 1172564
Total [candidate] (10.77 s) : 0, 10769701
section profiling
Agent [baseline] (1.244 s) : 0, 1243684
Total [baseline] (10.567 s) : 0, 10567013
Agent [candidate] (1.24 s) : 0, 1239933
Total [candidate] (10.622 s) : 0, 10622215
gantt
title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.85 ms) : 0, 668850
BytebuddyAgent [candidate] (670.884 ms) : 0, 670884
GlobalTracer [baseline] (305.443 ms) : 0, 305443
GlobalTracer [candidate] (306.571 ms) : 0, 306571
AppSec [baseline] (50.379 ms) : 0, 50379
AppSec [candidate] (50.198 ms) : 0, 50198
Remote Config [baseline] (674.768 µs) : 0, 675
Remote Config [candidate] (677.316 µs) : 0, 677
Telemetry [baseline] (7.287 ms) : 0, 7287
Telemetry [candidate] (7.662 ms) : 0, 7662
section appsec
BytebuddyAgent [baseline] (680.437 ms) : 0, 680437
BytebuddyAgent [candidate] (676.298 ms) : 0, 676298
GlobalTracer [baseline] (299.448 ms) : 0, 299448
GlobalTracer [candidate] (298.742 ms) : 0, 298742
AppSec [baseline] (155.371 ms) : 0, 155371
AppSec [candidate] (153.628 ms) : 0, 153628
Remote Config [baseline] (603.56 µs) : 0, 604
Remote Config [candidate] (626.926 µs) : 0, 627
Telemetry [baseline] (8.224 ms) : 0, 8224
Telemetry [candidate] (7.906 ms) : 0, 7906
IAST [baseline] (20.405 ms) : 0, 20405
IAST [candidate] (20.928 ms) : 0, 20928
section iast
BytebuddyAgent [baseline] (783.846 ms) : 0, 783846
BytebuddyAgent [candidate] (781.293 ms) : 0, 781293
GlobalTracer [baseline] (296.228 ms) : 0, 296228
GlobalTracer [candidate] (295.801 ms) : 0, 295801
AppSec [baseline] (48.267 ms) : 0, 48267
AppSec [candidate] (47.475 ms) : 0, 47475
Remote Config [baseline] (587.162 µs) : 0, 587
Remote Config [candidate] (591.926 µs) : 0, 592
Telemetry [baseline] (7.87 ms) : 0, 7870
Telemetry [candidate] (7.102 ms) : 0, 7102
IAST [baseline] (22.9 ms) : 0, 22900
IAST [candidate] (26.821 ms) : 0, 26821
section profiling
ProfilingAgent [baseline] (95.118 ms) : 0, 95118
ProfilingAgent [candidate] (94.602 ms) : 0, 94602
BytebuddyAgent [baseline] (663.777 ms) : 0, 663777
BytebuddyAgent [candidate] (661.464 ms) : 0, 661464
GlobalTracer [baseline] (388.526 ms) : 0, 388526
GlobalTracer [candidate] (388.178 ms) : 0, 388178
AppSec [baseline] (51.277 ms) : 0, 51277
AppSec [candidate] (51.601 ms) : 0, 51601
Remote Config [baseline] (717.716 µs) : 0, 718
Remote Config [candidate] (648.497 µs) : 0, 648
Telemetry [baseline] (7.28 ms) : 0, 7280
Telemetry [candidate] (7.395 ms) : 0, 7395
Profiling [baseline] (95.142 ms) : 0, 95142
Profiling [candidate] (94.626 ms) : 0, 94626
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1044625
Total [baseline] (8.524 s) : 0, 8523617
Agent [candidate] (1.044 s) : 0, 1044083
Total [candidate] (8.488 s) : 0, 8488347
section iast
Agent [baseline] (1.191 s) : 0, 1191207
Total [baseline] (9.034 s) : 0, 9033947
Agent [candidate] (1.174 s) : 0, 1173972
Total [candidate] (8.987 s) : 0, 8986711
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.173 s) : 0, 1172774
Total [baseline] (8.963 s) : 0, 8962999
Agent [candidate] (1.171 s) : 0, 1170777
Total [candidate] (8.946 s) : 0, 8945814
section iast_TELEMETRY_OFF
Agent [baseline] (1.171 s) : 0, 1170919
Total [baseline] (8.997 s) : 0, 8996528
Agent [candidate] (1.18 s) : 0, 1179928
Total [candidate] (9.015 s) : 0, 9014604
gantt
title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (667.384 ms) : 0, 667384
BytebuddyAgent [candidate] (666.715 ms) : 0, 666715
GlobalTracer [baseline] (305.657 ms) : 0, 305657
GlobalTracer [candidate] (305.54 ms) : 0, 305540
AppSec [baseline] (50.176 ms) : 0, 50176
AppSec [candidate] (50.073 ms) : 0, 50073
Remote Config [baseline] (682.375 µs) : 0, 682
Remote Config [candidate] (674.027 µs) : 0, 674
Telemetry [baseline] (7.32 ms) : 0, 7320
Telemetry [candidate] (7.652 ms) : 0, 7652
section iast
BytebuddyAgent [baseline] (795.107 ms) : 0, 795107
BytebuddyAgent [candidate] (781.275 ms) : 0, 781275
GlobalTracer [baseline] (299.206 ms) : 0, 299206
GlobalTracer [candidate] (296.127 ms) : 0, 296127
AppSec [baseline] (53.295 ms) : 0, 53295
AppSec [candidate] (47.488 ms) : 0, 47488
IAST [baseline] (22.378 ms) : 0, 22378
IAST [candidate] (28.039 ms) : 0, 28039
Remote Config [baseline] (561.721 µs) : 0, 562
Remote Config [candidate] (591.967 µs) : 0, 592
Telemetry [baseline] (6.964 ms) : 0, 6964
Telemetry [candidate] (6.97 ms) : 0, 6970
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (780.878 ms) : 0, 780878
BytebuddyAgent [candidate] (779.766 ms) : 0, 779766
GlobalTracer [baseline] (295.549 ms) : 0, 295549
GlobalTracer [candidate] (295.277 ms) : 0, 295277
AppSec [baseline] (52.105 ms) : 0, 52105
AppSec [candidate] (48.09 ms) : 0, 48090
IAST [baseline] (21.676 ms) : 0, 21676
IAST [candidate] (26.548 ms) : 0, 26548
Remote Config [baseline] (563.051 µs) : 0, 563
Remote Config [candidate] (587.114 µs) : 0, 587
Telemetry [baseline] (8.525 ms) : 0, 8525
Telemetry [candidate] (6.982 ms) : 0, 6982
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (781.132 ms) : 0, 781132
BytebuddyAgent [candidate] (788.326 ms) : 0, 788326
GlobalTracer [baseline] (296.095 ms) : 0, 296095
GlobalTracer [candidate] (296.634 ms) : 0, 296634
AppSec [baseline] (46.973 ms) : 0, 46973
AppSec [candidate] (47.451 ms) : 0, 47451
IAST [baseline] (25.602 ms) : 0, 25602
IAST [candidate] (26.344 ms) : 0, 26344
Remote Config [baseline] (607.949 µs) : 0, 608
Remote Config [candidate] (573.237 µs) : 0, 573
Telemetry [baseline] (6.97 ms) : 0, 6970
Telemetry [candidate] (6.93 ms) : 0, 6930
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section baseline
no_agent (1.339 ms) : 1320, 1359
. : milestone, 1339,
appsec (1.725 ms) : 1701, 1748
. : milestone, 1725,
appsec_no_iast (1.721 ms) : 1696, 1746
. : milestone, 1721,
iast (1.473 ms) : 1450, 1496
. : milestone, 1473,
profiling (1.513 ms) : 1487, 1538
. : milestone, 1513,
tracing (1.461 ms) : 1436, 1486
. : milestone, 1461,
section candidate
no_agent (1.356 ms) : 1336, 1375
. : milestone, 1356,
appsec (1.728 ms) : 1704, 1751
. : milestone, 1728,
appsec_no_iast (1.701 ms) : 1677, 1726
. : milestone, 1701,
iast (1.489 ms) : 1467, 1511
. : milestone, 1489,
profiling (1.511 ms) : 1485, 1537
. : milestone, 1511,
tracing (1.471 ms) : 1447, 1495
. : milestone, 1471,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section baseline
no_agent (371.471 µs) : 352, 391
. : milestone, 371,
iast (493.571 µs) : 472, 515
. : milestone, 494,
iast_FULL (553.233 µs) : 532, 574
. : milestone, 553,
iast_GLOBAL (516.3 µs) : 494, 539
. : milestone, 516,
iast_HARDCODED_SECRET_DISABLED (484.967 µs) : 464, 506
. : milestone, 485,
iast_INACTIVE (460.514 µs) : 438, 483
. : milestone, 461,
iast_TELEMETRY_OFF (477.044 µs) : 455, 499
. : milestone, 477,
tracing (443.898 µs) : 423, 465
. : milestone, 444,
section candidate
no_agent (378.015 µs) : 358, 398
. : milestone, 378,
iast (485.065 µs) : 464, 506
. : milestone, 485,
iast_FULL (552.785 µs) : 532, 574
. : milestone, 553,
iast_GLOBAL (512.236 µs) : 491, 534
. : milestone, 512,
iast_HARDCODED_SECRET_DISABLED (481.81 µs) : 461, 503
. : milestone, 482,
iast_INACTIVE (454.916 µs) : 434, 476
. : milestone, 455,
iast_TELEMETRY_OFF (473.866 µs) : 453, 495
. : milestone, 474,
tracing (450.919 µs) : 430, 472
. : milestone, 451,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section baseline
no_agent (1.456 ms) : 1445, 1467
. : milestone, 1456,
appsec (2.202 ms) : 2168, 2236
. : milestone, 2202,
iast (1.96 ms) : 1919, 2002
. : milestone, 1960,
iast_GLOBAL (2.005 ms) : 1963, 2048
. : milestone, 2005,
profiling (1.861 ms) : 1826, 1897
. : milestone, 1861,
tracing (1.835 ms) : 1802, 1868
. : milestone, 1835,
section candidate
no_agent (1.455 ms) : 1443, 1466
. : milestone, 1455,
appsec (2.207 ms) : 2172, 2242
. : milestone, 2207,
iast (1.956 ms) : 1915, 1997
. : milestone, 1956,
iast_GLOBAL (2.0 ms) : 1959, 2041
. : milestone, 2000,
profiling (1.844 ms) : 1811, 1877
. : milestone, 1844,
tracing (1.837 ms) : 1804, 1870
. : milestone, 1837,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~c126dd86e3, baseline=1.38.0-SNAPSHOT~1d41c00244
dateFormat X
axisFormat %s
section baseline
no_agent (15.001 s) : 15001000, 15001000
. : milestone, 15001000,
appsec (15.222 s) : 15222000, 15222000
. : milestone, 15222000,
iast (18.772 s) : 18772000, 18772000
. : milestone, 18772000,
iast_GLOBAL (17.71 s) : 17710000, 17710000
. : milestone, 17710000,
profiling (15.003 s) : 15003000, 15003000
. : milestone, 15003000,
tracing (15.248 s) : 15248000, 15248000
. : milestone, 15248000,
section candidate
no_agent (15.419 s) : 15419000, 15419000
. : milestone, 15419000,
appsec (14.89 s) : 14890000, 14890000
. : milestone, 14890000,
iast (18.943 s) : 18943000, 18943000
. : milestone, 18943000,
iast_GLOBAL (17.926 s) : 17926000, 17926000
. : milestone, 17926000,
profiling (15.205 s) : 15205000, 15205000
. : milestone, 15205000,
tracing (15.081 s) : 15081000, 15081000
. : milestone, 15081000,
|
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/agent/DebuggerAgent.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/DebugSnapshotProbe.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/agent/DebuggerTransformer.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/DebugSnapshotProbe.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/DebugSnapshotProbe.java
Outdated
Show resolved
Hide resolved
...ugger/debugger-bootstrap/src/main/java/datadog/trace/bootstrap/debugger/DebuggerContext.java
Outdated
Show resolved
Hide resolved
...ugger/debugger-bootstrap/src/main/java/datadog/trace/bootstrap/debugger/DebuggerContext.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/DistDebug.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/DistDebug.java
Outdated
Show resolved
Hide resolved
...ava-agent/agent-debugger/src/main/java/com/datadog/debugger/agent/ConfigurationAcceptor.java
Outdated
Show resolved
Hide resolved
e79a48e
to
612933e
Compare
Debugger benchmarksParameters
See matching parameters
SummaryFound 5 performance improvements and 0 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.
See unchanged results
Request duration reports for reportsgantt
title reports - request duration [CI 0.99] : candidate=None, baseline=None
dateFormat X
axisFormat %s
section baseline
noprobe (282.503 µs) : 260, 305
. : milestone, 283,
basic (287.577 µs) : 280, 296
. : milestone, 288,
loop (10.95 ms) : 10925, 10975
. : milestone, 10950,
section candidate
noprobe (298.874 µs) : 254, 344
. : milestone, 299,
basic (295.32 µs) : 286, 304
. : milestone, 295,
loop (10.5 ms) : 10466, 10533
. : milestone, 10500,
|
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/SpanDebuggerProbe.java
Outdated
Show resolved
Hide resolved
...ent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/SpanDebuggerProbeManager.java
Outdated
Show resolved
Hide resolved
cab3872
to
805e31e
Compare
805e31e
to
20c5129
Compare
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/agent/DebuggerAgent.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/SpanDebuggerProbe.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/exception/Fingerprinter.java
Outdated
Show resolved
Hide resolved
...va-agent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/DefaultSpanDebugger.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/SpanDebug.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/SpanDebug.java
Outdated
Show resolved
Hide resolved
1b15fd9
to
cdd9a02
Compare
cdd9a02
to
62db520
Compare
clean up varargs warning
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/FrameAware.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/SpanDebuggerProbe.java
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/SpanDebuggerProbe.java
Outdated
Show resolved
Hide resolved
...va-agent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/DefaultSpanDebugger.java
Show resolved
Hide resolved
...ent/agent-debugger/src/main/java/com/datadog/debugger/snapshot/SpanDebuggerProbeManager.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/SpanDebuggerProbe.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/SpanDebuggerProbe.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/SpanDebuggerProbe.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The CI tests show several instrumentation errors related to not finding the SpanOriginInfo
class - those will need to be fixed before this can be merged.
Also was it intentional to remove the InstrumenterConfig.get().isSpanOriginEnabled()
checks? Without those the span-origin code will now run everywhere - even when not using live debugging.
dd-trace-core/src/main/java/datadog/trace/core/propagation/ptags/PTagsFactory.java
Outdated
Show resolved
Hide resolved
dd-trace-core/src/main/java/datadog/trace/core/taginterceptor/TagInterceptor.java
Show resolved
Hide resolved
...gger-bootstrap/src/main/java/datadog/trace/bootstrap/debugger/spanorigin/SpanOriginInfo.java
Outdated
Show resolved
Hide resolved
...gger-bootstrap/src/main/java/datadog/trace/bootstrap/debugger/spanorigin/SpanOriginInfo.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/exception/Fingerprinter.java
Outdated
Show resolved
Hide resolved
@@ -21,7 +19,7 @@ public abstract class EntrySpanOriginInstrumentation extends Tracing implements | |||
@SuppressForbidden | |||
public EntrySpanOriginInstrumentation(String instrumentationName) { | |||
super(instrumentationName); | |||
this.matcher = namedOneOf(getAnnotations()); | |||
this.matcher = NameMatchers.namedOneOf(getAnnotations()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
side note: we prefer to static import NameMatchers
/ HierarchyMatchers
methods in the rest of the codebase
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, thanks for your patience!
This PR enhances the exist span origin feature with optional snapshot captures. It introduces variable configuration via the
_dd.p.debug
tag currently set via propagation tags. It's a follow on implementation of two RFCs:Both specs are currently in flux, however, so to some extent this PR is ahead of the curve. There are some aspects that could likely be cleaned up and I will try to call out the one specific case I'd like some feedback on in case a better option presents itself.