Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix NPE when tracing named parameterised tests in JUnit 4 instrumentation #6125

Merged
merged 1 commit into from
Nov 3, 2023
Merged

Fix NPE when tracing named parameterised tests in JUnit 4 instrumentation #6125

merged 1 commit into from
Nov 3, 2023

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Oct 31, 2023 •
edited by jira bot
Loading

What Does This Do

Fixes NullPointerException when JUnit 4 tracing listener receives events from a named parameterised test case.

Motivation

When a named parameterised test case is executed, there is a "test suite started" notification with a description that corresponds to a set of parameters.
The tracing listener is not interested in this event, but it should discard it rather than fail with exception.

Additional Notes

Jira ticket: CIVIS-7864

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Oct 31, 2023
@pr-commenter
Copy link

pr-commenter bot commented Oct 31, 2023

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~4ad36f5440 1.23.0-SNAPSHOT~73f770884a
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.029 s) : 0, 1029013
Total [baseline] (8.771 s) : 0, 8770699
Agent [candidate] (1.027 s) : 0, 1027263
Total [candidate] (8.736 s) : 0, 8736169
section iast
Agent [baseline] (1.143 s) : 0, 1142746
Total [baseline] (9.267 s) : 0, 9266555
Agent [candidate] (1.15 s) : 0, 1150334
Total [candidate] (9.242 s) : 0, 9241899
section iast_TELEMETRY_OFF
Agent [baseline] (1.136 s) : 0, 1135522
Total [baseline] (9.243 s) : 0, 9242561
Agent [candidate] (1.147 s) : 0, 1146727
Total [candidate] (9.244 s) : 0, 9244228
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.029 s -
Agent iast 1.143 s 113.733 ms (11.1%)
Agent iast_TELEMETRY_OFF 1.136 s 106.508 ms (10.4%)
Total tracing 8.771 s -
Total iast 9.267 s 495.856 ms (5.7%)
Total iast_TELEMETRY_OFF 9.243 s 471.862 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.027 s -
Agent iast 1.15 s 123.071 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.147 s 119.464 ms (11.6%)
Total tracing 8.736 s -
Total iast 9.242 s 505.73 ms (5.8%)
Total iast_TELEMETRY_OFF 9.244 s 508.059 ms (5.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (642.314 ms) : 0, 642314
BytebuddyAgent [candidate] (642.278 ms) : 0, 642278
GlobalTracer [baseline] (295.576 ms) : 0, 295576
GlobalTracer [candidate] (294.434 ms) : 0, 294434
AppSec [baseline] (49.77 ms) : 0, 49770
AppSec [candidate] (49.201 ms) : 0, 49201
Remote Config [baseline] (688.394 µs) : 0, 688
Remote Config [candidate] (686.81 µs) : 0, 687
Telemetry [baseline] (6.156 ms) : 0, 6156
Telemetry [candidate] (6.118 ms) : 0, 6118
section iast
BytebuddyAgent [baseline] (761.374 ms) : 0, 761374
BytebuddyAgent [candidate] (769.679 ms) : 0, 769679
GlobalTracer [baseline] (274.61 ms) : 0, 274610
GlobalTracer [candidate] (273.638 ms) : 0, 273638
AppSec [baseline] (46.552 ms) : 0, 46552
AppSec [candidate] (47.596 ms) : 0, 47596
Remote Config [baseline] (558.31 µs) : 0, 558
Remote Config [candidate] (586.446 µs) : 0, 586
Telemetry [baseline] (7.477 ms) : 0, 7477
Telemetry [candidate] (8.187 ms) : 0, 8187
IAST [baseline] (17.627 ms) : 0, 17627
IAST [candidate] (16.329 ms) : 0, 16329
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (755.315 ms) : 0, 755315
BytebuddyAgent [candidate] (766.638 ms) : 0, 766638
GlobalTracer [baseline] (274.642 ms) : 0, 274642
GlobalTracer [candidate] (274.434 ms) : 0, 274434
AppSec [baseline] (46.767 ms) : 0, 46767
AppSec [candidate] (46.502 ms) : 0, 46502
Remote Config [baseline] (551.729 µs) : 0, 552
Remote Config [candidate] (615.811 µs) : 0, 616
Telemetry [baseline] (5.377 ms) : 0, 5377
Telemetry [candidate] (6.8 ms) : 0, 6800
IAST [baseline] (18.369 ms) : 0, 18369
IAST [candidate] (17.261 ms) : 0, 17261
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1026722
Total [baseline] (9.31 s) : 0, 9310049
Agent [candidate] (1.024 s) : 0, 1024128
Total [candidate] (9.266 s) : 0, 9265931
section appsec
Agent [baseline] (1.115 s) : 0, 1115253
Total [baseline] (9.348 s) : 0, 9347884
Agent [candidate] (1.125 s) : 0, 1125284
Total [candidate] (9.401 s) : 0, 9400718
section iast
Agent [baseline] (1.14 s) : 0, 1139743
Total [baseline] (9.418 s) : 0, 9418164
Agent [candidate] (1.153 s) : 0, 1153146
Total [candidate] (9.5 s) : 0, 9500017
section profiling
Agent [baseline] (1.202 s) : 0, 1202012
Total [baseline] (9.551 s) : 0, 9551040
Agent [candidate] (1.201 s) : 0, 1200974
Total [candidate] (9.489 s) : 0, 9489377
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.027 s -
Agent appsec 1.115 s 88.531 ms (8.6%)
Agent iast 1.14 s 113.021 ms (11.0%)
Agent profiling 1.202 s 175.29 ms (17.1%)
Total tracing 9.31 s -
Total appsec 9.348 s 37.835 ms (0.4%)
Total iast 9.418 s 108.114 ms (1.2%)
Total profiling 9.551 s 240.991 ms (2.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.024 s -
Agent appsec 1.125 s 101.156 ms (9.9%)
Agent iast 1.153 s 129.017 ms (12.6%)
Agent profiling 1.201 s 176.846 ms (17.3%)
Total tracing 9.266 s -
Total appsec 9.401 s 134.787 ms (1.5%)
Total iast 9.5 s 234.086 ms (2.5%)
Total profiling 9.489 s 223.446 ms (2.4%) 
gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (642.122 ms) : 0, 642122
BytebuddyAgent [candidate] (640.542 ms) : 0, 640542
GlobalTracer [baseline] (294.581 ms) : 0, 294581
GlobalTracer [candidate] (293.326 ms) : 0, 293326
AppSec [baseline] (48.876 ms) : 0, 48876
AppSec [candidate] (49.071 ms) : 0, 49071
Remote Config [baseline] (678.899 µs) : 0, 679
Remote Config [candidate] (675.121 µs) : 0, 675
Telemetry [baseline] (6.079 ms) : 0, 6079
Telemetry [candidate] (6.097 ms) : 0, 6097
section appsec
BytebuddyAgent [baseline] (643.237 ms) : 0, 643237
BytebuddyAgent [candidate] (648.445 ms) : 0, 648445
GlobalTracer [baseline] (292.718 ms) : 0, 292718
GlobalTracer [candidate] (296.498 ms) : 0, 296498
AppSec [baseline] (138.558 ms) : 0, 138558
AppSec [candidate] (139.142 ms) : 0, 139142
Remote Config [baseline] (642.772 µs) : 0, 643
Remote Config [candidate] (649.549 µs) : 0, 650
Telemetry [baseline] (5.7 ms) : 0, 5700
Telemetry [candidate] (5.781 ms) : 0, 5781
section iast
BytebuddyAgent [baseline] (760.149 ms) : 0, 760149
BytebuddyAgent [candidate] (772.382 ms) : 0, 772382
GlobalTracer [baseline] (273.87 ms) : 0, 273870
GlobalTracer [candidate] (274.075 ms) : 0, 274075
AppSec [baseline] (46.558 ms) : 0, 46558
AppSec [candidate] (47.871 ms) : 0, 47871
Remote Config [baseline] (546.175 µs) : 0, 546
Remote Config [candidate] (1.275 ms) : 0, 1275
Telemetry [baseline] (6.096 ms) : 0, 6096
Telemetry [candidate] (6.836 ms) : 0, 6836
IAST [baseline] (18.191 ms) : 0, 18191
IAST [candidate] (16.214 ms) : 0, 16214
section profiling
BytebuddyAgent [baseline] (653.056 ms) : 0, 653056
BytebuddyAgent [candidate] (652.219 ms) : 0, 652219
GlobalTracer [baseline] (359.016 ms) : 0, 359016
GlobalTracer [candidate] (358.465 ms) : 0, 358465
AppSec [baseline] (49.371 ms) : 0, 49371
AppSec [candidate] (49.615 ms) : 0, 49615
Remote Config [baseline] (643.275 µs) : 0, 643
Remote Config [candidate] (644.695 µs) : 0, 645
Telemetry [baseline] (6.029 ms) : 0, 6029
Telemetry [candidate] (6.025 ms) : 0, 6025
ProfilingAgent [baseline] (80.573 ms) : 0, 80573
ProfilingAgent [candidate] (80.604 ms) : 0, 80604
Profiling [baseline] (80.597 ms) : 0, 80597
Profiling [candidate] (80.628 ms) : 0, 80628
Loading

Load

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~4ad36f5440 1.23.0-SNAPSHOT~73f770884a
config baseline candidate
end_time 2023-10-31T19:03:29 2023-10-31T19:19:49
start_time 2023-10-31T19:03:17 2023-10-31T19:19:36
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.047 µs) : 345, 385
.   : milestone, 365,
iast (457.26 µs) : 437, 478
.   : milestone, 457,
iast_FULL (522.281 µs) : 502, 543
.   : milestone, 522,
iast_INACTIVE (429.793 µs) : 409, 450
.   : milestone, 430,
iast_TELEMETRY_OFF (464.11 µs) : 442, 486
.   : milestone, 464,
tracing (422.081 µs) : 401, 443
.   : milestone, 422,
section candidate
no_agent (361.344 µs) : 341, 381
.   : milestone, 361,
iast (457.162 µs) : 436, 478
.   : milestone, 457,
iast_FULL (520.79 µs) : 500, 541
.   : milestone, 521,
iast_INACTIVE (430.716 µs) : 410, 452
.   : milestone, 431,
iast_TELEMETRY_OFF (449.57 µs) : 429, 470
.   : milestone, 450,
tracing (428.218 µs) : 407, 450
.   : milestone, 428,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.047 µs [344.786 µs, 385.307 µs] -
iast 457.26 µs [436.635 µs, 477.884 µs] 92.213 µs (25.3%)
iast_FULL 522.281 µs [501.678 µs, 542.883 µs] 157.234 µs (43.1%)
iast_INACTIVE 429.793 µs [409.264 µs, 450.323 µs] 64.746 µs (17.7%)
iast_TELEMETRY_OFF 464.11 µs [442.416 µs, 485.804 µs] 99.063 µs (27.1%)
tracing 422.081 µs [401.452 µs, 442.709 µs] 57.034 µs (15.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.344 µs [341.401 µs, 381.286 µs] -
iast 457.162 µs [436.123 µs, 478.2 µs] 95.818 µs (26.5%)
iast_FULL 520.79 µs [500.365 µs, 541.215 µs] 159.446 µs (44.1%)
iast_INACTIVE 430.716 µs [409.669 µs, 451.764 µs] 69.373 µs (19.2%)
iast_TELEMETRY_OFF 449.57 µs [428.848 µs, 470.291 µs] 88.226 µs (24.4%)
tracing 428.218 µs [406.914 µs, 449.521 µs] 66.874 µs (18.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.337 ms) : 1318, 1357
.   : milestone, 1337,
appsec (1.729 ms) : 1705, 1754
.   : milestone, 1729,
iast (1.46 ms) : 1436, 1485
.   : milestone, 1460,
profiling (1.488 ms) : 1463, 1513
.   : milestone, 1488,
tracing (1.448 ms) : 1424, 1471
.   : milestone, 1448,
section candidate
no_agent (1.348 ms) : 1329, 1368
.   : milestone, 1348,
appsec (1.704 ms) : 1679, 1728
.   : milestone, 1704,
iast (1.467 ms) : 1443, 1491
.   : milestone, 1467,
profiling (1.501 ms) : 1475, 1527
.   : milestone, 1501,
tracing (1.445 ms) : 1420, 1470
.   : milestone, 1445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.337 ms [1.318 ms, 1.357 ms] -
appsec 1.729 ms [1.705 ms, 1.754 ms] 391.868 µs (29.3%)
iast 1.46 ms [1.436 ms, 1.485 ms] 122.891 µs (9.2%)
profiling 1.488 ms [1.463 ms, 1.513 ms] 150.468 µs (11.3%)
tracing 1.448 ms [1.424 ms, 1.471 ms] 110.062 µs (8.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.329 ms, 1.368 ms] -
appsec 1.704 ms [1.679 ms, 1.728 ms] 355.308 µs (26.4%)
iast 1.467 ms [1.443 ms, 1.491 ms] 118.824 µs (8.8%)
profiling 1.501 ms [1.475 ms, 1.527 ms] 152.417 µs (11.3%)
tracing 1.445 ms [1.42 ms, 1.47 ms] 96.75 µs (7.2%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review November 3, 2023 14:30
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 96bc2b9 into master Nov 3, 2023
69 of 71 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/junit4-npe branch November 3, 2023 19:16
@github-actions github-actions bot added this to the 1.23.0 milestone Nov 3, 2023
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@romainkomorndatadog romainkomorndatadog romainkomorndatadog approved these changes

@tonyredondo tonyredondo Awaiting requested review from tonyredondo tonyredondo is a code owner automatically assigned from DataDog/ci-app-libraries-java

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: bug
Projects
None yet
Milestone
1.23.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @romainkomorndatadog