Inject coverage store field into Thread class #7284

nikita-tkachenko-datadog · 2024-07-05T13:43:58Z

What Does This Do

Optimises line-level per-test code coverage.
Coverage store lookup is optimised: instead of using a ThreadLocal, a dedicated field is injected into java.lang.Thread class.

Motivation

The original coverage store thread local showed up in the performance profiles.

Additional Notes

As thread is one of the core classes and likely cannot be safely retransformed in a running JVM, the injection is done the following way:

Maven/Gradle parent process determines which JVM is used for running the tests for a specific module
the home folder for the JVM is determined
compiled java.lang.Thread class is extracted from that JVM's files (depending on whether Java 8 or newer is used, it is either extracted from rt.jar or from the java.base module archive)
the class is patched (dedicated coverage store field is injected) and is saved to a temporary folder on disk
Maven/Gradle parent patches the command that is used for forking JVM that runs the test: depending on whether Java 8 or newer is used, the patched class is either prepended to bootstrap classpath or is used to patch the java.base module.

A static final MethodHandle is used for accessing the field. This is more practical than making the thread class implement an interface, which requires adding the interface to the java.base module and updating the module's list of exported packages.

Jira ticket: SDTEST-533

mcculls

This seems like a fragile solution - I would want to see benchmarks for the same workload before and after to see how much benefit this solution brings vs the added complexity. What happens with virtual threads in Java 22?

Also I notice that previously you were adding and removing the thread-local entry.

This would have been constantly mutating the thread-local map which increases the overhead. It is sometimes better to store a holder object, such as a one-element array, in the thread-local and update that than call ThreadLocal.remove.

ie. have

ThreadLocal<CoverageProbeStore[]> COVERAGE_PROBE_STORE = COVERAGE_PROBE_STORE = ThreadLocal.withInitial(() -> new CoverageProbeStore[1]);

// ...

COVERAGE_PROBE_STORE.get()[0] = myStore;

// ...

COVERAGE_PROBE_STORE.get()[0] = null;

and then run that simpler holder solution through the same benchmark for comparison.

cc @dougqh

pr-commenter · 2024-07-12T12:52:19Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/thread-coverage-store
git_commit_date	1720791451	1720791745
git_commit_sha	`f0eec59`	`3dce858`
release_version	1.38.0-SNAPSHOT~f0eec59df9	1.38.0-SNAPSHOT~3dce858a47

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1720794276	1720794276
ci_job_id	570885200	570885200
ci_pipeline_id	39036069	39036069
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 8 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.073 s) : 0, 1073402
Total [baseline] (10.377 s) : 0, 10377164
Agent [candidate] (1.069 s) : 0, 1069025
Total [candidate] (10.375 s) : 0, 10375303
section appsec
Agent [baseline] (1.189 s) : 0, 1188551
Total [baseline] (10.493 s) : 0, 10492656
Agent [candidate] (1.189 s) : 0, 1188539
Total [candidate] (10.533 s) : 0, 10532982
section iast
Agent [baseline] (1.18 s) : 0, 1179968
Total [baseline] (10.67 s) : 0, 10670092
Agent [candidate] (1.174 s) : 0, 1173668
Total [candidate] (10.707 s) : 0, 10707498
section profiling
Agent [baseline] (1.274 s) : 0, 1273921
Total [baseline] (10.611 s) : 0, 10610619
Agent [candidate] (1.275 s) : 0, 1275470
Total [candidate] (10.584 s) : 0, 10584086

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.073 s	-
Agent	appsec	1.189 s	115.149 ms (10.7%)
Agent	iast	1.18 s	106.566 ms (9.9%)
Agent	profiling	1.274 s	200.518 ms (18.7%)
Total	tracing	10.377 s	-
Total	appsec	10.493 s	115.492 ms (1.1%)
Total	iast	10.67 s	292.928 ms (2.8%)
Total	profiling	10.611 s	233.455 ms (2.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.069 s	-
Agent	appsec	1.189 s	119.514 ms (11.2%)
Agent	iast	1.174 s	104.643 ms (9.8%)
Agent	profiling	1.275 s	206.445 ms (19.3%)
Total	tracing	10.375 s	-
Total	appsec	10.533 s	157.679 ms (1.5%)
Total	iast	10.707 s	332.195 ms (3.2%)
Total	profiling	10.584 s	208.783 ms (2.0%)

gantt
    title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.692 ms) : 0, 671692
BytebuddyAgent [candidate] (669.047 ms) : 0, 669047
GlobalTracer [baseline] (307.996 ms) : 0, 307996
GlobalTracer [candidate] (306.744 ms) : 0, 306744
AppSec [baseline] (50.411 ms) : 0, 50411
AppSec [candidate] (50.117 ms) : 0, 50117
Remote Config [baseline] (682.786 µs) : 0, 683
Remote Config [candidate] (678.515 µs) : 0, 679
Telemetry [baseline] (7.698 ms) : 0, 7698
Telemetry [candidate] (7.626 ms) : 0, 7626
section appsec
BytebuddyAgent [baseline] (679.975 ms) : 0, 679975
BytebuddyAgent [candidate] (679.792 ms) : 0, 679792
GlobalTracer [baseline] (300.997 ms) : 0, 300997
GlobalTracer [candidate] (300.691 ms) : 0, 300691
AppSec [baseline] (153.427 ms) : 0, 153427
AppSec [candidate] (153.338 ms) : 0, 153338
Remote Config [baseline] (620.38 µs) : 0, 620
Remote Config [candidate] (619.049 µs) : 0, 619
Telemetry [baseline] (8.269 ms) : 0, 8269
Telemetry [candidate] (8.553 ms) : 0, 8553
IAST [baseline] (20.915 ms) : 0, 20915
IAST [candidate] (21.367 ms) : 0, 21367
section iast
BytebuddyAgent [baseline] (786.463 ms) : 0, 786463
BytebuddyAgent [candidate] (781.263 ms) : 0, 781263
GlobalTracer [baseline] (297.306 ms) : 0, 297306
GlobalTracer [candidate] (296.059 ms) : 0, 296059
AppSec [baseline] (48.16 ms) : 0, 48160
AppSec [candidate] (48.27 ms) : 0, 48270
Remote Config [baseline] (571.726 µs) : 0, 572
Remote Config [candidate] (581.45 µs) : 0, 581
Telemetry [baseline] (6.936 ms) : 0, 6936
Telemetry [candidate] (6.916 ms) : 0, 6916
IAST [baseline] (26.892 ms) : 0, 26892
IAST [candidate] (27.003 ms) : 0, 27003
section profiling
ProfilingAgent [baseline] (96.452 ms) : 0, 96452
ProfilingAgent [candidate] (97.158 ms) : 0, 97158
BytebuddyAgent [baseline] (668.558 ms) : 0, 668558
BytebuddyAgent [candidate] (669.695 ms) : 0, 669695
GlobalTracer [baseline] (391.417 ms) : 0, 391417
GlobalTracer [candidate] (390.962 ms) : 0, 390962
AppSec [baseline] (51.846 ms) : 0, 51846
AppSec [candidate] (51.809 ms) : 0, 51809
Remote Config [baseline] (661.167 µs) : 0, 661
Remote Config [candidate] (660.348 µs) : 0, 660
Telemetry [baseline] (7.356 ms) : 0, 7356
Telemetry [candidate] (7.342 ms) : 0, 7342
Profiling [baseline] (96.477 ms) : 0, 96477
Profiling [candidate] (97.183 ms) : 0, 97183

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066349
Total [baseline] (8.52 s) : 0, 8519583
Agent [candidate] (1.065 s) : 0, 1065135
Total [candidate] (8.506 s) : 0, 8506356
section iast
Agent [baseline] (1.177 s) : 0, 1176528
Total [baseline] (8.987 s) : 0, 8987176
Agent [candidate] (1.172 s) : 0, 1171972
Total [candidate] (8.991 s) : 0, 8990792
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.172 s) : 0, 1171572
Total [baseline] (8.928 s) : 0, 8927601
Agent [candidate] (1.175 s) : 0, 1174963
Total [candidate] (8.915 s) : 0, 8915249
section iast_TELEMETRY_OFF
Agent [baseline] (1.17 s) : 0, 1169712
Total [baseline] (8.935 s) : 0, 8935431
Agent [candidate] (1.169 s) : 0, 1169116
Total [candidate] (8.94 s) : 0, 8939729

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.066 s	-
Agent	iast	1.177 s	110.179 ms (10.3%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.172 s	105.222 ms (9.9%)
Agent	iast_TELEMETRY_OFF	1.17 s	103.363 ms (9.7%)
Total	tracing	8.52 s	-
Total	iast	8.987 s	467.593 ms (5.5%)
Total	iast_HARDCODED_SECRET_DISABLED	8.928 s	408.018 ms (4.8%)
Total	iast_TELEMETRY_OFF	8.935 s	415.848 ms (4.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	iast	1.172 s	106.837 ms (10.0%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.175 s	109.828 ms (10.3%)
Agent	iast_TELEMETRY_OFF	1.169 s	103.981 ms (9.8%)
Total	tracing	8.506 s	-
Total	iast	8.991 s	484.436 ms (5.7%)
Total	iast_HARDCODED_SECRET_DISABLED	8.915 s	408.893 ms (4.8%)
Total	iast_TELEMETRY_OFF	8.94 s	433.373 ms (5.1%)

gantt
    title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (667.288 ms) : 0, 667288
BytebuddyAgent [candidate] (666.828 ms) : 0, 666828
GlobalTracer [baseline] (305.997 ms) : 0, 305997
GlobalTracer [candidate] (305.502 ms) : 0, 305502
AppSec [baseline] (50.113 ms) : 0, 50113
AppSec [candidate] (49.951 ms) : 0, 49951
Remote Config [baseline] (678.978 µs) : 0, 679
Remote Config [candidate] (676.631 µs) : 0, 677
Telemetry [baseline] (7.671 ms) : 0, 7671
Telemetry [candidate] (7.583 ms) : 0, 7583
section iast
BytebuddyAgent [baseline] (784.769 ms) : 0, 784769
BytebuddyAgent [candidate] (781.485 ms) : 0, 781485
GlobalTracer [baseline] (296.862 ms) : 0, 296862
GlobalTracer [candidate] (295.801 ms) : 0, 295801
AppSec [baseline] (48.947 ms) : 0, 48947
AppSec [candidate] (48.117 ms) : 0, 48117
IAST [baseline] (24.728 ms) : 0, 24728
IAST [candidate] (25.449 ms) : 0, 25449
Remote Config [baseline] (576.337 µs) : 0, 576
Remote Config [candidate] (579.691 µs) : 0, 580
Telemetry [baseline] (7.038 ms) : 0, 7038
Telemetry [candidate] (7.012 ms) : 0, 7012
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (780.541 ms) : 0, 780541
BytebuddyAgent [candidate] (782.036 ms) : 0, 782036
GlobalTracer [baseline] (295.658 ms) : 0, 295658
GlobalTracer [candidate] (295.602 ms) : 0, 295602
AppSec [baseline] (48.058 ms) : 0, 48058
AppSec [candidate] (50.703 ms) : 0, 50703
IAST [baseline] (26.23 ms) : 0, 26230
IAST [candidate] (24.681 ms) : 0, 24681
Remote Config [baseline] (561.414 µs) : 0, 561
Remote Config [candidate] (567.557 µs) : 0, 568
Telemetry [baseline] (6.977 ms) : 0, 6977
Telemetry [candidate] (7.746 ms) : 0, 7746
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.113 ms) : 0, 779113
BytebuddyAgent [candidate] (778.228 ms) : 0, 778228
GlobalTracer [baseline] (295.289 ms) : 0, 295289
GlobalTracer [candidate] (295.395 ms) : 0, 295395
AppSec [baseline] (47.259 ms) : 0, 47259
AppSec [candidate] (47.237 ms) : 0, 47237
IAST [baseline] (27.091 ms) : 0, 27091
IAST [candidate] (27.325 ms) : 0, 27325
Remote Config [baseline] (570.378 µs) : 0, 570
Remote Config [candidate] (561.072 µs) : 0, 561
Telemetry [baseline] (6.838 ms) : 0, 6838
Telemetry [candidate] (6.778 ms) : 0, 6778

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-07-12T13:58:15	2024-07-12T14:05:05
git_branch	master	nikita-tkachenko/thread-coverage-store
git_commit_date	1720791451	1720791745
git_commit_sha	`f0eec59`	`3dce858`
release_version	1.38.0-SNAPSHOT~f0eec59df9	1.38.0-SNAPSHOT~3dce858a47
start_time	2024-07-12T13:58:02	2024-07-12T14:04:52

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1720793450	1720793450
ci_job_id	570885201	570885201
ci_pipeline_id	39036069	39036069
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1333, 1372
.   : milestone, 1353,
appsec (1.703 ms) : 1679, 1727
.   : milestone, 1703,
appsec_no_iast (1.71 ms) : 1685, 1735
.   : milestone, 1710,
iast (1.483 ms) : 1460, 1507
.   : milestone, 1483,
profiling (1.492 ms) : 1466, 1518
.   : milestone, 1492,
tracing (1.454 ms) : 1430, 1479
.   : milestone, 1454,
section candidate
no_agent (1.349 ms) : 1330, 1368
.   : milestone, 1349,
appsec (1.717 ms) : 1693, 1741
.   : milestone, 1717,
appsec_no_iast (1.715 ms) : 1691, 1739
.   : milestone, 1715,
iast (1.468 ms) : 1446, 1491
.   : milestone, 1468,
profiling (1.507 ms) : 1483, 1532
.   : milestone, 1507,
tracing (1.481 ms) : 1457, 1504
.   : milestone, 1481,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.353 ms [1.333 ms, 1.372 ms]	-
appsec	1.703 ms [1.679 ms, 1.727 ms]	350.248 µs (25.9%)
appsec_no_iast	1.71 ms [1.685 ms, 1.735 ms]	357.49 µs (26.4%)
iast	1.483 ms [1.46 ms, 1.507 ms]	130.813 µs (9.7%)
profiling	1.492 ms [1.466 ms, 1.518 ms]	139.442 µs (10.3%)
tracing	1.454 ms [1.43 ms, 1.479 ms]	101.699 µs (7.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.349 ms [1.33 ms, 1.368 ms]	-
appsec	1.717 ms [1.693 ms, 1.741 ms]	367.963 µs (27.3%)
appsec_no_iast	1.715 ms [1.691 ms, 1.739 ms]	366.07 µs (27.1%)
iast	1.468 ms [1.446 ms, 1.491 ms]	119.349 µs (8.8%)
profiling	1.507 ms [1.483 ms, 1.532 ms]	158.168 µs (11.7%)
tracing	1.481 ms [1.457 ms, 1.504 ms]	131.963 µs (9.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (371.543 µs) : 352, 391
.   : milestone, 372,
iast (480.8 µs) : 460, 502
.   : milestone, 481,
iast_FULL (551.968 µs) : 531, 573
.   : milestone, 552,
iast_GLOBAL (520.068 µs) : 497, 543
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (485.111 µs) : 464, 506
.   : milestone, 485,
iast_INACTIVE (463.712 µs) : 442, 486
.   : milestone, 464,
iast_TELEMETRY_OFF (470.534 µs) : 449, 492
.   : milestone, 471,
tracing (450.565 µs) : 430, 471
.   : milestone, 451,
section candidate
no_agent (366.702 µs) : 346, 387
.   : milestone, 367,
iast (485.855 µs) : 465, 507
.   : milestone, 486,
iast_FULL (556.257 µs) : 535, 577
.   : milestone, 556,
iast_GLOBAL (504.015 µs) : 483, 525
.   : milestone, 504,
iast_HARDCODED_SECRET_DISABLED (483.615 µs) : 463, 505
.   : milestone, 484,
iast_INACTIVE (456.122 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (475.595 µs) : 454, 497
.   : milestone, 476,
tracing (440.201 µs) : 420, 460
.   : milestone, 440,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	371.543 µs [352.267 µs, 390.819 µs]	-
iast	480.8 µs [459.983 µs, 501.617 µs]	109.257 µs (29.4%)
iast_FULL	551.968 µs [530.974 µs, 572.962 µs]	180.425 µs (48.6%)
iast_GLOBAL	520.068 µs [496.636 µs, 543.5 µs]	148.525 µs (40.0%)
iast_HARDCODED_SECRET_DISABLED	485.111 µs [463.767 µs, 506.455 µs]	113.568 µs (30.6%)
iast_INACTIVE	463.712 µs [441.648 µs, 485.776 µs]	92.169 µs (24.8%)
iast_TELEMETRY_OFF	470.534 µs [449.429 µs, 491.639 µs]	98.991 µs (26.6%)
tracing	450.565 µs [429.945 µs, 471.185 µs]	79.022 µs (21.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	366.702 µs [346.449 µs, 386.954 µs]	-
iast	485.855 µs [464.817 µs, 506.893 µs]	119.154 µs (32.5%)
iast_FULL	556.257 µs [535.159 µs, 577.355 µs]	189.556 µs (51.7%)
iast_GLOBAL	504.015 µs [482.787 µs, 525.243 µs]	137.313 µs (37.4%)
iast_HARDCODED_SECRET_DISABLED	483.615 µs [462.596 µs, 504.634 µs]	116.913 µs (31.9%)
iast_INACTIVE	456.122 µs [435.088 µs, 477.157 µs]	89.421 µs (24.4%)
iast_TELEMETRY_OFF	475.595 µs [454.221 µs, 496.97 µs]	108.894 µs (29.7%)
tracing	440.201 µs [419.953 µs, 460.449 µs]	73.5 µs (20.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/thread-coverage-store
git_commit_date	1720791451	1720791745
git_commit_sha	`f0eec59`	`3dce858`
release_version	1.38.0-SNAPSHOT~f0eec59df9	1.38.0-SNAPSHOT~3dce858a47

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1720793979	1720793979
ci_job_id	570885202	570885202
ci_pipeline_id	39036069	39036069
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.822 s) : 14822000, 14822000
.   : milestone, 14822000,
appsec (15.16 s) : 15160000, 15160000
.   : milestone, 15160000,
iast (18.881 s) : 18881000, 18881000
.   : milestone, 18881000,
iast_GLOBAL (17.695 s) : 17695000, 17695000
.   : milestone, 17695000,
profiling (15.336 s) : 15336000, 15336000
.   : milestone, 15336000,
tracing (14.979 s) : 14979000, 14979000
.   : milestone, 14979000,
section candidate
no_agent (15.749 s) : 15749000, 15749000
.   : milestone, 15749000,
appsec (14.859 s) : 14859000, 14859000
.   : milestone, 14859000,
iast (18.932 s) : 18932000, 18932000
.   : milestone, 18932000,
iast_GLOBAL (17.803 s) : 17803000, 17803000
.   : milestone, 17803000,
profiling (15.244 s) : 15244000, 15244000
.   : milestone, 15244000,
tracing (14.964 s) : 14964000, 14964000
.   : milestone, 14964000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.822 s [14.822 s, 14.822 s]	-
appsec	15.16 s [15.16 s, 15.16 s]	338.0 ms (2.3%)
iast	18.881 s [18.881 s, 18.881 s]	4.059 s (27.4%)
iast_GLOBAL	17.695 s [17.695 s, 17.695 s]	2.873 s (19.4%)
profiling	15.336 s [15.336 s, 15.336 s]	514.0 ms (3.5%)
tracing	14.979 s [14.979 s, 14.979 s]	157.0 ms (1.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.749 s [15.749 s, 15.749 s]	-
appsec	14.859 s [14.859 s, 14.859 s]	-890.0 ms (-5.7%)
iast	18.932 s [18.932 s, 18.932 s]	3.183 s (20.2%)
iast_GLOBAL	17.803 s [17.803 s, 17.803 s]	2.054 s (13.0%)
profiling	15.244 s [15.244 s, 15.244 s]	-505.0 ms (-3.2%)
tracing	14.964 s [14.964 s, 14.964 s]	-785.0 ms (-5.0%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~3dce858a47, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.456 ms) : 1444, 1467
.   : milestone, 1456,
appsec (2.212 ms) : 2177, 2246
.   : milestone, 2212,
iast (1.954 ms) : 1913, 1996
.   : milestone, 1954,
iast_GLOBAL (2.007 ms) : 1965, 2049
.   : milestone, 2007,
profiling (1.852 ms) : 1817, 1887
.   : milestone, 1852,
tracing (1.832 ms) : 1799, 1865
.   : milestone, 1832,
section candidate
no_agent (1.454 ms) : 1443, 1465
.   : milestone, 1454,
appsec (2.207 ms) : 2173, 2242
.   : milestone, 2207,
iast (1.955 ms) : 1913, 1997
.   : milestone, 1955,
iast_GLOBAL (2.004 ms) : 1961, 2047
.   : milestone, 2004,
profiling (1.854 ms) : 1820, 1888
.   : milestone, 1854,
tracing (1.831 ms) : 1797, 1864
.   : milestone, 1831,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.456 ms [1.444 ms, 1.467 ms]	-
appsec	2.212 ms [2.177 ms, 2.246 ms]	756.126 µs (51.9%)
iast	1.954 ms [1.913 ms, 1.996 ms]	498.839 µs (34.3%)
iast_GLOBAL	2.007 ms [1.965 ms, 2.049 ms]	551.422 µs (37.9%)
profiling	1.852 ms [1.817 ms, 1.887 ms]	396.507 µs (27.2%)
tracing	1.832 ms [1.799 ms, 1.865 ms]	376.657 µs (25.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.454 ms [1.443 ms, 1.465 ms]	-
appsec	2.207 ms [2.173 ms, 2.242 ms]	753.096 µs (51.8%)
iast	1.955 ms [1.913 ms, 1.997 ms]	500.974 µs (34.5%)
iast_GLOBAL	2.004 ms [1.961 ms, 2.047 ms]	550.026 µs (37.8%)
profiling	1.854 ms [1.82 ms, 1.888 ms]	400.264 µs (27.5%)
tracing	1.831 ms [1.797 ms, 1.864 ms]	376.59 µs (25.9%)

nikita-tkachenko-datadog added comp: ci visibility Continuous Integration Visibility tag: experimental Experimental changes labels Jul 5, 2024

nikita-tkachenko-datadog closed this Jul 5, 2024

nikita-tkachenko-datadog added the run-tests: all Run all tests label Jul 5, 2024

nikita-tkachenko-datadog reopened this Jul 5, 2024

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/thread-coverage-store branch from a571e55 to 66ac3d6 Compare July 5, 2024 13:46

nikita-tkachenko-datadog marked this pull request as ready for review July 8, 2024 07:55

nikita-tkachenko-datadog requested review from a team as code owners July 8, 2024 07:55

nikita-tkachenko-datadog requested review from juan-fernandez, mcculls and dougqh July 8, 2024 07:55

mcculls requested changes Jul 8, 2024

View reviewed changes

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/thread-coverage-store branch from 2c3ac2e to f62012f Compare July 12, 2024 12:05

nikita-tkachenko-datadog changed the base branch from master to nikita-tkachenko/segmentful-coverage-optimisation July 12, 2024 12:05

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/segmentful-coverage-optimisation branch from 9272c4c to 577d498 Compare July 12, 2024 12:16

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/thread-coverage-store branch from f62012f to 3af216e Compare July 12, 2024 12:17

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/segmentful-coverage-optimisation branch from 577d498 to 8519757 Compare July 12, 2024 13:39

Inject coverage store field into Thread class

3dce858

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/thread-coverage-store branch from 3af216e to 3dce858 Compare July 12, 2024 13:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inject coverage store field into Thread class #7284

Inject coverage store field into Thread class #7284

nikita-tkachenko-datadog commented Jul 5, 2024 •

edited

Loading

mcculls left a comment

pr-commenter bot commented Jul 12, 2024 •

edited

Loading

Inject coverage store field into Thread class #7284

Are you sure you want to change the base?

Inject coverage store field into Thread class #7284

Conversation

nikita-tkachenko-datadog commented Jul 5, 2024 • edited Loading

What Does This Do

Motivation

Additional Notes

mcculls left a comment

Choose a reason for hiding this comment

pr-commenter bot commented Jul 12, 2024 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

nikita-tkachenko-datadog commented Jul 5, 2024 •

edited

Loading

pr-commenter bot commented Jul 12, 2024 •

edited

Loading