Optimize per-test code coverage #7315

nikita-tkachenko-datadog · 2024-07-12T10:28:36Z

What Does This Do

Reduces the overhead of per-test code coverage.

Additional Notes

The following things were optimised:

for file-level and line-level coverage: split a single coverage probes storage into multiple instances, each confined to a single thread. The final report gathers data from every thread-local storage.
for line-level code coverage: replace bitmap with byte array for storing activated coverage probes
for line-level code coverage: avoid map-lookups when possible by caching last seen execution data in a field

Also some cosmetic improvements were done: moving classes around, renaming methods, improving documentation and comments.

Jira ticket: SDTEST-533

pr-commenter · 2024-07-12T11:15:00Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/segmentful-coverage-optimisation
git_commit_date	1720791451	1720791590
git_commit_sha	`f0eec59`	`8519757`
release_version	1.38.0-SNAPSHOT~f0eec59df9	1.38.0-SNAPSHOT~8519757f6d

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1720794191	1720794191
ci_job_id	570881735	570881735
ci_pipeline_id	39035849	39035849
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 10 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1063815
Total [baseline] (10.299 s) : 0, 10299356
Agent [candidate] (1.063 s) : 0, 1063072
Total [candidate] (10.304 s) : 0, 10304312
section appsec
Agent [baseline] (1.183 s) : 0, 1183121
Total [baseline] (10.512 s) : 0, 10512233
Agent [candidate] (1.182 s) : 0, 1182447
Total [candidate] (10.481 s) : 0, 10480667
section iast
Agent [baseline] (1.173 s) : 0, 1173006
Total [baseline] (10.726 s) : 0, 10726373
Agent [candidate] (1.171 s) : 0, 1170946
Total [candidate] (10.727 s) : 0, 10727136
section profiling
Agent [baseline] (1.264 s) : 0, 1264490
Total [baseline] (10.545 s) : 0, 10544921
Agent [candidate] (1.273 s) : 0, 1272779
Total [candidate] (10.559 s) : 0, 10559023

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.064 s	-
Agent	appsec	1.183 s	119.306 ms (11.2%)
Agent	iast	1.173 s	109.19 ms (10.3%)
Agent	profiling	1.264 s	200.674 ms (18.9%)
Total	tracing	10.299 s	-
Total	appsec	10.512 s	212.877 ms (2.1%)
Total	iast	10.726 s	427.017 ms (4.1%)
Total	profiling	10.545 s	245.565 ms (2.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	appsec	1.182 s	119.375 ms (11.2%)
Agent	iast	1.171 s	107.874 ms (10.1%)
Agent	profiling	1.273 s	209.707 ms (19.7%)
Total	tracing	10.304 s	-
Total	appsec	10.481 s	176.355 ms (1.7%)
Total	iast	10.727 s	422.823 ms (4.1%)
Total	profiling	10.559 s	254.711 ms (2.5%)

gantt
    title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (666.216 ms) : 0, 666216
BytebuddyAgent [candidate] (665.104 ms) : 0, 665104
GlobalTracer [baseline] (304.787 ms) : 0, 304787
GlobalTracer [candidate] (304.953 ms) : 0, 304953
AppSec [baseline] (49.978 ms) : 0, 49978
AppSec [candidate] (50.038 ms) : 0, 50038
Remote Config [baseline] (678.459 µs) : 0, 678
Remote Config [candidate] (671.569 µs) : 0, 672
Telemetry [baseline] (7.584 ms) : 0, 7584
Telemetry [candidate] (7.608 ms) : 0, 7608
section appsec
BytebuddyAgent [baseline] (677.279 ms) : 0, 677279
BytebuddyAgent [candidate] (675.796 ms) : 0, 675796
GlobalTracer [baseline] (299.196 ms) : 0, 299196
GlobalTracer [candidate] (298.762 ms) : 0, 298762
AppSec [baseline] (153.244 ms) : 0, 153244
AppSec [candidate] (153.614 ms) : 0, 153614
IAST [baseline] (20.843 ms) : 0, 20843
IAST [candidate] (20.129 ms) : 0, 20129
Remote Config [baseline] (622.344 µs) : 0, 622
Remote Config [candidate] (618.798 µs) : 0, 619
Telemetry [baseline] (7.643 ms) : 0, 7643
Telemetry [candidate] (8.895 ms) : 0, 8895
section iast
BytebuddyAgent [baseline] (780.778 ms) : 0, 780778
BytebuddyAgent [candidate] (780.175 ms) : 0, 780175
GlobalTracer [baseline] (296.237 ms) : 0, 296237
GlobalTracer [candidate] (295.822 ms) : 0, 295822
AppSec [baseline] (50.292 ms) : 0, 50292
AppSec [candidate] (48.525 ms) : 0, 48525
IAST [baseline] (24.718 ms) : 0, 24718
IAST [candidate] (25.401 ms) : 0, 25401
Remote Config [baseline] (565.297 µs) : 0, 565
Remote Config [candidate] (567.502 µs) : 0, 568
Telemetry [baseline] (6.92 ms) : 0, 6920
Telemetry [candidate] (6.863 ms) : 0, 6863
section profiling
BytebuddyAgent [baseline] (662.572 ms) : 0, 662572
BytebuddyAgent [candidate] (667.865 ms) : 0, 667865
GlobalTracer [baseline] (388.148 ms) : 0, 388148
GlobalTracer [candidate] (391.411 ms) : 0, 391411
AppSec [baseline] (51.685 ms) : 0, 51685
AppSec [candidate] (51.634 ms) : 0, 51634
Remote Config [baseline] (659.418 µs) : 0, 659
Remote Config [candidate] (666.395 µs) : 0, 666
Telemetry [baseline] (7.409 ms) : 0, 7409
Telemetry [candidate] (7.398 ms) : 0, 7398
ProfilingAgent [baseline] (96.69 ms) : 0, 96690
ProfilingAgent [candidate] (95.978 ms) : 0, 95978
Profiling [baseline] (96.716 ms) : 0, 96716
Profiling [candidate] (96.002 ms) : 0, 96002

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062903
Total [baseline] (8.505 s) : 0, 8505173
Agent [candidate] (1.07 s) : 0, 1069733
Total [candidate] (8.519 s) : 0, 8518890
section iast
Agent [baseline] (1.171 s) : 0, 1170891
Total [baseline] (8.951 s) : 0, 8950571
Agent [candidate] (1.185 s) : 0, 1185471
Total [candidate] (9.042 s) : 0, 9041529
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.174 s) : 0, 1173655
Total [baseline] (8.928 s) : 0, 8928242
Agent [candidate] (1.17 s) : 0, 1170416
Total [candidate] (8.91 s) : 0, 8909966
section iast_TELEMETRY_OFF
Agent [baseline] (1.175 s) : 0, 1174862
Total [baseline] (8.97 s) : 0, 8969819
Agent [candidate] (1.185 s) : 0, 1184658
Total [candidate] (8.986 s) : 0, 8985872

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	iast	1.171 s	107.987 ms (10.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.174 s	110.751 ms (10.4%)
Agent	iast_TELEMETRY_OFF	1.175 s	111.959 ms (10.5%)
Total	tracing	8.505 s	-
Total	iast	8.951 s	445.398 ms (5.2%)
Total	iast_HARDCODED_SECRET_DISABLED	8.928 s	423.068 ms (5.0%)
Total	iast_TELEMETRY_OFF	8.97 s	464.645 ms (5.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.07 s	-
Agent	iast	1.185 s	115.737 ms (10.8%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.17 s	100.683 ms (9.4%)
Agent	iast_TELEMETRY_OFF	1.185 s	114.925 ms (10.7%)
Total	tracing	8.519 s	-
Total	iast	9.042 s	522.639 ms (6.1%)
Total	iast_HARDCODED_SECRET_DISABLED	8.91 s	391.077 ms (4.6%)
Total	iast_TELEMETRY_OFF	8.986 s	466.982 ms (5.5%)

gantt
    title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (664.847 ms) : 0, 664847
BytebuddyAgent [candidate] (669.462 ms) : 0, 669462
GlobalTracer [baseline] (305.435 ms) : 0, 305435
GlobalTracer [candidate] (306.699 ms) : 0, 306699
AppSec [baseline] (49.935 ms) : 0, 49935
AppSec [candidate] (50.424 ms) : 0, 50424
Remote Config [baseline] (675.877 µs) : 0, 676
Remote Config [candidate] (675.25 µs) : 0, 675
Telemetry [baseline] (7.586 ms) : 0, 7586
Telemetry [candidate] (7.693 ms) : 0, 7693
section iast
BytebuddyAgent [baseline] (780.002 ms) : 0, 780002
BytebuddyAgent [candidate] (789.812 ms) : 0, 789812
GlobalTracer [baseline] (295.455 ms) : 0, 295455
GlobalTracer [candidate] (299.115 ms) : 0, 299115
AppSec [baseline] (48.644 ms) : 0, 48644
AppSec [candidate] (49.337 ms) : 0, 49337
IAST [baseline] (25.882 ms) : 0, 25882
IAST [candidate] (26.051 ms) : 0, 26051
Remote Config [baseline] (559.136 µs) : 0, 559
Remote Config [candidate] (590.75 µs) : 0, 591
Telemetry [baseline] (6.871 ms) : 0, 6871
Telemetry [candidate] (7.019 ms) : 0, 7019
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (781.244 ms) : 0, 781244
BytebuddyAgent [candidate] (779.306 ms) : 0, 779306
GlobalTracer [baseline] (295.544 ms) : 0, 295544
GlobalTracer [candidate] (295.29 ms) : 0, 295290
AppSec [baseline] (47.405 ms) : 0, 47405
AppSec [candidate] (49.562 ms) : 0, 49562
IAST [baseline] (28.416 ms) : 0, 28416
IAST [candidate] (25.275 ms) : 0, 25275
Remote Config [baseline] (565.82 µs) : 0, 566
Remote Config [candidate] (558.641 µs) : 0, 559
Telemetry [baseline] (6.929 ms) : 0, 6929
Telemetry [candidate] (6.915 ms) : 0, 6915
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (781.877 ms) : 0, 781877
BytebuddyAgent [candidate] (788.571 ms) : 0, 788571
GlobalTracer [baseline] (296.507 ms) : 0, 296507
GlobalTracer [candidate] (298.901 ms) : 0, 298901
AppSec [baseline] (47.46 ms) : 0, 47460
AppSec [candidate] (47.63 ms) : 0, 47630
IAST [baseline] (28.017 ms) : 0, 28017
IAST [candidate] (28.393 ms) : 0, 28393
Remote Config [baseline] (571.134 µs) : 0, 571
Remote Config [candidate] (580.843 µs) : 0, 581
Telemetry [baseline] (6.817 ms) : 0, 6817
Telemetry [candidate] (6.87 ms) : 0, 6870

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-07-12T13:54:46	2024-07-12T14:01:32
git_branch	master	nikita-tkachenko/segmentful-coverage-optimisation
git_commit_date	1720791451	1720791590
git_commit_sha	`f0eec59`	`8519757`
release_version	1.38.0-SNAPSHOT~f0eec59df9	1.38.0-SNAPSHOT~8519757f6d
start_time	2024-07-12T13:54:33	2024-07-12T14:01:18

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1720793236	1720793236
ci_job_id	570881736	570881736
ci_pipeline_id	39035849	39035849
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 11 metrics, 16 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	worse [+41.913µs; +95.657µs] or [+2.843%; +6.489%]	unstable [-807.320op/s; +333.246op/s] or [-25.229%; +10.414%]	1.543ms	2962.963op/s	1.474ms	3200.000op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.325 ms) : 1306, 1344
.   : milestone, 1325,
appsec (1.706 ms) : 1683, 1729
.   : milestone, 1706,
appsec_no_iast (1.722 ms) : 1698, 1746
.   : milestone, 1722,
iast (1.451 ms) : 1428, 1473
.   : milestone, 1451,
profiling (1.474 ms) : 1449, 1499
.   : milestone, 1474,
tracing (1.461 ms) : 1438, 1485
.   : milestone, 1461,
section candidate
no_agent (1.329 ms) : 1310, 1348
.   : milestone, 1329,
appsec (1.689 ms) : 1667, 1712
.   : milestone, 1689,
appsec_no_iast (1.711 ms) : 1686, 1735
.   : milestone, 1711,
iast (1.489 ms) : 1467, 1511
.   : milestone, 1489,
profiling (1.543 ms) : 1518, 1568
.   : milestone, 1543,
tracing (1.45 ms) : 1425, 1475
.   : milestone, 1450,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.325 ms [1.306 ms, 1.344 ms]	-
appsec	1.706 ms [1.683 ms, 1.729 ms]	381.304 µs (28.8%)
appsec_no_iast	1.722 ms [1.698 ms, 1.746 ms]	397.527 µs (30.0%)
iast	1.451 ms [1.428 ms, 1.473 ms]	125.781 µs (9.5%)
profiling	1.474 ms [1.449 ms, 1.499 ms]	149.391 µs (11.3%)
tracing	1.461 ms [1.438 ms, 1.485 ms]	136.547 µs (10.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.329 ms [1.31 ms, 1.348 ms]	-
appsec	1.689 ms [1.667 ms, 1.712 ms]	359.868 µs (27.1%)
appsec_no_iast	1.711 ms [1.686 ms, 1.735 ms]	381.317 µs (28.7%)
iast	1.489 ms [1.467 ms, 1.511 ms]	159.914 µs (12.0%)
profiling	1.543 ms [1.518 ms, 1.568 ms]	213.652 µs (16.1%)
tracing	1.45 ms [1.425 ms, 1.475 ms]	120.426 µs (9.1%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (366.506 µs) : 347, 386
.   : milestone, 367,
iast (482.064 µs) : 460, 504
.   : milestone, 482,
iast_FULL (546.629 µs) : 526, 568
.   : milestone, 547,
iast_GLOBAL (505.349 µs) : 483, 528
.   : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (476.915 µs) : 456, 498
.   : milestone, 477,
iast_INACTIVE (455.384 µs) : 433, 478
.   : milestone, 455,
iast_TELEMETRY_OFF (463.2 µs) : 442, 484
.   : milestone, 463,
tracing (453.702 µs) : 433, 474
.   : milestone, 454,
section candidate
no_agent (371.973 µs) : 352, 391
.   : milestone, 372,
iast (476.422 µs) : 455, 497
.   : milestone, 476,
iast_FULL (549.806 µs) : 529, 571
.   : milestone, 550,
iast_GLOBAL (507.013 µs) : 485, 529
.   : milestone, 507,
iast_HARDCODED_SECRET_DISABLED (472.191 µs) : 451, 493
.   : milestone, 472,
iast_INACTIVE (448.907 µs) : 428, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (465.415 µs) : 444, 487
.   : milestone, 465,
tracing (437.085 µs) : 416, 458
.   : milestone, 437,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	366.506 µs [347.275 µs, 385.736 µs]	-
iast	482.064 µs [460.187 µs, 503.942 µs]	115.559 µs (31.5%)
iast_FULL	546.629 µs [525.526 µs, 567.731 µs]	180.123 µs (49.1%)
iast_GLOBAL	505.349 µs [482.748 µs, 527.95 µs]	138.844 µs (37.9%)
iast_HARDCODED_SECRET_DISABLED	476.915 µs [455.965 µs, 497.866 µs]	110.41 µs (30.1%)
iast_INACTIVE	455.384 µs [433.212 µs, 477.555 µs]	88.878 µs (24.3%)
iast_TELEMETRY_OFF	463.2 µs [442.414 µs, 483.986 µs]	96.694 µs (26.4%)
tracing	453.702 µs [433.176 µs, 474.229 µs]	87.197 µs (23.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	371.973 µs [352.478 µs, 391.468 µs]	-
iast	476.422 µs [455.444 µs, 497.399 µs]	104.448 µs (28.1%)
iast_FULL	549.806 µs [528.642 µs, 570.97 µs]	177.833 µs (47.8%)
iast_GLOBAL	507.013 µs [485.018 µs, 529.008 µs]	135.039 µs (36.3%)
iast_HARDCODED_SECRET_DISABLED	472.191 µs [451.334 µs, 493.049 µs]	100.218 µs (26.9%)
iast_INACTIVE	448.907 µs [427.715 µs, 470.099 µs]	76.934 µs (20.7%)
iast_TELEMETRY_OFF	465.415 µs [444.103 µs, 486.726 µs]	93.441 µs (25.1%)
tracing	437.085 µs [416.159 µs, 458.012 µs]	65.112 µs (17.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/segmentful-coverage-optimisation
git_commit_date	1720791451	1720791590
git_commit_sha	`f0eec59`	`8519757`
release_version	1.38.0-SNAPSHOT~f0eec59df9	1.38.0-SNAPSHOT~8519757f6d

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1720794015	1720794015
ci_job_id	570881737	570881737
ci_pipeline_id	39035849	39035849
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.175 s) : 15175000, 15175000
.   : milestone, 15175000,
appsec (14.895 s) : 14895000, 14895000
.   : milestone, 14895000,
iast (18.61 s) : 18610000, 18610000
.   : milestone, 18610000,
iast_GLOBAL (17.964 s) : 17964000, 17964000
.   : milestone, 17964000,
profiling (15.202 s) : 15202000, 15202000
.   : milestone, 15202000,
tracing (15.255 s) : 15255000, 15255000
.   : milestone, 15255000,
section candidate
no_agent (15.445 s) : 15445000, 15445000
.   : milestone, 15445000,
appsec (15.145 s) : 15145000, 15145000
.   : milestone, 15145000,
iast (18.726 s) : 18726000, 18726000
.   : milestone, 18726000,
iast_GLOBAL (18.226 s) : 18226000, 18226000
.   : milestone, 18226000,
profiling (14.854 s) : 14854000, 14854000
.   : milestone, 14854000,
tracing (15.042 s) : 15042000, 15042000
.   : milestone, 15042000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.175 s [15.175 s, 15.175 s]	-
appsec	14.895 s [14.895 s, 14.895 s]	-280.0 ms (-1.8%)
iast	18.61 s [18.61 s, 18.61 s]	3.435 s (22.6%)
iast_GLOBAL	17.964 s [17.964 s, 17.964 s]	2.789 s (18.4%)
profiling	15.202 s [15.202 s, 15.202 s]	27.0 ms (0.2%)
tracing	15.255 s [15.255 s, 15.255 s]	80.0 ms (0.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.445 s [15.445 s, 15.445 s]	-
appsec	15.145 s [15.145 s, 15.145 s]	-300.0 ms (-1.9%)
iast	18.726 s [18.726 s, 18.726 s]	3.281 s (21.2%)
iast_GLOBAL	18.226 s [18.226 s, 18.226 s]	2.781 s (18.0%)
profiling	14.854 s [14.854 s, 14.854 s]	-591.0 ms (-3.8%)
tracing	15.042 s [15.042 s, 15.042 s]	-403.0 ms (-2.6%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~8519757f6d, baseline=1.38.0-SNAPSHOT~f0eec59df9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1451, 1474
.   : milestone, 1463,
appsec (2.231 ms) : 2196, 2266
.   : milestone, 2231,
iast (1.983 ms) : 1941, 2025
.   : milestone, 1983,
iast_GLOBAL (2.014 ms) : 1972, 2056
.   : milestone, 2014,
profiling (1.865 ms) : 1831, 1900
.   : milestone, 1865,
tracing (1.84 ms) : 1807, 1873
.   : milestone, 1840,
section candidate
no_agent (1.465 ms) : 1453, 1476
.   : milestone, 1465,
appsec (2.221 ms) : 2187, 2256
.   : milestone, 2221,
iast (1.976 ms) : 1934, 2017
.   : milestone, 1976,
iast_GLOBAL (2.015 ms) : 1974, 2056
.   : milestone, 2015,
profiling (1.854 ms) : 1821, 1888
.   : milestone, 1854,
tracing (1.849 ms) : 1815, 1882
.   : milestone, 1849,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.463 ms [1.451 ms, 1.474 ms]	-
appsec	2.231 ms [2.196 ms, 2.266 ms]	768.154 µs (52.5%)
iast	1.983 ms [1.941 ms, 2.025 ms]	520.761 µs (35.6%)
iast_GLOBAL	2.014 ms [1.972 ms, 2.056 ms]	551.726 µs (37.7%)
profiling	1.865 ms [1.831 ms, 1.9 ms]	402.809 µs (27.5%)
tracing	1.84 ms [1.807 ms, 1.873 ms]	377.712 µs (25.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.465 ms [1.453 ms, 1.476 ms]	-
appsec	2.221 ms [2.187 ms, 2.256 ms]	756.576 µs (51.7%)
iast	1.976 ms [1.934 ms, 2.017 ms]	510.836 µs (34.9%)
iast_GLOBAL	2.015 ms [1.974 ms, 2.056 ms]	549.829 µs (37.5%)
profiling	1.854 ms [1.821 ms, 1.888 ms]	389.225 µs (26.6%)
tracing	1.849 ms [1.815 ms, 1.882 ms]	383.866 µs (26.2%)

nikita-tkachenko-datadog added tag: performance Performance related changes comp: ci visibility Continuous Integration Visibility run-tests: all Run all tests labels Jul 12, 2024

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/segmentful-coverage-optimisation branch 2 times, most recently from 9272c4c to 577d498 Compare July 12, 2024 12:16

Optimize per-test code coverage

8519757

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/segmentful-coverage-optimisation branch from 577d498 to 8519757 Compare July 12, 2024 13:39

nikita-tkachenko-datadog marked this pull request as ready for review July 12, 2024 15:18

nikita-tkachenko-datadog requested review from a team as code owners July 12, 2024 15:18

nikita-tkachenko-datadog requested review from romainkomorndatadog, mcculls and PerfectSlayer July 12, 2024 15:18

romainkomorndatadog approved these changes Jul 15, 2024

View reviewed changes

nikita-tkachenko-datadog merged commit f88def8 into master Jul 15, 2024
132 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/segmentful-coverage-optimisation branch July 15, 2024 10:07

github-actions bot added this to the 1.38.0 milestone Jul 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize per-test code coverage #7315

Optimize per-test code coverage #7315

nikita-tkachenko-datadog commented Jul 12, 2024 •

edited

Loading

pr-commenter bot commented Jul 12, 2024 •

edited

Loading

Optimize per-test code coverage #7315

Optimize per-test code coverage #7315

Conversation

nikita-tkachenko-datadog commented Jul 12, 2024 • edited Loading

What Does This Do

Additional Notes

pr-commenter bot commented Jul 12, 2024 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

nikita-tkachenko-datadog commented Jul 12, 2024 •

edited

Loading

pr-commenter bot commented Jul 12, 2024 •

edited

Loading