Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add smoke tests for telemetry #7928

Closed
wants to merge 6 commits into from
Closed

Add smoke tests for telemetry #7928

wants to merge 6 commits into from
+201 −11

Conversation

smola
Copy link
Member

@smola smola commented Nov 11, 2024 •
edited
Loading

What Does This Do

Add basic smoke tests for telemetry, so far testing:

  • app-started
  • app-dependencies-loaded

Adds a groovy extension to smoke tests, @RunLast, to force these tests to run at the end. This improves testing time, since telemetry tests need to wait for telemetry being sent in the background, so other tests can run meanwhile. Without this, given that these are defined in a parent class, they would run first.

Motivation

We have almost no tests doing integration testing of telemetry dependencies. System-tests do not provide enough coverage here, while our unit tests can miss all kinds of issues.

Additional Notes

Contributor Checklist

Jira ticket: APPSEC-55773 (related)

@smola smola added tag: no release notes Changes to exclude from release notes comp: telemetry Telemetry labels Nov 11, 2024
@smola smola force-pushed the smola/telemetry-smoke-tests branch from 0b12d08 to 521c091 Compare November 11, 2024 14:13
@pr-commenter
Copy link

pr-commenter bot commented Nov 11, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/telemetry-smoke-tests
git_commit_date 1731415045 1731417472
git_commit_sha 62b2aa0 5928e82
release_version 1.43.0-SNAPSHOT~62b2aa0ae5 1.43.0-SNAPSHOT~5928e829c5
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1731420841 1731420841
ci_job_id 703619458 703619458
ci_pipeline_id 48794047 48794047
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 6 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.086 s) : 0, 1086145
Total [baseline] (8.6 s) : 0, 8600279
Agent [candidate] (1.094 s) : 0, 1093696
Total [candidate] (8.623 s) : 0, 8622800
section iast
Agent [baseline] (1.207 s) : 0, 1206756
Total [baseline] (9.128 s) : 0, 9128183
Agent [candidate] (1.207 s) : 0, 1207442
Total [candidate] (9.157 s) : 0, 9156696
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.212 s) : 0, 1212034
Total [baseline] (9.114 s) : 0, 9113770
Agent [candidate] (1.213 s) : 0, 1213018
Total [candidate] (9.126 s) : 0, 9125856
section iast_TELEMETRY_OFF
Agent [baseline] (1.207 s) : 0, 1206887
Total [baseline] (9.128 s) : 0, 9128325
Agent [candidate] (1.216 s) : 0, 1215896
Total [candidate] (9.166 s) : 0, 9165898
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent iast 1.207 s 120.611 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.212 s 125.889 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.207 s 120.742 ms (11.1%)
Total tracing 8.6 s -
Total iast 9.128 s 527.904 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.114 s 513.491 ms (6.0%)
Total iast_TELEMETRY_OFF 9.128 s 528.046 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.094 s -
Agent iast 1.207 s 113.746 ms (10.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.213 s 119.322 ms (10.9%)
Agent iast_TELEMETRY_OFF 1.216 s 122.2 ms (11.2%)
Total tracing 8.623 s -
Total iast 9.157 s 533.896 ms (6.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.126 s 503.056 ms (5.8%)
Total iast_TELEMETRY_OFF 9.166 s 543.098 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (690.678 ms) : 0, 690678
BytebuddyAgent [candidate] (693.417 ms) : 0, 693417
GlobalTracer [baseline] (316.213 ms) : 0, 316213
GlobalTracer [candidate] (320.166 ms) : 0, 320166
AppSec [baseline] (53.938 ms) : 0, 53938
AppSec [candidate] (54.8 ms) : 0, 54800
Remote Config [baseline] (671.53 µs) : 0, 672
Remote Config [candidate] (685.607 µs) : 0, 686
Telemetry [baseline] (10.702 ms) : 0, 10702
Telemetry [candidate] (10.742 ms) : 0, 10742
section iast
BytebuddyAgent [baseline] (802.45 ms) : 0, 802450
BytebuddyAgent [candidate] (802.199 ms) : 0, 802199
GlobalTracer [baseline] (304.517 ms) : 0, 304517
GlobalTracer [candidate] (305.051 ms) : 0, 305051
AppSec [baseline] (57.266 ms) : 0, 57266
AppSec [candidate] (57.62 ms) : 0, 57620
IAST [baseline] (20.656 ms) : 0, 20656
IAST [candidate] (20.633 ms) : 0, 20633
Remote Config [baseline] (610.625 µs) : 0, 611
Remote Config [candidate] (617.074 µs) : 0, 617
Telemetry [baseline] (7.508 ms) : 0, 7508
Telemetry [candidate] (7.507 ms) : 0, 7507
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (805.841 ms) : 0, 805841
BytebuddyAgent [candidate] (806.015 ms) : 0, 806015
GlobalTracer [baseline] (306.252 ms) : 0, 306252
GlobalTracer [candidate] (306.747 ms) : 0, 306747
AppSec [baseline] (57.374 ms) : 0, 57374
AppSec [candidate] (57.604 ms) : 0, 57604
IAST [baseline] (20.65 ms) : 0, 20650
IAST [candidate] (20.665 ms) : 0, 20665
Remote Config [baseline] (603.049 µs) : 0, 603
Remote Config [candidate] (604.017 µs) : 0, 604
Telemetry [baseline] (7.485 ms) : 0, 7485
Telemetry [candidate] (7.527 ms) : 0, 7527
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (801.218 ms) : 0, 801218
BytebuddyAgent [candidate] (808.174 ms) : 0, 808174
GlobalTracer [baseline] (305.599 ms) : 0, 305599
GlobalTracer [candidate] (307.448 ms) : 0, 307448
AppSec [baseline] (57.078 ms) : 0, 57078
AppSec [candidate] (57.767 ms) : 0, 57767
IAST [baseline] (21.138 ms) : 0, 21138
IAST [candidate] (20.514 ms) : 0, 20514
Remote Config [baseline] (615.991 µs) : 0, 616
Remote Config [candidate] (606.707 µs) : 0, 607
Telemetry [baseline] (7.478 ms) : 0, 7478
Telemetry [candidate] (7.481 ms) : 0, 7481
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.088 s) : 0, 1087808
Total [baseline] (10.444 s) : 0, 10443832
Agent [candidate] (1.083 s) : 0, 1083442
Total [candidate] (10.438 s) : 0, 10437752
section appsec
Agent [baseline] (1.219 s) : 0, 1218960
Total [baseline] (10.673 s) : 0, 10673137
Agent [candidate] (1.224 s) : 0, 1224400
Total [candidate] (10.708 s) : 0, 10707984
section iast
Agent [baseline] (1.211 s) : 0, 1210605
Total [baseline] (10.963 s) : 0, 10962641
Agent [candidate] (1.21 s) : 0, 1210196
Total [candidate] (10.914 s) : 0, 10913624
section profiling
Agent [baseline] (1.287 s) : 0, 1287098
Total [baseline] (10.778 s) : 0, 10777877
Agent [candidate] (1.291 s) : 0, 1290622
Total [candidate] (10.728 s) : 0, 10728005
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent appsec 1.219 s 131.152 ms (12.1%)
Agent iast 1.211 s 122.797 ms (11.3%)
Agent profiling 1.287 s 199.29 ms (18.3%)
Total tracing 10.444 s -
Total appsec 10.673 s 229.305 ms (2.2%)
Total iast 10.963 s 518.809 ms (5.0%)
Total profiling 10.778 s 334.045 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.224 s 140.959 ms (13.0%)
Agent iast 1.21 s 126.755 ms (11.7%)
Agent profiling 1.291 s 207.18 ms (19.1%)
Total tracing 10.438 s -
Total appsec 10.708 s 270.232 ms (2.6%)
Total iast 10.914 s 475.871 ms (4.6%)
Total profiling 10.728 s 290.253 ms (2.8%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.775 ms) : 0, 689775
BytebuddyAgent [candidate] (688.33 ms) : 0, 688330
GlobalTracer [baseline] (317.138 ms) : 0, 317138
GlobalTracer [candidate] (316.985 ms) : 0, 316985
AppSec [baseline] (54.405 ms) : 0, 54405
AppSec [candidate] (54.376 ms) : 0, 54376
Remote Config [baseline] (669.477 µs) : 0, 669
Remote Config [candidate] (671.213 µs) : 0, 671
Telemetry [baseline] (12.045 ms) : 0, 12045
Telemetry [candidate] (9.311 ms) : 0, 9311
section appsec
BytebuddyAgent [baseline] (705.434 ms) : 0, 705434
BytebuddyAgent [candidate] (709.782 ms) : 0, 709782
GlobalTracer [baseline] (314.557 ms) : 0, 314557
GlobalTracer [candidate] (315.124 ms) : 0, 315124
AppSec [baseline] (165.622 ms) : 0, 165622
AppSec [candidate] (165.681 ms) : 0, 165681
Remote Config [baseline] (640.418 µs) : 0, 640
Remote Config [candidate] (638.806 µs) : 0, 639
Telemetry [baseline] (8.891 ms) : 0, 8891
Telemetry [candidate] (8.487 ms) : 0, 8487
IAST [baseline] (20.106 ms) : 0, 20106
IAST [candidate] (21.75 ms) : 0, 21750
section iast
BytebuddyAgent [baseline] (804.607 ms) : 0, 804607
BytebuddyAgent [candidate] (804.669 ms) : 0, 804669
GlobalTracer [baseline] (305.314 ms) : 0, 305314
GlobalTracer [candidate] (305.609 ms) : 0, 305609
AppSec [baseline] (57.75 ms) : 0, 57750
AppSec [candidate] (57.566 ms) : 0, 57566
Remote Config [baseline] (619.295 µs) : 0, 619
Remote Config [candidate] (606.182 µs) : 0, 606
Telemetry [baseline] (7.536 ms) : 0, 7536
Telemetry [candidate] (7.418 ms) : 0, 7418
IAST [baseline] (21.016 ms) : 0, 21016
IAST [candidate] (20.56 ms) : 0, 20560
section profiling
BytebuddyAgent [baseline] (683.87 ms) : 0, 683870
BytebuddyAgent [candidate] (688.166 ms) : 0, 688166
GlobalTracer [baseline] (402.834 ms) : 0, 402834
GlobalTracer [candidate] (403.303 ms) : 0, 403303
AppSec [baseline] (55.301 ms) : 0, 55301
AppSec [candidate] (54.935 ms) : 0, 54935
Remote Config [baseline] (670.449 µs) : 0, 670
Remote Config [candidate] (667.818 µs) : 0, 668
Telemetry [baseline] (13.98 ms) : 0, 13980
Telemetry [candidate] (11.975 ms) : 0, 11975
ProfilingAgent [baseline] (91.309 ms) : 0, 91309
ProfilingAgent [candidate] (92.177 ms) : 0, 92177
Profiling [baseline] (91.332 ms) : 0, 91332
Profiling [candidate] (92.2 ms) : 0, 92200
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-12T13:44:16 2024-11-12T13:51:15
git_branch master smola/telemetry-smoke-tests
git_commit_date 1731415045 1731417472
git_commit_sha 62b2aa0 5928e82
release_version 1.43.0-SNAPSHOT~62b2aa0ae5 1.43.0-SNAPSHOT~5928e829c5
start_time 2024-11-12T13:44:03 2024-11-12T13:51:01
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1731419827 1731419827
ci_job_id 703619460 703619460
ci_pipeline_id 48794047 48794047
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5
    dateFormat X
    axisFormat %s
section baseline
no_agent (377.366 µs) : 358, 397
.   : milestone, 377,
iast (491.901 µs) : 471, 513
.   : milestone, 492,
iast_FULL (658.745 µs) : 637, 680
.   : milestone, 659,
iast_GLOBAL (524.968 µs) : 503, 547
.   : milestone, 525,
iast_HARDCODED_SECRET_DISABLED (490.811 µs) : 470, 512
.   : milestone, 491,
iast_INACTIVE (458.312 µs) : 437, 480
.   : milestone, 458,
iast_TELEMETRY_OFF (481.212 µs) : 459, 503
.   : milestone, 481,
tracing (448.946 µs) : 429, 469
.   : milestone, 449,
section candidate
no_agent (375.531 µs) : 356, 395
.   : milestone, 376,
iast (491.843 µs) : 471, 513
.   : milestone, 492,
iast_FULL (648.09 µs) : 627, 669
.   : milestone, 648,
iast_GLOBAL (528.197 µs) : 506, 551
.   : milestone, 528,
iast_HARDCODED_SECRET_DISABLED (492.893 µs) : 471, 514
.   : milestone, 493,
iast_INACTIVE (455.271 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (485.133 µs) : 464, 507
.   : milestone, 485,
tracing (455.712 µs) : 435, 477
.   : milestone, 456,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.366 µs [357.589 µs, 397.143 µs] -
iast 491.901 µs [470.565 µs, 513.237 µs] 114.535 µs (30.4%)
iast_FULL 658.745 µs [637.262 µs, 680.229 µs] 281.379 µs (74.6%)
iast_GLOBAL 524.968 µs [503.349 µs, 546.588 µs] 147.602 µs (39.1%)
iast_HARDCODED_SECRET_DISABLED 490.811 µs [469.636 µs, 511.986 µs] 113.445 µs (30.1%)
iast_INACTIVE 458.312 µs [436.687 µs, 479.937 µs] 80.946 µs (21.5%)
iast_TELEMETRY_OFF 481.212 µs [459.446 µs, 502.978 µs] 103.846 µs (27.5%)
tracing 448.946 µs [428.551 µs, 469.34 µs] 71.579 µs (19.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.531 µs [355.648 µs, 395.414 µs] -
iast 491.843 µs [470.591 µs, 513.095 µs] 116.312 µs (31.0%)
iast_FULL 648.09 µs [626.855 µs, 669.326 µs] 272.559 µs (72.6%)
iast_GLOBAL 528.197 µs [505.855 µs, 550.54 µs] 152.667 µs (40.7%)
iast_HARDCODED_SECRET_DISABLED 492.893 µs [471.31 µs, 514.476 µs] 117.362 µs (31.3%)
iast_INACTIVE 455.271 µs [434.078 µs, 476.465 µs] 79.741 µs (21.2%)
iast_TELEMETRY_OFF 485.133 µs [463.539 µs, 506.728 µs] 109.603 µs (29.2%)
tracing 455.712 µs [434.74 µs, 476.684 µs] 80.181 µs (21.4%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.336 ms) : 1316, 1356
.   : milestone, 1336,
appsec (1.748 ms) : 1725, 1770
.   : milestone, 1748,
appsec_no_iast (1.73 ms) : 1705, 1755
.   : milestone, 1730,
iast (1.492 ms) : 1470, 1515
.   : milestone, 1492,
profiling (1.528 ms) : 1503, 1553
.   : milestone, 1528,
tracing (1.467 ms) : 1443, 1491
.   : milestone, 1467,
section candidate
no_agent (1.357 ms) : 1338, 1377
.   : milestone, 1357,
appsec (1.754 ms) : 1731, 1777
.   : milestone, 1754,
appsec_no_iast (1.753 ms) : 1729, 1777
.   : milestone, 1753,
iast (1.49 ms) : 1468, 1513
.   : milestone, 1490,
profiling (1.499 ms) : 1476, 1522
.   : milestone, 1499,
tracing (1.496 ms) : 1472, 1520
.   : milestone, 1496,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.336 ms [1.316 ms, 1.356 ms] -
appsec 1.748 ms [1.725 ms, 1.77 ms] 411.195 µs (30.8%)
appsec_no_iast 1.73 ms [1.705 ms, 1.755 ms] 393.853 µs (29.5%)
iast 1.492 ms [1.47 ms, 1.515 ms] 156.174 µs (11.7%)
profiling 1.528 ms [1.503 ms, 1.553 ms] 191.87 µs (14.4%)
tracing 1.467 ms [1.443 ms, 1.491 ms] 130.665 µs (9.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.338 ms, 1.377 ms] -
appsec 1.754 ms [1.731 ms, 1.777 ms] 396.714 µs (29.2%)
appsec_no_iast 1.753 ms [1.729 ms, 1.777 ms] 395.654 µs (29.2%)
iast 1.49 ms [1.468 ms, 1.513 ms] 133.181 µs (9.8%)
profiling 1.499 ms [1.476 ms, 1.522 ms] 141.658 µs (10.4%)
tracing 1.496 ms [1.472 ms, 1.52 ms] 138.379 µs (10.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/telemetry-smoke-tests
git_commit_date 1731415045 1731417472
git_commit_sha 62b2aa0 5928e82
release_version 1.43.0-SNAPSHOT~62b2aa0ae5 1.43.0-SNAPSHOT~5928e829c5
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1731420336 1731420336
ci_job_id 703619461 703619461
ci_pipeline_id 48794047 48794047
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.464 ms) : 1453, 1476
.   : milestone, 1464,
appsec (2.332 ms) : 2291, 2373
.   : milestone, 2332,
iast (2.076 ms) : 2024, 2129
.   : milestone, 2076,
iast_GLOBAL (2.124 ms) : 2072, 2177
.   : milestone, 2124,
profiling (1.942 ms) : 1901, 1984
.   : milestone, 1942,
tracing (1.908 ms) : 1869, 1947
.   : milestone, 1908,
section candidate
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.339 ms) : 2297, 2380
.   : milestone, 2339,
iast (2.064 ms) : 2012, 2115
.   : milestone, 2064,
iast_GLOBAL (2.108 ms) : 2056, 2160
.   : milestone, 2108,
profiling (1.931 ms) : 1890, 1972
.   : milestone, 1931,
tracing (1.912 ms) : 1873, 1952
.   : milestone, 1912,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.464 ms [1.453 ms, 1.476 ms] -
appsec 2.332 ms [2.291 ms, 2.373 ms] 867.444 µs (59.2%)
iast 2.076 ms [2.024 ms, 2.129 ms] 612.182 µs (41.8%)
iast_GLOBAL 2.124 ms [2.072 ms, 2.177 ms] 660.158 µs (45.1%)
profiling 1.942 ms [1.901 ms, 1.984 ms] 477.941 µs (32.6%)
tracing 1.908 ms [1.869 ms, 1.947 ms] 444.103 µs (30.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.455 ms, 1.478 ms] -
appsec 2.339 ms [2.297 ms, 2.38 ms] 872.119 µs (59.5%)
iast 2.064 ms [2.012 ms, 2.115 ms] 597.004 µs (40.7%)
iast_GLOBAL 2.108 ms [2.056 ms, 2.16 ms] 641.118 µs (43.7%)
profiling 1.931 ms [1.89 ms, 1.972 ms] 464.52 µs (31.7%)
tracing 1.912 ms [1.873 ms, 1.952 ms] 445.587 µs (30.4%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~5928e829c5, baseline=1.43.0-SNAPSHOT~62b2aa0ae5
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.063 s) : 15063000, 15063000
.   : milestone, 15063000,
appsec (15.223 s) : 15223000, 15223000
.   : milestone, 15223000,
iast (19.188 s) : 19188000, 19188000
.   : milestone, 19188000,
iast_GLOBAL (17.998 s) : 17998000, 17998000
.   : milestone, 17998000,
profiling (14.86 s) : 14860000, 14860000
.   : milestone, 14860000,
tracing (15.187 s) : 15187000, 15187000
.   : milestone, 15187000,
section candidate
no_agent (15.765 s) : 15765000, 15765000
.   : milestone, 15765000,
appsec (15.353 s) : 15353000, 15353000
.   : milestone, 15353000,
iast (18.68 s) : 18680000, 18680000
.   : milestone, 18680000,
iast_GLOBAL (18.006 s) : 18006000, 18006000
.   : milestone, 18006000,
profiling (15.494 s) : 15494000, 15494000
.   : milestone, 15494000,
tracing (15.29 s) : 15290000, 15290000
.   : milestone, 15290000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.063 s [15.063 s, 15.063 s] -
appsec 15.223 s [15.223 s, 15.223 s] 160.0 ms (1.1%)
iast 19.188 s [19.188 s, 19.188 s] 4.125 s (27.4%)
iast_GLOBAL 17.998 s [17.998 s, 17.998 s] 2.935 s (19.5%)
profiling 14.86 s [14.86 s, 14.86 s] -203.0 ms (-1.3%)
tracing 15.187 s [15.187 s, 15.187 s] 124.0 ms (0.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.765 s [15.765 s, 15.765 s] -
appsec 15.353 s [15.353 s, 15.353 s] -412.0 ms (-2.6%)
iast 18.68 s [18.68 s, 18.68 s] 2.915 s (18.5%)
iast_GLOBAL 18.006 s [18.006 s, 18.006 s] 2.241 s (14.2%)
profiling 15.494 s [15.494 s, 15.494 s] -271.0 ms (-1.7%)
tracing 15.29 s [15.29 s, 15.29 s] -475.0 ms (-3.0%)

@smola smola force-pushed the smola/telemetry-smoke-tests branch 2 times, most recently from 1196aca to 606a2ef Compare November 11, 2024 16:03
@smola smola force-pushed the smola/telemetry-smoke-tests branch from 9c1f53c to 5928e82 Compare November 12, 2024 13:34
@smola smola closed this Nov 14, 2024
@smola smola deleted the smola/telemetry-smoke-tests branch November 14, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comp: telemetry Telemetry tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@smola