[Profiler] Check UDS exists when specified in agent url env var #6439
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
area:profiler
Datadog ReportBranch report: ✅ 0 Failed, 457109 Passed, 3314 Skipped, 21h 10m 42.59s Total Time ⌛ Performance Regressions vs Default Branch (1)
|
gleocadie
force-pushed
the
gleocadie/fix-customer-bug-for-uds
branch
from
64678d9 to
931a4c1
Compare
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6439) - mean (68ms) : 66, 71
. : milestone, 68,
master - mean (68ms) : 66, 71
. : milestone, 68,
section CallTarget+Inlining+NGEN
This PR (6439) - mean (983ms) : 952, 1015
. : milestone, 983,
master - mean (978ms) : 955, 1000
. : milestone, 978,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6439) - mean (107ms) : 104, 110
. : milestone, 107,
master - mean (107ms) : 105, 109
. : milestone, 107,
section CallTarget+Inlining+NGEN
This PR (6439) - mean (676ms) : 664, 688
. : milestone, 676,
master - mean (679ms) : 662, 696
. : milestone, 679,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6439) - mean (91ms) : 88, 93
. : milestone, 91,
master - mean (91ms) : 89, 93
. : milestone, 91,
section CallTarget+Inlining+NGEN
This PR (6439) - mean (636ms) : 620, 652
. : milestone, 636,
master - mean (637ms) : 617, 656
. : milestone, 637,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6439) - mean (193ms) : 189, 197
. : milestone, 193,
master - mean (194ms) : 189, 199
. : milestone, 194,
section CallTarget+Inlining+NGEN
This PR (6439) - mean (1,104ms) : 1069, 1140
. : milestone, 1104,
master - mean (1,099ms) : 1063, 1134
. : milestone, 1099,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6439) - mean (278ms) : 273, 284
. : milestone, 278,
master - mean (277ms) : 271, 284
. : milestone, 277,
section CallTarget+Inlining+NGEN
This PR (6439) - mean (877ms) : 841, 913
. : milestone, 877,
master - mean (873ms) : 848, 898
. : milestone, 873,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6439) - mean (267ms) : 263, 272
. : milestone, 267,
master - mean (271ms) : 265, 276
. : milestone, 271,
section CallTarget+Inlining+NGEN
This PR (6439) - mean (851ms) : 814, 887
. : milestone, 851,
master - mean (856ms) : 826, 886
. : milestone, 856,
|
Throughput/Crank Report ⚡Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6439) (11.315M) : 0, 11314922
master (11.176M) : 0, 11175779
benchmarks/2.9.0 (11.033M) : 0, 11032866
section Automatic
This PR (6439) (7.211M) : 0, 7211261
master (7.188M) : 0, 7187812
benchmarks/2.9.0 (7.786M) : 0, 7785853
section Trace stats
master (7.592M) : 0, 7592225
section Manual
master (11.202M) : 0, 11201615
section Manual + Automatic
This PR (6439) (6.716M) : 0, 6715991
master (6.624M) : 0, 6623622
section DD_TRACE_ENABLED=0
master (10.184M) : 0, 10184081
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6439) (9.688M) : 0, 9688275
master (9.299M) : 0, 9299402
benchmarks/2.9.0 (9.495M) : 0, 9494821
section Automatic
This PR (6439) (6.472M) : 0, 6472114
master (6.339M) : 0, 6338804
section Trace stats
master (6.490M) : 0, 6490416
section Manual
master (9.438M) : 0, 9437615
section Manual + Automatic
This PR (6439) (5.864M) : 0, 5863652
master (5.979M) : 0, 5979389
section DD_TRACE_ENABLED=0
master (8.785M) : 0, 8784619
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6439) (9.619M) : 0, 9618591
master (9.992M) : 0, 9991636
benchmarks/2.9.0 (10.020M) : 0, 10019592
section Automatic
This PR (6439) (6.402M) : 0, 6402238
master (6.531M) : 0, 6530893
benchmarks/2.9.0 (7.255M) : 0, 7255257
section Trace stats
master (7.153M) : 0, 7152931
section Manual
master (10.055M) : 0, 10055115
section Manual + Automatic
This PR (6439) (6.038M) : 0, 6038282
master (5.851M) : 0, 5850726
section DD_TRACE_ENABLED=0
master (9.524M) : 0, 9524080
|
Benchmarks Report for tracer 🐌Benchmarks for #6439 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.RedisBenchmark - Faster 🎉 Same allocations ✔️
|
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 | 1.145 | 1,436.16 | 1,254.38 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.44μs | 0.785ns | 2.94ns | 0.0159 | 0 | 0 | 1.14 KB |
| master | SendReceive |
netcoreapp3.1 | 1.88μs | 1.56ns | 5.82ns | 0.0152 | 0 | 0 | 1.14 KB |
| master | SendReceive |
net472 | 2.21μs | 0.867ns | 3.01ns | 0.184 | 0 | 0 | 1.16 KB |
| #6439 | SendReceive |
net6.0 | 1.25μs | 0.456ns | 1.76ns | 0.0161 | 0 | 0 | 1.14 KB |
| #6439 | SendReceive |
netcoreapp3.1 | 1.72μs | 0.77ns | 2.88ns | 0.0156 | 0 | 0 | 1.14 KB |
| #6439 | SendReceive |
net472 | 2.21μs | 0.908ns | 3.52ns | 0.183 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.79μs | 0.86ns | 3.33ns | 0.0222 | 0 | 0 | 1.6 KB |
| master | EnrichedLog |
netcoreapp3.1 | 4.02μs | 1.3ns | 4.85ns | 0.0225 | 0 | 0 | 1.65 KB |
| master | EnrichedLog |
net472 | 4.55μs | 3.92ns | 15.2ns | 0.323 | 0 | 0 | 2.04 KB |
| #6439 | EnrichedLog |
net6.0 | 2.72μs | 0.72ns | 2.7ns | 0.0217 | 0 | 0 | 1.6 KB |
| #6439 | EnrichedLog |
netcoreapp3.1 | 3.96μs | 2.26ns | 8.75ns | 0.0218 | 0 | 0 | 1.65 KB |
| #6439 | EnrichedLog |
net472 | 4.43μs | 2.97ns | 11.1ns | 0.323 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️
Slower ⚠️ in #6439
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0
1.124
407.23
457.77
Faster 🎉 in #6439
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0
1.141
556.20
487.32
| Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 | 1.124 | 407.23 | 457.77 |
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 | 1.141 | 556.20 | 487.32 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 407ns | 0.154ns | 0.576ns | 0.00802 | 0 | 0 | 576 B |
| master | StartFinishSpan |
netcoreapp3.1 | 543ns | 0.315ns | 1.22ns | 0.00782 | 0 | 0 | 576 B |
| master | StartFinishSpan |
net472 | 702ns | 0.585ns | 2.27ns | 0.0915 | 0 | 0 | 578 B |
| master | StartFinishScope |
net6.0 | 556ns | 0.232ns | 0.869ns | 0.00977 | 0 | 0 | 696 B |
| master | StartFinishScope |
netcoreapp3.1 | 711ns | 0.489ns | 1.83ns | 0.00939 | 0 | 0 | 696 B |
| master | StartFinishScope |
net472 | 923ns | 0.52ns | 2.01ns | 0.104 | 0 | 0 | 658 B |
| #6439 | StartFinishSpan |
net6.0 | 458ns | 0.3ns | 1.12ns | 0.00806 | 0 | 0 | 576 B |
| #6439 | StartFinishSpan |
netcoreapp3.1 | 601ns | 0.456ns | 1.71ns | 0.00775 | 0 | 0 | 576 B |
| #6439 | StartFinishSpan |
net472 | 665ns | 0.315ns | 1.22ns | 0.0916 | 0 | 0 | 578 B |
| #6439 | StartFinishScope |
net6.0 | 487ns | 0.388ns | 1.5ns | 0.00978 | 0 | 0 | 696 B |
| #6439 | StartFinishScope |
netcoreapp3.1 | 653ns | 0.381ns | 1.48ns | 0.00941 | 0 | 0 | 696 B |
| #6439 | StartFinishScope |
net472 | 841ns | 0.801ns | 3.1ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 630ns | 0.423ns | 1.58ns | 0.00979 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 897ns | 0.46ns | 1.78ns | 0.00943 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
net472 | 1.11μs | 0.573ns | 2.22ns | 0.104 | 0 | 0 | 658 B |
| #6439 | RunOnMethodBegin |
net6.0 | 660ns | 0.436ns | 1.69ns | 0.00995 | 0 | 0 | 696 B |
| #6439 | RunOnMethodBegin |
netcoreapp3.1 | 927ns | 0.83ns | 3.21ns | 0.00949 | 0 | 0 | 696 B |
| #6439 | RunOnMethodBegin |
net472 | 1.14μs | 0.629ns | 2.35ns | 0.104 | 0 | 0 | 658 B |
gleocadie
force-pushed
the
gleocadie/fix-customer-bug-for-uds
branch
from
931a4c1 to
cdddb56
Compare
andrewlock
reviewed
Dec 17, 2024
There was a problem hiding this comment.
I'm not sure about this. In the tracer we explicitly don't do this... If the customer sets the agent URL to a UDS path, I feel quite strongly that we shouldn't just ignore it, whether it exists or not.
There's a good reason for this too - sometimes the agent is slow to start (or may be restarting), and so the UDS path might temporarily not exist, but will be created by the agent shortly. In those cases, if the tracer initializes before the agent (something that does happen, both in testing and in the wild) then the profiler is permemnatly stuck with the wrong settings 😬
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary of changes
If
DD_TRACE_AGENT_URLcontains a UDS (Unix Domain Socket), check if it exists before using it.Reason for change
If we do not check for UDS existence when passing through
DD_TRACE_AGENT_URLenv var, the profiler might end up in failing at sending profiles.Implementation details
Check if the UDS exists. If not fallback on agent mode.
Test coverage
Other details