What are your plans regarding telemetry?

[SebastianStehle]

Hi,

I am just curious to understand what your plans are regarding telemetry? OpenTelemetry is not ready yet, but when metrics are ready the whole telemetry abstraction could potentially be replaced with OpenTelemetry.

[ReubenBond]

Good question. I spoke recently with Noah Falk about what the options are. I would like to see us migrate to the new System.Diagnostics.Metrics APIs, which support OpenTelemetry, and remove any notion of telemetry configuration and providers in Orleans.

Do you have a preference or ideas here?

[SebastianStehle]

I would prefer that as well. I do not see a reason for another abstraction and it is easy to work with. I have migrated a few applications to OpenTelemetry traces and it works great. The big advantage for me is that you do not need any other client library for instrumentation anymore. And the best thing is that many players seem to support OTLP, the open telemetry protocol. It works great for NewRelic and Google is also migrating: https://cloud.google.com/trace/docs/setup#recommended_client_libraries

[SebastianStehle]

@ReubenBond Would it be possible to backport the telemetry improvements from 4.0 to 3.X?

[ReubenBond]

We would accept PRs, but it's unlikely that the team would do this ourselves. If you specifically mean the Open Telemetry support, then the version implemented by this PR can be ported as-is or with little changes: #6853

For the enhancements to that tracing support (eg, this PR and the PRs it references), some of those rely on changes which are only feasible in 4.0, so some of those enhancements cannot be backported without great effort. If porting the changes from the first PR, it would be worth looking at the subsequent ones to see what applies (eg, DistributedContextPropagator).

For the new .NET Metrics (eg, visible to dotnet-counters), that is possible to support since System.Diagnostics.DiagnosticSource supports .NET Standard 2.0, but the team is unlikely to undertake that work ourselves (and it's not implemented in main yet)

[SebastianStehle]

Yes, I am talking about the first PR only. Right now the activity context is already propagated (for single nodes), but it seems that child activities are only added to the first grain call or so.