Joining parent trace without sampling

[rauanmayemir]

I have the following env variables setup:

OTEL_TRACES_EXPORTER=otlp
OTEL_LOGS_EXPORTER=otlp
OTEL_METRICS_EXPORTER=otlp
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
OTEL_EXPORTER_OTLP_ENDPOINT=http://collector-opentelemetry-collector:4317
OTEL_PHP_TRACES_PROCESSOR=batch
OTEL_TRACES_SAMPLER=always_on

Despite explicitly setting sampler to always_on, most of the traces (like probably 90% or higher) from PHP backend are missing and not reaching my OTLP ingestion backend.

I start traces before they come into PHP backend and based on what elastic is showing me, they have sampled=true on them. What do I need to do for PHP tracer to join the parent trace and send ALL the traces without sampling?

[brettmc]

I start traces before they come into PHP backend

That sounds like distributed tracing, which requires Context Propagation - presumably the thing before the PHP backend which starts the trace is generating a trace id and a span id, and then those are injected into an outgoing HTTP request in an HTTP header.
If that's what is happening, are you using those values so that your first PHP span has a remote parent? https://opentelemetry.io/docs/languages/php/propagation/#manual-context-propagation

and send ALL the traces without sampling?

I'm not sure what you mean here.

[rauanmayemir]

@brettmc Correct. I verified that my tooling extracts traceparent header into the context and starts the trace.

I'm not sure what you mean here.

I mean that for every global distributed trace I'm collecting, PHP backend side span is missing most of the time. I've assumed that something is wrong with sampling making php instrumentation discard spans.

Thanks for the links, I'll dig deeper and report to downstream tooling if there are any bugs.

[brettmc]

Thanks for the links, I'll dig deeper and report to downstream tooling if there are any bugs.

I would also recommend switching over to the console exporter to confirm that traces are being generated (which rules out sampling etc), and then I'd use otlp with debug logging turned on in the collector to confirm that traces are making it that far. That's also a good time to check that trace IDs align with your remote spans.