[BUG] Datadog Tracer v1.48.0 WARN: Won't propagate tag: maximum trace tags header len (128) reached.

[cesarvspr]

Version of dd-trace-go

v1.48.0

Describe what happened:

Code:

          err := tracer.Inject(span.Context(), tracer.HTTPHeadersCarrier(c.Writer.Header()))
		if err != nil {
			logger.Error(fmt.Sprintf("Failed to inject trace in response: %s", err.Error()))
		}

I see the internal tracer lib logging:

Datadog Tracer v1.48.0 WARN: Won't propagate tag: maximum trace tags header len (128) reached.

err was nil and my logger was not called.

Describe what you expected:

Ideally, a method would allow the validation of span before injection or some way to suppress this log.

Steps to reproduce the issue:

Inject an invalid span.

Additional environment details (Version of Go, Operating System, etc.):

[mtoffl01]

The warning implies that some of the information relevant to distributed tracing is getting dropped because we've hit the "maximum trace tags header" size limit (of 128). All information (e.g, tags) up to the point where the limit was reached will not be impacted and trace injection will still occur. It's possible you'll see some impact on the stability of distributed tracing — e.g, if a sampling decision tag gets dropped, then you may see unexpected gaps in your trace — but the presence of the warning does not mean trace injection will fail.

This behavior could happen for a perfectly valid span.

A quick fix for this would be to set the env var DD_TRACE_X_DATADOG_TAGS_MAX_LENGTH to a value greater than 128 but less than 512. If this does not address the problem, please open up a Support Ticket with information on the service you're instrumenting and how you're generating the "invalid" span, along with a link to this github issue.