enhancement: block until span send seccessful

[zdyj3170101136]

for now, the client sent trace use an unblocking channel.

if the channel push, the span would dropped.

this would cause if a trace is too large, have too much span, then it would loss some span in trace, unavoidable.

[darccio]

@zdyj3170101136 Please see the WithPartialFlushing option that helps in this kind of situations:

WithPartialFlushing enables flushing of partially finished traces. This is done after "numSpans" have finished in a single local trace at which point all finished spans in that trace will be flushed, freeing up any memory they were consuming. This can also be configured by setting DD_TRACE_PARTIAL_FLUSH_ENABLED to true, which will default to 1000 spans unless overriden with DD_TRACE_PARTIAL_FLUSH_MIN_SPANS. Partial flushing is disabled by default.

[zdyj3170101136]

@zdyj3170101136 Please see the WithPartialFlushing option that helps in this kind of situations:

WithPartialFlushing enables flushing of partially finished traces. This is done after "numSpans" have finished in a single local trace at which point all finished spans in that trace will be flushed, freeing up any memory they were consuming. This can also be configured by setting DD_TRACE_PARTIAL_FLUSH_ENABLED to true, which will default to 1000 spans unless overriden with DD_TRACE_PARTIAL_FLUSH_MIN_SPANS. Partial flushing is disabled by default.

my meaning is, the client would push span to channel and if channel full it would dropped.

so if one program have high qps, it's span per second would too high, so a huge percent of span would dropped.

i think the client should promise each span should be sent to agent.

[darccio]

@zdyj3170101136 We recommend to use sampling too in this kind of situations. Trying to guarantee that every span is sent to the agent could lead to high memory consumption if any back pressure is generated by the agent. For further discussion, we also recommend to get in touch with support. They'll help you to identify the root cause of the issue.

[zdyj3170101136]

@zdyj3170101136我们建议在这种情况下也使用抽样。如果代理产生任何背压，尝试保证每个跨度都发送到代理可能会导致高内存消耗。如需进一步讨论，我们还建议联系支持人员。他们将帮助您确定问题的根本原因。

here is my company solution:

let every agent sent full trace, and sample at collector.

so that we will get useful trace: like p99, error and so on.

[darccio]

@zdyj3170101136 We recommend in this case to contact support, to understand better your use case and be able to help. Please refer to this GitHub issue in the support ticket.