From 6c6f2b779989880c59007c62af0ace20d395f45d Mon Sep 17 00:00:00 2001 From: Benjamin Wohlwend Date: Thu, 25 Nov 2021 01:55:31 +0100 Subject: [PATCH] Added non-agent-specific docs on span compression (#6438) Co-authored-by: Brandon Morelli Co-authored-by: Brandon Morelli Co-authored-by: Marc Lopez Rubio (cherry picked from commit 9ebd95ed3f506071bdbd8e9f2070b78847fe1b27) --- docs/features.asciidoc | 3 ++ docs/span-compression.asciidoc | 65 ++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100644 docs/span-compression.asciidoc diff --git a/docs/features.asciidoc b/docs/features.asciidoc index e16b55c903..cbf788ee87 100644 --- a/docs/features.asciidoc +++ b/docs/features.asciidoc @@ -12,6 +12,7 @@ * <> * <> * <> +* <> include::./apm-data-security.asciidoc[] @@ -26,3 +27,5 @@ include::./open-telemetry.asciidoc[] include::./log-correlation.asciidoc[] include::./cross-cluster-search.asciidoc[] + +include::./span-compression.asciidoc[] diff --git a/docs/span-compression.asciidoc b/docs/span-compression.asciidoc new file mode 100644 index 0000000000..e10c118352 --- /dev/null +++ b/docs/span-compression.asciidoc @@ -0,0 +1,65 @@ +[[span-compression]] +=== Span compression + +In some cases, APM agents may collect large amounts of very similar or identical spans in a transaction. +For example, this can happen if spans are captured inside of a loop, or in unoptimized SQL queries that use multiple queries instead of joins to fetch related data. +In such cases, the upper limit of spans per transaction (by default, 500 spans) can be reached quickly, causing the agent to stop capturing potentially more relevant spans for a given transaction. + +Such repeated similar spans often aren't very relevant for themselves, especially if they are of very short duration. +They also can clutter the UI, and cause processing and storage overhead. + +To address this problem, the APM agents can compress such spans into a single span. +The compressed span retains most of the original span information, such as overall duration and the number of spans it represents. + +Regardless of the compression strategy, a span is elegible for compression if: + +- It has not propagated its trace context. +- Is an _exit_ span (such as database query spans). +- Its outcome is not `"failure"`. + + +[float] +[[span-compression-strategy]] +=== Compression strategies + +The APM agent can select between two strategies to decide if two adjacent spans can be compressed. +Both strategies have the benefit that only one previous span needs to be kept in memory. +This is important to ensure that the agent doesn't require large amounts of memory to enable span compression. + +[float] +[[span-compression-same]] +==== Same-Kind strategy + +The agent selects this strategy if two adjacent spans have the same: + + * span type + * span subtype + * `destination.service.resource` (e.g. database name) + +[float] +[[span-compression-exact]] +==== Exact-Match strategy + +The agent selects this strategy if two adjacent spans have the same: + + * span name + * span type + * span subtype + * `destination.service.resource` (e.g. database name) + +[float] +[[span-compression-settings]] +=== Settings + +The agent has configuration settings to define upper thresholds in terms of span duration for both strategies. +For the "Same-Kind" strategy, the limit is 5ms. For the "Exact-Match" strategy, the limit is 50ms. +Spans with longer duration are not compressed. Please refer to the agent documentation for specifics. + +[float] +[[span-compression-support]] +=== Agent support + +Support for span compression is available in these agents: + + * Go: https://www.elastic.co/guide/en/apm/agent/go/master/configuration.html#config-span-compression-exact-match-duration[`ELASTIC_APM_SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION`], https://www.elastic.co/guide/en/apm/agent/go/master/configuration.html#config-span-compression-same-kind-duration[`ELASTIC_APM_SPAN_COMPRESSION_SAME_KIND_MAX_DURATION`] + * Python: {apm-py-ref}/configuration.html#config-span-compression-exact-match-max_duration[`span_compression_exact_match_max_duration`], {apm-py-ref}/configuration.html#config-span-compression-same-kind-max-duration[`span_compression_same_kind_max_duration`] \ No newline at end of file