Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added non-agent-specific docs on span compression #6438

Merged
merged 9 commits into from
Nov 25, 2021
3 changes: 3 additions & 0 deletions docs/features.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
* <<open-telemetry>>
* <<log-correlation>>
* <<cross-cluster-search>>
* <<span-compression>>

include::./apm-data-security.asciidoc[]

Expand All @@ -26,3 +27,5 @@ include::./open-telemetry.asciidoc[]
include::./log-correlation.asciidoc[]

include::./cross-cluster-search.asciidoc[]

include::./span-compression.asciidoc[]
58 changes: 58 additions & 0 deletions docs/span-compression.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
[[span-compression]]
=== Span compression

In some cases, the agents may collect large amounts of very similar or identical spans in a transaction.
This can e.g. happen if spans are captured inside a loop, or for unoptimized SQL queries that use multiple queries instead of joins to fetch related data.
In such cases, the upper limit of spans per transaction (by default 500 spans) can be reached quickly, causing the agent to stop capturing possibly more relevant spans for a given transaction.
beniwohli marked this conversation as resolved.
Show resolved Hide resolved

Such repeated similar spans often aren't very relevant for themselves, especially if they are of very short duration.
They also can clutter the UI, and cause processing and storage overhead.

To address this problem, the APM agents can compress such spans into a single span.
The compressed span retains some information on the original spans, such as overall count and duration.
beniwohli marked this conversation as resolved.
Show resolved Hide resolved

[float]
[[span-compression-strategy]]
=== Compression strategies

The agent can select between two strategies to decide if two adjacent spans can be compressed.
beniwohli marked this conversation as resolved.
Show resolved Hide resolved
Both strategies have the benefit that only one previous span needs to be kept in memory.
This is important to ensure that the agent doesn't require large amounts of memory to enable span compression.

[float]
[[span-compression-same]]
==== Same-Kind strategy

The agent selects this strategy, if two adjacent spans have the same
beniwohli marked this conversation as resolved.
Show resolved Hide resolved

* span type
* span subtype
* `destination.service.resource` (e.g. database name)

[float]
[[span-compression-exact]]
==== Exact-Match strategy

The agent selects this strategy, if two adjacent spans have the same
beniwohli marked this conversation as resolved.
Show resolved Hide resolved

* span name
* span type
* span subtype
* `destination.service.resource` (e.g. database name)

[float]
[[span-compression-settings]]
=== Settings

The agent has configuration settings to define upper thresholds in terms of span duration for both strategies.
For the "Same-Kind" strategy, the limit is 5ms. For the "Exact-Match" strategy, the limit is 50ms.
Spans with longer duration are not compressed. Please refer to the agent documentation for specifics.

[float]
[[span-compression-support]]
=== Agent support

Support for span compression is available in these agents:

* Go
* Python
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have docs to link to yet? I did a quick search in each agent's book and couldn't find anything.