RFC: Error transmission through batching components #11947
Conversation
…tor into jmacd/error_transmission_rfc
…y-collector into jmacd/error_transmission_rfc
jmacd
changed the title
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #11947 +/- ##
==========================================
+ Coverage 91.59% 91.62% +0.03%
==========================================
Files 449 447 -2
Lines 23761 23731 -30
==========================================
- Hits 21763 21743 -20
+ Misses 1623 1613 -10
Partials 375 375 ☔ View full report in Codecov by Sentry. |
|
I will present this RFC in the next two Collector SIG meetings 1/7/2025 (APAC/PT) and 1/15/2025 (NA). |
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
| Concurrency behavior varies. In the case where `MergeSplit()` is used, there is | ||
| a potential for multiple requests to emit from a single request. In this case, | ||
| steps 4 and 5 are executed repeatedly while there are more requests, meaning: | ||
|
|
||
| 1. Exports are synchronous and sequential. | ||
| 2. An error causes aborting of subsequent parts of the request. |
There was a problem hiding this comment.
This is not the case anymore. This was resolved as part of #10368. All the split requests are sent concurrently now
| to enqueue the request. If the queue is full, the queue sender returns a queue-full | ||
| error immediately. |
There was a problem hiding this comment.
It's now configurable to block the caller if the queue is full
| behavior that leads to data loss. This requires one or more changes | ||
| in the exporterhelper: | ||
|
|
||
| 1. Disable the queue sender by default. In this configuration, requests |
There was a problem hiding this comment.
This is a significant change in the behavior. I don't remember any other user requests to make the in-memory queue disabled by default. I believe a significant portion of users would favor keeping in-memory queue enabled by default, even if it's prone to data loss in case of Collector's crash. If we want to change that, there should be a more extended discussion and evaluation. AFAIR, in-memory buffering is a common default setting in other data processing agents.
| - Recognize partial splitting. When a request is split, leaving | ||
| part of a request that is not full, it remains in the active | ||
| request (i.e., still pending). |
There was a problem hiding this comment.
I'm afraid this might introduce a significant delay when several incoming requests that are partially split block each other until the timeout is reached.
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
Description
Draft RFC addresses #11308.
Link to tracking issue
Fixes #11308.
Testing
n/a
Documentation
This is a new RFC. As these changes are accepted and made, user-facing
documentation will be added.