sampling/eventstorage: introduce event storage #4108
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Steps errorsExpand to view the steps failures
|
Introduce a package which provides local event storage, tailored for tail-based sampling. Storage is backed by Badger (github.com/dgraph-io/badger). The Storage type provides two types for readiing/writing data: ReadWriter, and ShardedReadWriter. ReadWriter is not safe for concurrent access, while ShardedReadWriter is. ShardedReadWriter provides locked access to one of several ReadWriters, sharding on trace ID. Currently we encode transactions and spans as JSON. We should look into developing a more efficient codec later, using protobuf or similar.
axw
force-pushed
the
tailsampling-eventstorage
branch
from
5eafce8 to
eeb1595
Compare
| } | ||
|
|
||
| // ReadEvents reads events with the given trace ID from storage into a batch. | ||
| func (rw *ReadWriter) ReadEvents(traceID string, out *model.Batch) error { |
There was a problem hiding this comment.
Since ReadEvents might flush pending writes and it is generally not safe for concurrent usage, a comment would be helpful here, indicating that this would require a write lock for safe concurrent usage.
There was a problem hiding this comment.
The ReadWriter type's doc comment already states the whole type is not safe for concurrent usage. Is there still something unclear?
There was a problem hiding this comment.
I would not necessarily expect a write action on a ReadEvents method, so when using and locking it in the caller, there might only be a read lock used and not a write lock. But one would probably look at the code and realize that, so it's ok.
There was a problem hiding this comment.
Okay, understood - I'll add a comment to ReadEvents stating that it may also cause writes. I don't think we'll end up with read/write locks as the event storage is write-heavy, and ReadEvents is only called periodically.
T for Transaction S for Span u for unsampled s for sampled
* sampling/eventstorage: introduce event storage Introduce a package which provides local event storage, tailored for tail-based sampling. Storage is backed by Badger (github.com/dgraph-io/badger). The Storage type provides two types for readiing/writing data: ReadWriter, and ShardedReadWriter. ReadWriter is not safe for concurrent access, while ShardedReadWriter is. ShardedReadWriter provides locked access to one of several ReadWriters, sharding on trace ID. Currently we encode transactions and spans as JSON. We should look into developing a more efficient codec later, using protobuf or similar.
* sampling/eventstorage: introduce event storage Introduce a package which provides local event storage, tailored for tail-based sampling. Storage is backed by Badger (github.com/dgraph-io/badger). The Storage type provides two types for readiing/writing data: ReadWriter, and ShardedReadWriter. ReadWriter is not safe for concurrent access, while ShardedReadWriter is. ShardedReadWriter provides locked access to one of several ReadWriters, sharding on trace ID. Currently we encode transactions and spans as JSON. We should look into developing a more efficient codec later, using protobuf or similar.
Motivation/summary
Introduce a package which provides local event storage, tailored for tail-based sampling. Storage is backed by Badger (github.com/dgraph-io/badger).
The Storage type provides two types for readiing/writing data: ReadWriter, and ShardedReadWriter. ReadWriter is not safe for concurrent access, while ShardedReadWriter is. ShardedReadWriter provides locked access to one of several ReadWriters, sharding on trace ID.
Currently we encode transactions and spans as JSON. We should look into developing a more efficient codec later, using protobuf or similar.
I've struck out metrics/monitoring for now, we should do that in a follow-up.
Checklist
- [ ] I have updated CHANGELOG.asciidocI have considered changes for:
- [ ] documentation- [ ] logging (add log lines, choose appropriate log selector, etc.)- [ ] metrics and monitoring (create issue for Kibana team to add metrics to visualizations, e.g. Kibana#44001)- [ ] telemetry- [ ] Elasticsearch Service (https://cloud.elastic.co)- [ ] Elastic Cloud Enterprise (https://www.elastic.co/products/ece)- [ ] Elastic Cloud on Kubernetes (https://www.elastic.co/elastic-cloud-kubernetes)How to test these changes
make testRelated issues
None.