-
Notifications
You must be signed in to change notification settings - Fork 19
Log buffer sizing
Tjahzi uses ManyToOneRingBuffer from Agrona library for a Log buffer
component. It is placed between logging threads and thread that sends log entries to Loki.
[via 10kB thread local buffer]
│
+-------------+ Log <──┘
│ Application │----------------+
│ Thread 1 │ │
+-------------+ │
│
. \│/
+----+---------------+
. Log │ │
--------------->┤ *Log buffer* ├--> ...
. │ │
+----------+---------+
. /│\
│
+-------------+ Log │
│ Application │----------------------+
│ Thread N │
+-------------+
This code makes sure that putting log entry onto the buffer has low overhead. For this reason it avoids using integer division for calculating record offset and uses simple and fast binary math. The downside is that its size must be a power of two (plus some additional bytes at the end used for bookkeeping, see TRAILER_LENGTH
in RingBufferDescriptor).
Tjahzi can be configured to create Log buffer
with sizes from 64KB to 1GB and will automatically add few bytes required by ManyToOneRingBuffer
at the end.
If configured size is not a power of two it will log a warning and will use nearest power of two greater than provided value.
When configuring log4j2 appender it cen be set like this:
<appenders>
<Loki name="Loki" bufferSizeMegabytes="64">
...
</Loki>
</appenders>