Skip to content

Log buffer sizing

Tomasz Kowalczewski edited this page Dec 20, 2020 · 11 revisions

Tjahzi uses ManyToOneRingBuffer from Agrona library for a Log buffer component. It is placed between logging threads and thread that sends log entries to Loki.

                [via 10kB thread local buffer]
                           │                                          
+-------------+    Log  <──┘                                                
│ Application │----------------+                                          
│  Thread 1   │                │                                          
+-------------+                │                                          
                               │                                          
       .                      \│/                                          
                          +----+---------------+       
       .      Log         │                    │       
          --------------->┤    *Log buffer*    ├--> ...
       .                  │                    │       
                          +----------+---------+       
       .                            /│\                                    
                                     │                                    
+-------------+      Log             │                                    
│ Application │----------------------+                                    
│  Thread N   │                                                           
+-------------+                                                           

This code makes sure that putting log entry onto the buffer has low overhead. For this reason it avoids using integer division for calculating record offset and uses simple and fast binary math. The downside is that its size must be a power of two (plus some additional bytes at the end used for bookkeeping, see TRAILER_LENGTH in RingBufferDescriptor).

Clone this wiki locally