You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be very powerful if Lua could be used as a Processor not just for Logs, but also Traces and Metrics. This would allow users to address issues such as implementing sampling on traces (currently an option with https://opentelemetry.io/docs/concepts/sampling/)
Describe alternatives you've considered
Ability to convert a Trace or Metric to a log and back again—this would allow all of Fluent Bit's features to be leveraged.
Additional context
Unable to apply sampling or filtering of traces to create metrics for traces with outlier thresholds (e.g a span execution time greater than X) and to sample traces to reduce the traffic volumes being sent onwards. While processing multiple events if we can sample the the latency for events that aren't showing anomalies - we can start to analyze performance levels over time without sharing every event.
We would need the downstream handling to also deal with null records.
The Processor with Lua could then look like:
service:
log_level: debug
http_server: on
flush: 1
pipeline:
inputs:
- name: opentelemetry
listen: 0.0.0.0
port: 4317
processors:
traces:
- name: lua
call: modify
code: |
function modify(tag, timestamp, record)
sampleThreshold = 0.7
if (math.random() > sampleThreshold)
record = null
-- we're assuming that returning a null record would effectively be dropped
return 1, timestamp, record
end
outputs:
- name: stdout
match: "*"
The text was updated successfully, but these errors were encountered:
It would be very powerful if Lua could be used as a Processor not just for Logs, but also Traces and Metrics. This would allow users to address issues such as implementing sampling on traces (currently an option with https://opentelemetry.io/docs/concepts/sampling/)
Describe alternatives you've considered
Ability to convert a Trace or Metric to a log and back again—this would allow all of Fluent Bit's features to be leveraged.
Additional context
Unable to apply sampling or filtering of traces to create metrics for traces with outlier thresholds (e.g a span execution time greater than X) and to sample traces to reduce the traffic volumes being sent onwards. While processing multiple events if we can sample the the latency for events that aren't showing anomalies - we can start to analyze performance levels over time without sharing every event.
We would need the downstream handling to also deal with null records.
The Processor with Lua could then look like:
service:
log_level: debug
http_server: on
flush: 1
pipeline:
inputs:
- name: opentelemetry
listen: 0.0.0.0
port: 4317
processors:
traces:
- name: lua
call: modify
code: |
function modify(tag, timestamp, record)
sampleThreshold = 0.7
if (math.random() > sampleThreshold)
record = null
-- we're assuming that returning a null record would effectively be dropped
return 1, timestamp, record
end
outputs:
- name: stdout
match: "*"
The text was updated successfully, but these errors were encountered: