Plugin: how to get callback from S3 output plugin(s)

[cdancy]

Hey All,

We have a use-case where right now, after fluentbit sends logs to S3, we have an AWS lambda which fires and does some things based on what files get uploaded. These is leading to millions of lambda invocations daily which is no good. I started thinking that maybe we could just write a fluentbit plugin that, after each of our S3 output plugins kicks, it would then "do what the lambda did" which is just interacting with AWS dynamodb.

Is something like that possible with the plugin architecture? Meaning for potentially X number of S3 output plugins defined we could get something like a callback so that we could in turn fire off a dynamodb request after the fact? Any other ideas in this regard?

Thanks ahead of time,
Chris