Process status monitoring with fluent-bit

[BlaBlaGITTE]

Hi all,
Is there any way to monitor a Linux process (under systemd) using fluent-bit and create a kind of alert?

[patrick-stephens]

Alert depends on your monitoring system really, e.g. Grafana alerting, something in the OS, etc.
What specifically do you want to monitor? Log output or process health/metrics?

Systemd log input handling is provided: https://docs.fluentbit.io/manual/pipeline/inputs/systemd
This could be fed to for example Loki and alerts triggered for specific logs.

[BlaBlaGITTE]

for eg. inside /var/log/fluent-bit.log I see:
...
[2023/07/06 11:40:14] [ info] [output:http:http.0] thread worker #0 stopped
...

it means that fluent-bit client is stopped, can I create a sort of alarm using (for eg: Open-Search Dashboard or similar)?

thanks

[patrick-stephens]

So yeah, you can push logs from systemd to wherever you like and then create alerts on them - how you do it depends on the destination tool you are using. Log-based alerting is available in Grafana, I assume something similar is present in whatever tool you are using (e.g. Opensearch) but you'll need to check the specifics of that.

Refer to the docs for how to ingest from Systemd and then send via whatever output you want.