Skip to content

Commit

Permalink
Kafka observ lib (#1291)
Browse files Browse the repository at this point in the history
* Update grafonnet dependency in commonlib

* Add support for building tables in commonlib.

* Revert newline

* Update tests

* Fix tests

* Add tests

* Add kafka-observ-lib

* Drop lag and topics dashboards

* Add kafka topics overview dashboard in kafka-mixin (use kafka-observ-lib)

* Update readme

* Update legendFormat

* Add alert

* Fix stub

* Add consumerGroupLagTime signal

* Add KafkaLagIsTooHigh alert

* Fix typo in alert

* Add dynamic severity
  • Loading branch information
v-zhuravlev authored Jul 24, 2024
1 parent 148af6a commit caa17bd
Show file tree
Hide file tree
Showing 21 changed files with 501 additions and 1,679 deletions.
3 changes: 1 addition & 2 deletions kafka-mixin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,11 @@ The Kafka Mixin is a set of configurable, reusable, and extensible dashboards ba

This mixin includes the following dashboards:
Kafka Overview - Gives an overview of your Kafka cluster resource usage, throughput, and general healthiness
kafka-topics - Gives informations about the throughput of specific (filterable) topics
Kafka topics overview - Gives informations about the throughput of specific (filterable) topics and consumer groups (rate, lag)
Zookeeper Overview - Gives an overview of your Zookeeper nodes resource usage and general healthiness
Ksqldb Overview - Gives an overview of your ksqldb cluster as queries resource usage, throughput, and general healthiness
Connect Overview - Gives an overview of your Kafka Connect cluster and tasks resource usage, throughput, and general healthiness
Schema Registry Overview - Gives an overview of your Kafka Schema Registry cluster resource usage, throughput, and general healthiness
Kafka Lag Overview - Gives Lag metrics by time and offset count on topics and partitions of the cluster

To use them, you need to have `mixtool` and `jsonnetfmt` installed. If you have a working Go development environment, it's easiest to run the following:

Expand Down
1 change: 1 addition & 0 deletions kafka-mixin/config.libsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@
ksqldbFilteringSelector: 'job=~"integrations/kafka-ksqldb|integrations/kafka"',
groupLabels: ['job', 'kafka_cluster'],
instanceLabels: ['instance'],
dashboardTags: ['kafka-integration'],
},
}
2 changes: 0 additions & 2 deletions kafka-mixin/dashboards/dashboards.libsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,7 @@
grafanaDashboards+:: {
'connect-overview.json': (import 'connect-overview.json'),
'kafka-ksqldb-overview.json': (import 'kafka-ksqldb-overview.json'),
'kafka-lag-overview.json': (import 'kafka-lag-overview.json'),
'kafka-overview.json': (import 'kafka-overview.json'),
'kafka-topics.json': (import 'kafka-topics.json'),
'schema-registry-overview.json': (import 'schema-registry-overview.json'),
'zookeeper-overview.json': (import 'zookeeper-overview.json'),
},
Expand Down
Loading

0 comments on commit caa17bd

Please sign in to comment.