Skip to content

Latest commit

 

History

History
65 lines (48 loc) · 2.45 KB

kafka-quotas.md

File metadata and controls

65 lines (48 loc) · 2.45 KB

Quotas for Kafka

Introduction:

Quotas regulate the traffic volume of clients. Quotas define the limit of throughput.

Motivation:

To pretend that clients monopolize broker resources and slow down other clients. High traffic can cause to a DoS behaviour.

Order of Quotas

smaller number hits higher number.
The order of precedence for quota configuration is:
Stored in Zookeeper under: # Means:
1./config/users//clients/ # Combination of user and client ID
2./config/users//clients/ # ?
3./config/users/ # Only User quota
4./config/users//clients/ #
5./config/users//clients/ # ?
6./config/users/ # User default quota
7./config/clients/ # Only ClientID
8./config/clients/ # ? Client ID default

More details:
quotas design-quotas KIP-55

Setting Quotas Kafka

Set default quotas:

To set for all clients or users default quotas.
command: kafka-configs --zookeeper localhost:2181 --alter --add-config 'producer_byte_rate= {rate}, consumer_byte_rate={rate}' --entity-type {entity-type} --entity-default
example: kafka-configs --zookeeper localhost:2181 --alter --add-config 'producer_byte_rate=1024, consumer_byte_rate=2048' --entity-type clients --entity-default

options:
For user add: --entity-type users
For clients add: --entity-type clients

Custom Quotas:

command: kafka-configs --zookeeper localhost:2181 --alter --add-config 'producer_byte_rate={ }, consumer_byte_rate={ } --entity-type users --entity-name {username} --entity-type clients --entity-name {username}
example:

Created topic "topic_name".

List and describe Quotas

command: kafka-configs.sh --zookeeper localhost:2181 --describe
example:

TODO

options:
Use entity-type users to list all or a specific user
--entity-type users --entity-name {specific_user_name} #Optional

Use entity-type clients to list all or a specific client
--entity-type clients --entity-name specific_client_id #Optional