Monitor Kafka Consumer groups in Zabbix using Python
Information about Kafka Consumers groups and consumers LAG are retrieved using Kafka API. ( Kafka API Documentation )
This monitoring tool is working for Kafka Broker version > 0.9.0 and consumers that are using Consumer API which are committing the offset into Kafka.
Install kafka-python
pip install kafka-pythonZabbix Consumer groups discovery
python monitor_consumers.py --bootstrap-server <kafka_broker_list> --list Check if a consumer group exists
python monitor_consumers.py --bootstrap-server <kafka-broker-list> --group <consumer-group> --is-presentGet Lag for a specific consumer group
python monitor_consumers.py --bootstrap-server <kafka-broker-list> --group <consumer-group> --lagGet number of consumers from a group
python monitor_consumers.py --bootstrap-server <kafka-broker-list> --group <consumer-group> --membersMonitor Kafka Consumer Groups optional arguments: -h, --help show this help message and exit --bootstrap-server BOOTSTRAP_SERVER Kafka brokers list --list List Kafka Consumer Groups --group CONSUMER_GROUP --lag Lag consumer group --members Members consumer group --is-present Check if a consumer group exists --timeout TIMEOUT Kafka API timeout in ms --security-protocol Protocol used to communicate with brokers --sasl-mechanism SASL mechanism type --sasl-plain-username Username for SASL PLAIN authentication --sasl-plain-password Password for SASL PLAIN authentication --ssl-certfile Certificate file --ssl-keyfile Key file --nocheckcertificate Desactivate the verification of other peers certificates and hostname server