Checking Mirroring Progress
You can use Kafka's Consumer Offset Checker command-line tool to assess how well your mirror is keeping up with the source cluster. The Consumer Offset Checker checks the number of messages read and written, and reports the lag for each consumer in a specified consumer group.
The following command runs the Consumer Offset Checker for group
KafkaMirror, topic test-topic. The
--zkconnect argument points to the ZooKeeper host and port on the
source cluster.
/usr/hdp/current/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group KafkaMirror --zkconnect source-cluster-zookeeper:2181 --topic test-topic Group Topic Pid Offset logSize Lag Owner KafkaMirror test-topic 0 5 5 0 none KafkaMirror test-topic 1 3 4 1 none KafkaMirror test-topic 2 6 9 3 none
Table 5.2. Consumer Offset Checker Options
--group | (Required) Specifies the consumer group. |
--zkconnect | Specifies the ZooKeeper connect string. The default is
localhost:2181. |
--broker-info | Lists broker information |
--help | Lists offset checker options. |
--topic | Specifies a comma-separated list of consumer topics. If you do not specify a topic, the offset checker will display information for all topics under the given consumer group. |