kafka-*-perf-test tool can be used in several ways. In general, it is
expected that these tools should be used on a test or development cluster.
kafka-*-perf-test tools allow you toÉ
- Measuring read and/or write throughput.
- Stress testing the cluster based on specific parameters (such as message size).
- Load testing for the purpose of evaluating specific metrics or determining the impact of cluster configuration changes.
kafka-producer-perf-test script can either create a randomly generated
kafka-producer-perf-test --topic TOPIC --record-size SIZE_IN_BYTES
or randomly read from a set of provided records:
kafka-producer-perf-test --topic TOPIC --payload-delimiter DELIMITER --payload-file INPUT_FILE
where the INPUT_FILE is a concatenated set of pre-generated messages
separated by DELIMITER. This script keeps producing messages or limited
based on the
kafka-consumer-perf-test --broker-list host1:port1,host2:port2,... --zookeeper zk1:port1,zk2:port2,... --topic TOPIC
The flags of most interest for this command are:
--group gid: If you run more than one instance of this test, you will want to set different ids for each instance.
--num-fetch-threads: Defaults to 1. Increase if higher throughput testing is needed.
--from-latest: To start consuming from the latest offset. May be needed for certain types of testing.