kafka-*-perf-test
The kafka-*-perf-test
tools can be used in several ways. In general, it is
expected that these tools should be used on a test or development cluster.
The kafka-*-perf-test
tools allow you to:
- Measure, read, and write throughput.
- Stress test the cluster based on specific parameters (such as message size).
- Load test for the purpose of evaluating specific metrics or determining the impact of cluster configuration changes.
The kafka-producer-perf-test
script can either create a randomly generated
byte record:
kafka-producer-perf-test --topic TOPIC --record-size SIZE_IN_BYTES
or randomly read from a set of provided records:
kafka-producer-perf-test --topic TOPIC --payload-delimiter DELIMITER --payload-file INPUT_FILE
where the INPUT_FILE is a concatenated set of pre-generated messages
separated by DELIMITER. This script keeps producing messages or limited
based on the --num-records
flag.
The kafka-consumer-perf-test
is:
kafka-consumer-perf-test --broker-list host1:port1,host2:port2,... --zookeeper zk1:port1,zk2:port2,... --topic TOPIC
The flags of most interest for this command are:
--group gid
: If you run more than one instance of this test, you will want to set different ids for each instance.--num-fetch-threads
: Defaults to 1. Increase if higher throughput testing is needed.--from-latest
: To start consuming from the latest offset. May be needed for certain types of testing.