Learn how you can use the
kafka-console-producer tool to produce
messages to a topic.
This tool is used to write messages to a topic in a text based format.
The following examples demonstrate the basic usage of the tool. In addition to reviewing these
examples, you can also use the
--help option to see a list of all available
Produce messages to a topic
To start producing message to a topic, you need to run the tool and specify a server as well as a topic.
kafka-console-producer --broker-list [HOST1:PORT1] --topic [TOPIC]
>my first message >my second message
cat [FILE] | kafka-console-producer --broker-list [HOST1:PORT1] --topic [TOPIC]
Produce messages to a topic in a secure cluster
In order to use the tool on a secure cluster, additional client configuration is required.
You do this by creating a
.properties file that contains the necessary
configurations to run the tool on a secure cluster. Which properties are configured in this
file depends on the security configuration of your cluster. See the client configuration
topics in Securing Apache Kafka to learn more about which exact properties you need
to use for your security configuration. Once the file is created, you can use
--producer.config to pass the file to the tool.
security.protocol = SASL_SSL sasl.mechanism=GSSAPI sasl.kerberos.service.name = kafka ssl.truststore.location = /var/private/ssl/kafka.client.truststore.jks ssl.truststore.password = test1234 sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true;
This example shows what properties you have to set when both Kerberos and TLS/SSL are configured.
- Create a
Use the example above. Make changes as necessary.
- Run the tool with the
kafka-console-producer --broker-list [HOST1:PORT1] --topic [TOPIC] --producer.config client.properties
Define a key-value delimiter
It is possible to define a key-value delimiter for the given producer instance. The
delimiter can vary each time you run the tool. For example, if you run the tool with the
delimiter set to
- and then a second time using
will know how to store the data. The first term is always stored as the key, the second as
the value. As a result, when the data written this way is consumed, the keys and values are
consumed irrespective of what delimiter was used when the records were written to the topic.
To produce messages with key-value delimiters, you need to set two properties,
key.seperator. Both properties can be set
kafka-console-producer --broker-list [HOST1:PORT1] --topic [TOPIC] --property parse.key=true --property key.separator=":"
Configure retry backoff
kafka-console-producer --broker-list [HOST1:PORT1] --topic [TOPIC] --retry-backoff-ms 1000