Producing data in Avro format
You can run the sample producer application to produce data to a Kafka topic in Avro format. You can send data to a Kafka topic in Avro format. Learn how to store a schema in CDPʼs Schema Registry and use a simple Java Kafka client to send and read data using that schema.
On your computer, clone this repository using
gitand change to the
git clone https://github.com/asdaraujo/cdp-examples cd cdp-examples/kafka-client-avro
Build the client binaries.
mvn clean package
Create a copy of the producer properties template file.
cp src/main/resources/producer.properties.template producer.properties
producer.propertiesfile and replace the following placeholders with the corresponding values:
The Machine User name (prefixed with
srv_). Note that this needs to be replaced at two locations in the template.
The Machine User’s Workload Password. Note that this needs to be replaced at two locations in the template.
The hostnames of three cluster brokers. Also, ensure that the broker port numbers match the numbers seen in the Streams Messaging Manager (SMM) Brokers page.
The path of the
truststore.jksfile created previously.
The Schema Registry endpoint found previously.
Run the producer application with the following
java \ -cp ./target/kafka-client-avro-1.0-SNAPSHOT.jar \ com.cloudera.examples.MachineDataProducer \ ./producer.properties \ ./src/main/avro/MachineData.v1.avscThe producer application (
MachineDataProducerclass) takes the following arguments:
- An Avro schema definition file that the producer uses to produce messages.
... INFO c.c.examples.MachineDataProducer - Successfully produced message to partition [machine-data-avro-8], offset 
Leave the SSH session open and the producer running for now.