Creating a Kafka sink

You can use the registered Kafka source to create a Kafka Virtual Table Sink. You only need to specify the Kafka topic where the data should be generated when executing your SQL Stream job.

Make sure that you have created a Kafka topic.
important
When creating the topic for the Kafka sink, make sure to not use log compaction as it can cause the SQL job to fail.
Make sure that you have the right permissions set in Ranger.

Go to your cluster in Cloudera Manager.
Click on SQL Stream Builder from the list of Services.
Click on SQLStreamBuilder Console.
The Streaming SQL Console opens up in a new window.
Select Console from the main menu.
Select the Virtual Tables tab.
Select the Virtual Table Sink sub-tab.
Select Add Sink > Apache Kafka.
The Kafka Sink window appears.
Provide a name to the Virtual Table.

note
You will select the provided name when adding a sink on the Compose tab.
Select a Kafka topic from the list.

note
The automatically created topics for the websocket output is also listed here. Select the topic you want to use for the SQL job.
Click Save Changes.

note

All output is JSON encoded data in {column>:<value>} format.

The Kafka Virtual Table Sink is ready for queries to be run. You can add the created sink on the Compose tab by selecting its name from the list of sinks.