After registering a Kafka data provider, you can use the Add table wizard in
Streaming SQL Console to create a Kafka table.
- Make sure that you have registered Kafka as a Data Provider.
- Make sure that you have created topics in Kafka.
- Make sure there is generated data in the Kafka topic.
- Make sure that you have the right permissions set in Ranger.
-
Navigate to the Streaming SQL Console.
-
Go to your cluster in Cloudera Manager.
-
Select SQL Stream Builder from the list of services.
-
Click .
The Streaming SQL Console opens in a new window.
-
Click Create Job or select a previous job on the
Getting Started page.
You are redirected to the Console page.
-
Click .
The Kafka Table window appears.
-
Provide a Table Name.
-
Select a registered Kafka provider as Kafka
cluster.
-
Select the Data format.
- You can select JSON as data format.
- You can select AVRO as data format.
-
Select a Kafka topic from the list.
-
Determine the Schema for the Kafka table.
-
Add a customized schema to the Schema Definition
field.
-
Click Detect Schema to read a sample of the JSON
messages and automatically infer the schema.
-
Customize your Kafka Table with the following options:
-
Configure the Event Time if you do not want to
use the default Kafka Timestamps.
- Disable the Use Kafka Timestamps
feature.
- Provide the name of the Input Timestamp
Column.
- Add a name for the Event Time
Column.
- Add a value to the Watermark
Seconds.
-
Configure an Input Transform on the Data
Transformations.
-
Configure any Kafka properties required on the
Properties tab.
-
Select a policy for deserialization errors on the
Deserialization tab.
For more information about how to configure the Kafka table, see the
Configuring Kafka tables section.
-
Click Create and Review.
The Kafka Table is ready to be used for the SQL job either
at the FROM
or at the INSERT INTO
statements.