Creating Flink tables using Templates
You can use the predefined templates to create tables by choosing one of the connector templates on the Console page of Streaming SQL Console. The Flink SQL templates are predefined examples of CREATE TABLE statements which you can fill out with your job specific values.
You can create tables using the predefined templates in SQL Stream Builder. The
predefined templates consist of the
CREATE TABLE statement, and every
connection property that is needed for the given connector. You only need to fill out
the templates and execute them in the Streaming SQL Console to create the table. When
filling out the templates, you can add configuration based on what is specified in the
given connector template. You can also customize the table name, column names and
timestamp information for a template.
You can choose from the following templates based on the connector type:
- The BlackHole connector can be used to write all input records into. It is designed for high performance testing and UDF to output, not a substantive sink.
- The Data generator connector can be used to sample randomly generated data in SSB. You can use this connector to try out and test SQL queries as records are generated until the job is running.
- The Faker connector can be used to generate fake data based on the Java faker expression. You can use this connector to try out and test SQL queries as records are generated until the job is running.
- The filesystem connector can be used to access partitioned files in file
systems, and enables reading and writing from a local or distributed file system
such as local, HDFS, S3 and so on. You only need to specify the data format that
is used in the file system. You can choose from the following formats:
- The JDBC connector enables reading data from and writing data into any relational databases with a JDBC driver. You can use PostgreSQL, MySQL and Hive as databases for the connector.
- Debezium CDC
- You can use the Debezium CDC connector to stream changes in real-time from MySQL, PostgreSQL, Oracle into Kafka. Debezium provides a unified format schema for changelog and supports serializing messages using JSON and Avro.
- Go to your cluster in Cloudera Manager.
- Click on SQL Stream Builder from the list of Services.
- Click on the SQLStreamBuilder Console.
You are redirected to the Streaming SQL Console.
- Select Console from the main menu.
- Click Templates under the SQL window.
- Select the template you want to use.
The template is imported to the SQL window.
- Customize the fields of the template.
- Click Execute.
The table is created based on the selected template. You can review the table using the Table tab.
For full reference on the Flink SQL DDL functionality, see the official Apache Flink documentation.