Cloudera Docs

Adding custom DDL templates and connectors

You can add custom templates for the existing catalogs, and also add new custom templates and connector to SQL Stream Builder to further customize the SQL Stream job for your production environment.

You can customize SSB based on your requirements with the following available scenarios for Flink DDL:
  • Add custom DDL templates for default connectors
  • Add custom DDL templates with custom connectors
  • Add custom connectors to use in SSB
You can specify the new template by uploading the template file to the User Defined DDL Templates directory. If you need to add a custom connector that can be used with the template, you need to upload the corresponding connector jar file to the Flink SQL Connector Jar directory. Both directories can be configured through the SQL Stream Builder service in Cloudera Manager.
  1. Go to your cluster in Cloudera Manager.
  2. Click SQL Stream Builder from the list of services.
  3. Select Configuration tab.
  4. Search for User Defined DDL Templates Directory and Flink SQL Connector Jar Directory configurations.
    You need to upload the custom template and connector to these directories.
  5. Check the predefined value for the configurations.
  6. Connect to the host with ssh where the Streaming SQL Console is running.
    For example:
    ssh root@docstest-1.vpc.cloudera.com
  7. Upload the custom template and the connector jar to the corresponding directories using the command line tool.
    For example:
    scp path/flink-faker-template root@docstest-1.vpc.cloudera.com:.
  8. Review that the template and connector file is in the directory.
    For example:
    ll /usr/share/flink-ddl-templates/
    If you see the template and connector file in the directories, you must restart the Streaming SQL Console service.
  9. Click on Instances tab.
  10. Check the Streaming SQL Console role type checkbox.
  11. Click on Actions > Restart.
    After the Streaming SQL Console service restarted, the new template and connector should be available for Flink DDL.
  12. Click on SQLStreamBuilder Console.
  13. Click on Tables from the main tabs.
  14. Select Add table > Flink DDL.
    The Flink DDL window appears.
  15. Click on Available templates drop-down menu.
    The custom template is listed after the predefined templates.