Cloudera Docs

Adding Catalogs

You need to add Schema Registry, Kudu, Hive or other services as a Catalog using the Streaming SQL Console in SQL Stream Builder (SSB) to use them with Flink DDL.

  • Make sure that you have required service on your cluster.
  • Make sure that you have the right permissions set in Ranger for SSB and the services.
  1. Navigate to the Streaming SQL Console.
    1. Go to your cluster in Cloudera Manager.
    2. Select SQL Stream Builder from the list of services.
    3. Click SQLStreamBuilder Console.
    The Streaming SQL Console opens in a new window.
  2. Click Data Providers from the main menu.
  3. Click Catalogs tab.
  4. Click Register Catalog.
    The Add Catalog window appears.
  5. Add a Name to your catalog.
  6. Select the Catalog Type from the following options:
    1. Select Schema Registry from the Catalog Type drop-down.
    2. Select the Kafka cluster you registered as Data Provider.
    3. Enable TLS, if needed for the communication.
      1. If you enabled TLS, provide the Schema Registry Truststore location and password to the SR TrustStore and SR TrustStore Password field.
    4. Add the Schema Registry URL.
      1. Go to your cluster in Cloudera Manager.
      2. Select Schema Registry from the list of services.
      3. Click on Instances.
      4. Copy the Hostname of Schema Registry.
      5. Add the default port of Schema Registry after the hostname.
        Example:
        http://docs-test-1.vpc.cloudera.com:7788/api/v1
    1. Select Kudu from the Catalog Type drop-down.
    2. Add the host URL of Kudu Masters.
      1. Go to your cluster in Cloudera Manager.
      2. Select Kudu from the list of services.
      3. Click on Instances.
      4. Copy the Hostname of the Master Default Group.
      5. Add the default port of Kudu after the hostname.
        Example:
        docs-test-1.vpc.cloudera.com:7051
    1. Select Hive from the Catalog Type drop-down.
    2. Provide a Name to the Hive catalog created in SSB.
    3. Provide the name of the Hive Default Database.
    1. Select Custom from the Catalog Type drop-down.
    2. Provide a Property Key.
    3. Proivde a Property Value.

      If needed, you can specify more custom properties by using the plus icon.

  7. Click on Add Filter.
    1. Provide a Database and Table filter if you want to select specific tables to use from the catalog.
  8. Click on Validate.
  9. If the validation is successful, click Create.
You are ready to use the added catalog in SSB with Flink DDL. The already existing schemas in Schema Registry, tables in Kudu and Hive are automatically imported to SSB.