Masking information before using source control

When exporting projects from SQL Stream Builder, possible sensitive parameters in CREATE TABLE statements, data sources, or the SQL job will not be redacted. To conceal sensitive information, environment variables can be used.

  1. Navigate to the Streaming SQL Console.
    1. Navigate to Management Console > Environments, and select the environment where you have created your cluster.
    2. Select the Streaming Analytics cluster from the list of Data Hub clusters.
    3. Select Streaming SQL Console from the list of services.
      The Streaming SQL Console opens in a new window.
  2. Open a project from the Projects page of Streaming SQL Console.
    1. Select an already existing project from the list by clicking the Open button or Switch button.
    2. Create a new project by clicking the New Project button.
    3. Import a project by clicking the Import button.
    You are redirected to the Explorer view of the project.
  3. Click Environments from the Project Manager.
  4. Click New Environment.
  5. Provide a Name for the environment.
    For example, you can name it Dev.
  6. Add keys and values as needed.
    For example, you can add psql_password as key, and provide the password as value.
  7. Click Activate to apply the specified information for the project.
  8. When creating the table, reference the environment variable as shown in the following example:
    CREATE TABLE  `postgres_cdc_table_1682585103` (
     `order_id` INT,
     `city` VARCHAR(2147483647),
     `street_address` VARCHAR(2147483647),
     `amount` INT,
     `order_time` TIMESTAMP(3),
     `order_status` VARCHAR(2147483647)
    ) WITH (
     'connector' = 'postgres-cdc', 
     'password' = '${ssb.env.psql_password}' 
     'username' = '...'