Security configurations

Configuring Flink for securing your workloads and the Cloudera SQL Stream Builder.

Cloudera SQL Stream Builder enables you to deploy the Flink SQL jobs in an easier, however you need to ensure the proper security of your workloads and Cloudera SQL Stream Builder. This means that the following tools are available for you to secure your FlinkDeployment in your Kubernetes cluster setup.

Fernet key

When deploying Cloudera SQL Stream Builder, you can specify a fernetKey, which is used to encrypt job definitions for Flink jobs started with Cloudera SQL Stream Builder. Job definitions may include sensitive data, such as connector credentials in table DDL.

The Helm chart creates a Kubernetes Secret from fernetKey only when the value is non-empty. The chart ships a placeholder string; clear ssb.fernetKey (set it to an empty string) before install so Cloudera SQL Stream Builder generates a unique key on first startup. When you provide a non-empty key, the Secret is created in the same namespace as Cloudera SQL Stream Builder and mounted by Cloudera SQL Stream Builder and Flink pods.

User management

When installing Cloudera SQL Stream Builder, a default user (admin/admin) is created automatically, and registration is enabled. You have the option to enable or disable user registration, and you can also modify the default user(s) based on your requirements.

LDAP authentication

To enable LDAP, create a secret with the parameters of the LDAP connection before starting helm install.

Example for secure setup with LDAP:

kubectl create secret generic ssb-ldap -n flink \
  --from-literal=SSB_LDAP_URL=ldaps://ldap.example.com:636 \
  --from-literal=SSB_LDAP_BASE_DN=dc=example,dc=com \
  --from-literal=SSB_LDAP_USERNAME=cn=admin,dc=example,dc=com \
  --from-literal=SSB_LDAP_PASSWORD=password \
  --from-literal=SSB_LDAP_USER_DN_PATTERNS=uid={0},ou=people \
  --from-literal=SSB_LDAP_USER_SEARCH_BASE= \
  --from-literal=SSB_LDAP_USER_SEARCH_FILTER= \
  --from-literal=SSB_LDAP_GROUP_SEARCH_BASE= \
  --from-literal=SSB_LDAP_GROUP_SEARCH_FILTER= \
  --from-file=ldap_truststore.jks=[*** YOUR PATH ***]/truststore.jks \
  --from-literal=SSB_LDAP_SSL_TRUSTSTORE_PASSWORD=changeit

In the values.yaml file, set ldap to true, specify your secret, and list the usernames for the admin users.

ssb:
  userManagement:
    type: ldap
    ldap:
      secure: true
      secretRef: ssb-ldap
      admins:
        - admin

Ingress

By default Cloudera SQL Stream Builder does not set up any Ingress. This can be changed using the ingress configuration. The Ingress resource is created in the same namespace as Cloudera SQL Stream Builder.

Ingress can be used to easily enable TLS/HTTPs to Cloudera SQL Stream Builder, but it can also be used to set up authentication. For more information see Ingress.

Gateway API HTTPRoute

Clusters that use the Kubernetes Gateway API can expose Cloudera SQL Stream Builder with an HTTPRoute instead of an Ingress. Configure ssb.httpRoute in values.yaml. When left empty, no HTTPRoute is created.

Example:

ssb:
  httpRoute:
    spec:
      parentRefs:
        - name: my-gateway
      hostnames:
        - ssb-sse.example.com
      rules:
        - matches:
            - path:
                type: PathPrefix
                value: /
          backendRefs:
            - name: ssb-sse
              port: 18121

See the Gateway API HTTPRoute specification for parent gateway, filter, and TLS options.

Storage configuration

It is recommended to add some kind of persistent data storage for Flink to be able to save checkpoints and savepoints. In most cases this is some kind of blob storage (for example, S3) that needs authentication to access.

You can use the storageConfiguration configuration to set up the storage for Cloudera SQL Stream Builder. The configuration should be a valid flink-conf.yaml file, which can contain sensitive data, such as s3.access-key, s3.secret-key,and so on.

The Helm chart creates a Secret in the same namespace as Cloudera SQL Stream Builder, and Cloudera SQL Stream Builder creates a new Secret for each new Flink deployment created by the user.

Volume mounts

It is possible to mount existing volumes to Cloudera SQL Stream Builder and all created Flink pods using the podVolumes and podVolumeMounts configurations. You need to ensure that these volumes exist in the namespace of the Cloudera SQL Stream Builder and Flink pods that will be created by SSB.

The configurations can be used to mount ConfigMaps, Secret, or any kind of volumes to the SSB and Flink pods. For example, you can mount hive-site.xml, core-site.xml, krb5.conf and some keytabs as ConfigMaps and Secrets to be able to connect to Hive with SSB/Flink.

You can also configure custom truststores using a Kubernetes Secret.

Example:

kubectl -n flink create secret generic custom-truststore  \
     --from-file=truststore.jks

In the values.yaml file add the configuration:

podVolumes:
  create: false
  data:
    - name: hadoop-conf-volume
      configMap:
        name: hadoop-conf
    - name: custom-truststore-volume
      secret:
        secretName: custom-truststore
  podVolumeMounts:
    create: false
    data:
    - name: hadoop-conf-volume
      mountPath: /etc/hadoop/conf
      readOnly: true
    - name: custom-truststore-volume
      mountPath: /opt/flink/tls/
      readOnly: true

If you mounted a custom truststore, you can reference it when creating a table from SSB. Provide the truststore properties according to the connector type.

An example for the Kafka connector:

CREATE TABLE `example_kafka` (
    `id` INT,
    `name` STRING
  ) WITH (
    'connector' = 'kafka',
    'topic' = 'example_topic',
    'properties.bootstrap.servers' = 'kafka-ssl.example.com:9092',
    'properties.security.protocol' = 'SSL',
    'properties.ssl.truststore.location' = '/opt/flink/tls/truststore.jks',
    'properties.ssl.truststore.password' = 'changeit'
  );

Kerberos configuration

To enable Kerberos authentication, you need to add the Hadoop dependencies to the Cloudera Streaming Analytics images as described in the Customize container images section.

After adding the dependencies, you need to ensure that the Hadoop configuration and krb5.conf files are added as a configmap using the following commands:

kubectl -n flink create configmap hadoop-conf --from-file core-site.xml=core-site.xml --from-file hdfs-site.xml=hdfs-site.xml

kubectl -n flink create configmap krb5-conf --from-file krb5.conf=krb5.conf

When the configmaps are in place, the following configuration properties should be updated in the values.yml file for the configuration files to be mounted on the containers:

ssb:
  podVolumes:
   create: true
   data:
   - name: hadoop-conf-volume
     configMap:
      name: hadoop-conf
   - name: krb5-conf-volume
     configMap:
      name: krb5-conf
  podVolumeMounts:
   create: true
   data:
   - name: hadoop-conf-volume
     mountPath: /etc/hadoop/conf
     readOnly: true
   - name: krb5-conf-volume
     mountPath: /etc/krb5.conf
     subPath: krb5.conf

After setting up images and configurations, you can use Streaming SQL Console to specify your keytabs in the Keytab Manager. Once the keytab is successfully validated, Kerberos will be automatically configured when a new job is deployed.