Installing Cloudera Streaming Analytics - Kubernetes Operator in an environment with internet access

Complete these steps to install Cloudera Streaming Analytics - Kubernetes Operator if your Kubernetes cluster has internet access. Installing the Cloudera Streaming Analytics - Kubernetes Operator enables you to deploy and manage Flink and Cloudera SQL Stream Builder in Kubernetes.

Cloudera Streaming Analytics - Kubernetes Operator is installed in your Kubernetes cluster with the provided Helm chart through the helm install command. When you install the chart, Helm installs the Custom Resource Descriptors (CRD) included in Cloudera Streaming Analytics - Kubernetes Operator, and deploys the Apache Kubernetes Flink Operator (Flink Operator), Cloudera SQL Stream Builder engine (in Technical Preview), and a PostgreSQL database for Cloudera SQL Stream Builder.

Installing Cloudera Streaming Analytics - Kubernetes Operator does not create or deploy a Flink cluster. The Flink cluster is created after the installation by deploying the Flink Deployment resource in the Kubernetes cluster with kubectl or oc, or when you execute a SQL job in Streaming SQL Console.

By default the Flink Operator (deployed with installation) watches and manages all the Flink clusters that are deployed in the single same namespace as the Flink Operator. However, you can also configure it to watch and manage multiple namespaces. This allows you to manage multiple Flink clusters deployed in different namespaces, using a single Cloudera Streaming Analytics - Kubernetes Operator installation.
  • Ensure that your Kubernetes environment meets requirements listed in System requirements.
  • Your Kubernetes cluster requires internet connectivity to complete these steps, as it must be able to reach the Cloudera Docker registry.
  • Ensure that you have access to a valid Cloudera license.
  • Ensure that you have access to your Cloudera credentials (username and password, that are provided together with the Cloudera license) required to access the Cloudera Docker registry (and, if needed, the Cloudera Archive), where installation artifacts are hosted.
  • Review the Helm chart reference before installation.

    The Helm chart accepts various configuration properties that you can set during installation. You can customize your installation using these properties.

  • If you want to use the Webhook of Flink Operator, ensure that you have cert-manager installed on your Kubernetes cluster, which you can install using the following command:
    kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.8.2/cert-manager.yaml
    kubectl wait -n cert-manager --for=condition=Available deployment --all
    • The webhook functionality is enabled by default. You can disable it using the following command, and skip the cert-manager installation:
      --set flink-kubernetes-operator.webhook.create=false
  1. Create a namespace in your Kubernetes cluster where you will install and use the Cloudera Streaming Analytics - Kubernetes Operator.
    kubectl create namespace [***NAMESPACE***]
    This is the namespace where you install Flink and Cloudera SQL Stream Builder. Use this namespace you create in all installation steps that follow.
  2. Create a Kubernetes secret to contain your Cloudera credentials.
    kubectl create secret docker-registry [***SECRET NAME***] \
        --docker-server container.repository.cloudera.com \
        --docker-username [***USERNAME***] \
        --docker-password [***PASSWORD***] \
        --namespace [***NAMESPACE***]
    Ensure that the placeholders are replaced with your specific information:
    1. Provide a desired name for [***SECRET NAME***].
    2. Replace [***USERNAME***] and [***PASSWORD***] with your Cloudera credentials.
    3. Provide the same name for [***NAMESPACE***] that you created in the previous step.
  3. Log in to Cloudera Docker registry with helm.
    helm registry login container.repository.cloudera.com
    Enter your Cloudera credentials when prompted.
  4. Install Cloudera Streaming Analytics - Kubernetes Operator with helm install.
    helm install csa-operator --namespace [***NAMESPACE***] \
        --set 'flink-kubernetes-operator.imagePullSecrets[0].name=[***SECRET NAME***]' \
        --set 'ssb.sse.image.imagePullSecrets[0].name=[***SECRET NAME***]' \
        --set 'ssb.sqlRunner.image.imagePullSecrets[0].name=[***SECRET NAME***]' \
        --set-file flink-kubernetes-operator.clouderaLicense.fileContent=[***PATH TO LICENSE FILE***] \
    oci://container.repository.cloudera.com/cloudera-helm/csa-operator/csa-operator --version 1.1.2-b17
    Ensure that the placeholders are replaced with your specific information:
    1. Provide the same name for [***NAMESPACE***] that you created in Step 1.
    2. Provide the same name for [***SECRET NAME***] that you created in the previous step. imagePullSecrets specifies what secret is used to pull images from the Cloudera registry. Setting this property is mandatory, otherwise Helm will be unable to pull the necessary images from the Cloudera Docker registry.
    3. Replace [***PATH TO LICENSE FILE***] with the full (absolute) path to your Cloudera license file. clouderaLicense.fileContent is used to register your license. When this property is set, a secret is generated that contains the license you specify. Setting this property is mandatory. The Cloudera Streaming Analytics - Kubernetes Operator will not function without a valid license.
    4. You can use --set to set various other properties of the Helm chart. This enables you to customize your installation. For example, by default the Flink Operator has access to watch all namespaces. However, you can configure a list of specific namespaces to watch using watchNamespaces.. For example, in case you created multiple namespaces, you can configure the Flink Operator to only watch specific ones with --set flink-kubernetes-operator.watchNamespaces={[***NAMESPACE1***], [***NAMESPACE2***]}. For more information about deploying Flink and SSB in multiple namespaces, see the Namespace management documentation.
  5. Check that the Flink Operator, and the Cloudera SQL Stream Builder engine with its database are running.
    kubectl get pods -n [***NAMESPACE***]
    NAME                            READY   STATUS      RESTARTS    AGE
    flink-kubernetes-operator       1/2     Running      0          7s
    ssb-postgresql                  1/1     Running      0          7s
    ssb-sse                         1/1     Running      0          7s
After successfully installing the Cloudera Streaming Analytics - Kubernetes Operator, you can start using Flink and Cloudera SQL Stream Builder (in Technical Preview) on Kubernetes. The Getting Started with Flink and Getting Started with Cloudera SQL Stream Builder guides can help you with the basic operations.