Installing Cloudera Flow Management - Kubernetes Operator (air-gap)

You can install Cloudera Flow Management - Kubernetes Operator after meeting all prerequisites and installing dependencies, either using the cfmctl CLI tool or Helm.

Complete these steps to install Cloudera Flow Management - Kubernetes Operator if your Kubernetes cluster does not have internet access, or if you want to install it from a self-hosted registry. Installing Cloudera Flow Management - Kubernetes Operator installs the applications and resources that enable you to deploy and manage NiFi and NiFi Registry in Kubernetes.

  1. Copy the installation artifacts to a local registry using the docker pull, docker tag, and docker push commands.
    docker pull container.repository.cloudera.com/cloudera/cfm-operator:[***OPERATOR VERSION***]
    
    docker tag container.repository.cloudera.com/cloudera/cfm-operator:[***OPERATOR VERSION***] [***PRIVATE REGISTRY[:PORT]/PATH/TAG:OPERATOR VERSION***]
    
    docker push [***PATH TO SELF-HOSTED REGISTRY***]/cfm-operator:[***OPERATOR VERSION***]
    
    For example:
    docker pull container.repository.cloudera.com/cloudera/cfm-operator:2.9.0-b96
    
    docker tag container.repository.cloudera.com/cloudera/cfm-operator:2.9.0-b96 us-central1-docker.pkg.dev/nifi/cfm-k8s/cfm-operator:2.9.0-b96
    
    docker push us-central1-docker.pkg.dev/nifi/cfm-k8s/cfm-operator:2.9.0-b96
    For more information on pulling, pushing, and tagging Docker images, see the Docker documentation.
  2. Create a namespace for the Cloudera Flow Management - Kubernetes Operator if it does not already exist.
    kubectl create namespace [***OPERATOR NAMESPACE***]
    Replace [***OPERATOR NAMESPACE***] with the desired namespace for Cloudera Flow Management - Kubernetes Operator.

    For example:

    $ kubectl create namespace cfm-operator-system
  3. Install cert-manager.

    Follow the instructions for installing the cert-manager Operator for RedHat OpenShift.

    helm install cert-manager jetstack/cert-manager \
    --version [***CERT MANAGER VERSION***]\
    --namespace cert-manager \
    --create-namespace \
    --set installCRDs=true
    

    Replace ***CERT MANAGER VERSION***] with the certificate manager version you want to install.

  4. Create a Kubernetes secret containing your Cloudera credentials.
    kubectl create secret docker-registry [***SECRET NAME***] \
      --namespace [***OPERATOR NAMESPACE***] \
      --docker-server [***CONTAINER REGISTRY***] \
      --docker-username [***USERNAME***] \
      --docker-password [***PASSWORD***]
    

    Replace:

    • [***SECRET NAME***] with the desired Kubernetes secret name.
    • [***USERNAME***] and [***PASSWORD***] with your internal registry credentials.

    • [***OPERATOR NAMESPACE***] with the Cloudera Flow Management - Kubernetes Operator installation namespace.

    • [***CONTAINER REGISTRY***] with your internal registry URL.

  5. Optional: Install the cfmctl CLI tool. While installing the tool is not strictly required for the operation of Cloudera Flow Management - Kubernetes Operator, it makes performing common tasks more convenient. The examples in this documentation make heavy use of the cfmctl CLI tool.

    The cfmctl tool allows you to:

    • Manage your environment

    • Check the current state and existence of prerequisites in an environment

    • Install and uninstall the operator

    • Quickstart install NiFi clusters

    • Perform common configuration tasks using flags, with the ability to provide a helmvalues.yaml file

    • Install using default image location without the need to provide it manually

    1. Copy the CLI tool version appropriate for your environment to the Cloudera Flow Management - Kubernetes Operator installation directory and run it.
    2. Make the tool executable.
      chmod +x [***CFMCTL FILE***]
      Replace [***CFMCTL FILE***] with the name of the executable file that you have downloaded.
  6. Install Cloudera Flow Management - Kubernetes Operator.
    Install Cloudera Flow Management - Kubernetes Operator using the cfmctl install command:
    ./cfmctl install --license [***LICENSE***] \
    --image-repository "[***IMAGE REPOSITORY***]" \
    --image-tag "[***OPERATOR VERSION***]" \
    –values [***VALUES.YAML***] \
    --namespace [***OPERATOR NAMESPACE***]

    Replace

    • [***LICENSE***] with the license file. This flag is mandatory.

    • [***IMAGE REPOSITORY***] Defaults to “container.repository.cloudera.com/cloudera/cfm-operator” unless a Helm values.yaml is provided. This flag is optional.

    • [***OPERATOR VERSION***] Defaults to “latest” unless a Helm values.yaml is provided. This flag is optional.

    • [***VALUES.YAML***] with a Helm values.yaml file to supply any variables to the underlying Helm chart that is not available through cfmctl command flags. This flag is optional.

    • [***OPERATOR NAMESPACE***] with the desired operator installation namespace. Defaults to "cfm-operator-system".

    This command installs the CustomResourceDefinitions and Helm chart for the operator, and starts the operator.

    $ ./cfmctl install --license ./license.txt --image-repository "container.repository.cloudera.com/cloudera/cfm-operator" --image-tag "2.8.0-b94"
    
    2024-06-11T21:22:19.678+0200	INFO	cli.install	cmd/install.go:90	installing chart	{"namespace": "cfm-operator-system"}
    2024-06-11T21:22:23.820+0200	INFO	cli.install.helmclient	cmd/install.go:162	creating 1 resource(s)
    2024-06-11T21:22:24.601+0200	INFO	cli.install.helmclient	cmd/install.go:162	creating 18 resource(s)
    2024-06-11T21:22:26.063+0200	INFO	cli.install.helmclient	cmd/install.go:162	beginning wait for 18 resources with timeout of 10m0s
    2024-06-11T21:22:26.697+0200	INFO	cli.install.helmclient	cmd/install.go:162	Deployment is not ready: cfm-operator-system/cfm-operator. 0 out of 1 expected pods are ready
    …
    2024-06-11T21:24:28.414+0200	INFO	cli.install.helmclient	cmd/install.go:162	release installed successfully: cfm-operator/cfm-operator-0.0.0-dev
    1. Create your license secret.
      kubectl create secret generic cfm-operator-license --from-file=license.txt=[***PATH/TO/LICENSE.TXT***] -n [***OPERATOR NAMESPACE***]
      Replace
      • [***PATH/TO/LICENSE.TXT***] with the relative path to the license file
      • [***OPERATOR NAMESPACE***] with the namespace where you install Cloudera Flow Management - Kubernetes Operator
    2. Run Helm install.
      helm install cfm-operator [***PATH TO OPERATOR HELM CHART***] \
             --create-namespace \
             --namespace [***OPERATOR NAMESPACE***] \
             --set installCRDs=true \
             --set image.repository=[***IMAGE REPOSITORY***] \
             --set image.tag=[***OPERATOR VERSION***] \
             --set licenseSecret=cfm-operator-license
      
      Replace
      • [***PATH TO OPERATOR HELM CHART***] with the path to the downloaded and unpacked Cloudera Flow Management - Kubernetes Operator Helm chart, for example,
        ./charts/cfm-operator
      • [***OPERATOR NAMESPACE***] with the desired installation namespace, for example,
        cfm-operator-system
      • [***IMAGE REPOSITORY***] with the Cloudera Flow Management - Kubernetes Operator image repository.
        • If you install from the Cloudera Docker Registry, replace it with
          container.repository.cloudera.com/cloudera/cfm-operator
        • If you install from a self-hosted private registry, replace it with your internal registry URL.
      • [***OPERATOR VERSION***] with your desired Cloudera Flow Management - Kubernetes Operator version, for example,
        2.9.0
  7. Validate your installation.
    1. Check if CustomResourceDefinitions for NiFi were installed or updated:
      kubectl get crds | grep nifi
      Expect a similar output:
      
      nifiregistries.cfm.cloudera.com 2024-01-25T21:31:28Z
      nifis.cfm.cloudera.com 2024-01-25T21:31:29Z
    2. Check if a Cloudera Flow Management - Kubernetes Operator pod is up and running:
      kubectl get pods -n cfm-operator-system
      Expect a similar output:
      
      NAME                            READY   STATUS    RESTARTS   AGE
      cfm-operator-545bfbc96b-sx4jt   2/2     Running   0          18m
      

With the operator installed and running, you can create and manage instances of NiFi and NiFi Registry by manipulating the Kubernetes object definitions.