Cloudera Docs

CFM installation workflow

Before you start installing Cloudera Flow Management (CFM) on a CDP Private Cloud cluster, it is helpful to understand the steps involved. There are three main installation scenarios depending on the type of cluster or clusters on which you install the different components.

The Cloudera Flow Management (CFM) installation workflow consists of preparing your CDP Private Cloud Base cluster and then installing NiFi and NiFi Registry on a Base or on a Compute cluster, or on a mixed cluster layout. See the following diagram for the procedure steps, and review the description of each deployment scenario following the diagram.

Link to documentation for preparing your CDP Private Cloud BaseLink to documentation for installing NiFi and NiFi Registry on a Base clusterLink to documentation for installing NiFi and NiFi Registry on a Compute clusterLink to documentation for installing NiFi on a Compute cluster and NiFi Registry on a Base cluster

Scenario 1: NiFi and NiFi Registry on a Base cluster

Installing NiFi and NiFi Registry on your Base cluster is a simplified deployment during which you create only one cluster. You can do this if you are setting up in trial deployments or in simple production scenarios.

A Base cluster is a large cluster that contains SDX services like Ranger and Atlas, as well as compute and storage services like NiFi, Impala, and Hive. It is sometimes called a Regular cluster in Cloudera Manager, and it is also known as an SDX cluster, a Shared cluster, or a Data Lake cluster.

You can install NiFi Registry on the Base cluster. If you are using multiple NiFi services, you would likely want to share the NiFi Registry instance across all of these NiFi services.

In simplified deployment scenarios, you can also install NiFi on the Base cluster to achieve a single-cluster setup. With this scenario, you may have many services on the Base cluster relying on the same Zookeeper quorum, which may not suit your use case depending on the expected workload.

For information on the installation steps, see Installing NiFi and NiFi Registry on your Base cluster.

When you install both NiFi and NiFi Registry on your Base cluster, your cluster layout will look similar to the following:

Scenario 2: NiFi and NiFi Registry on a Compute cluster

Although it is not a recommended setup, it is possible to install both NiFi and NiFi Registry on a Compute cluster.

A Compute cluster consists of compute nodes only. The Compute cluster is then connected to a Base cluster with SDX services like Ranger and Atlas using a Shared Data Context.

Installing NiFi on a Compute cluster is recommended when you want to:
  • deploy multiple NiFi clusters for larger scale production deployments
  • isolate your dataflows
  • have different authorization models per dataflow
and in other similar deployment scenarios.

For information on the installation steps, see Installing NiFi and NiFi Registry on a Compute cluster.

When you install both NiFi and NiFi Registry on a Compute cluster, your cluster layout will look similar to the following:

Scenario 3: NiFi on a Compute cluster and NiFi Registry on a Base cluster

For production deployments, it is recommended to install NiFi on one or more Compute clusters, and to install NiFi Registry on the Base cluster. This provides a dedicated Zookeeper quorum for each NiFi cluster. Use this setup when you want to share dataflows across multiple NiFi Compute clusters in your CDP environment.

For information on the installation steps, see Installing NiFi on a Compute cluster and NiFi Registry on a Base cluster.

When you install NiFi on one or more Compute clusters and NiFi Registry on the Base cluster, your cluster layout will look similar to the following: