Scaling down a NiFi cluster

You can also remove nodes from your NiFi cluster for cost optimization. For example, scale down the NiFi cluster when there is less data to be processed in order to save infrastructure costs.

You have a NiFi flow which is running.

Log into CDP Public Cloud UI.
Go to Management Console > Environments.
Select the datahub cluster where your NiFi flow is running.

Before scaling down you can go to the NiFi UI by clicking NiFi to check the current number of NiFi nodes you have in your NiFi cluster, as shown in the following image:
Click Actions, and select Resize.
The Cluster Resize window appears.
Select your hostgroup from the dropdown, decrease the number of nodes by clicking the - icon, and then click Resize.

note
You need to select the hostgroup carefully. For light duty templates, the hostgroup is nifi_scaling. For heavy duty templates, the hostgroup is nifi.
It takes some time for the change to take place. In the background, the following processes occur:
1. The nodes to be removed are disconnected from the NiFi cluster.
2. Processing of data on these nodes is stopped.
3. The data located on the nodes to be removed are moved to the remaining nodes of the cluster. This ensures no data loss during a scale down event.
4. The NiFi service is stopped on these nodes
5. The underlying virtual machines are stopped and deleted.
You can go to the NiFi UI to check that the nodes are successfully removed from the NiFi cluster:

The scale down event is complete only after the nodes are no longer listed in the Hardware tab of the Data Hub Clusters page, as shown in the following image:

You have now successfully scaled down your NiFi cluster.