You can also remove nodes from your NiFi cluster for cost optimization. For example,
scale down the NiFi cluster when there is less data to be processed in order to save
infrastructure costs.
You have a NiFi flow which is running.
Log into CDP Public Cloud UI.
Go to Management Console > Environments.
Select the datahub cluster where your NiFi flow is running.
Before scaling down you can go to the NiFi UI by clicking
NiFi to check the current number of NiFi nodes
you have in your NiFi cluster, as shown in the following image:
Click Actions, and select
Resize.
The Cluster Resize window appears.
Select your hostgroup from the dropdown, decrease the number of nodes by
clicking the - icon, and then click
Resize.
It takes some time for the change to take place. In the background,
the following processes occur:
The nodes to be removed are disconnected from the NiFi cluster.
Processing of data on these nodes is stopped.
The data located on the nodes to be removed are moved to the
remaining nodes of the cluster. This ensures no data loss during a
scale down event.
The NiFi service is stopped on these nodes
The underlying virtual machines are stopped and deleted.
You can go to the NiFi UI to check that the nodes are successfully removed
from the NiFi cluster:
The scale down event is complete only after the nodes are no longer
listed in the Hardware tab of the Data Hub
Clusters page, as shown in the following image:
You have now successfully scaled down your NiFi cluster.