Scaling up a NiFi cluster

This guide explains how to add new nodes to your existing NiFi cluster to increase throughput.

Ensure your NiFi flow is running and stable before starting the scale-up process.

Log into the CDP Public Cloud UI.
Go to Management Console > Environments.
Select the Data Hub cluster where your NiFi flow is running.
Optional: Open the NiFi UI by clicking the NiFi link to review the current number of nodes in your cluster.
In the CDP UI, click Actions > Resize.

The Cluster Resize window appears.
Select the appropriate hostgroup from the dropdown, increase the number of nodes by clicking the + icon, and then click Resize.
- Use nifi_scaling for light-duty templates.
- Use nifi for heavy-duty templates.
Increase the number of nodes by clicking the + icon, then click Resize.

The scaling operation will begin and may take several minutes. Monitor the NiFi UI to confirm when the new nodes have joined the cluster.

important

NiFi allows great flexibility in flow design, but custom components such as manually added or hotfix-applied NAR files are not automatically propagated to the newly added nodes during a cluster scale-up. You need to manually transfer these NAR files.
Manually copy any custom NAR files from existing nodes to each newly added node to ensure consistent functionality across the cluster.
Restart NiFi on the new nodes if necessary to load the custom components properly.
Verify the new nodes have the correct NAR files and are functioning as expected.

You have now successfully scaled up your NiFi cluster.