Scaling Streams Messaging clustersPDF version

Scaling KRaft

Learn how to scale KRaft nodes (KRaft Controllers) in a Streams Messaging cluster.

Streams Messaging clusters that you provision with any of the default Streams Messaging cluster definitions may include optional KRaft nodes. KRaft nodes house the KRaft Controller roles. If your cluster includes KRaft nodes, the Kafka service on the cluster is running KRaft mode and uses KRaft for metadata management instead of ZooKeeper. The number of KRaft nodes in the cluster can be scaled. This is done by adding nodes to or removing nodes from the KRaft Nodes host group using the Resize action in Data Hub.

  • Ensure that the cluster, its hosts, and all its services are healthy.
  • Ensure that the Kafka brokers and KRaft Controllers are commissioned and running.
  1. Log in to the CDP web interface.
  2. Go to Management Console > Data Hub Clusters.
  3. Find and select the Data Hub cluster you want to scale.
  4. Click Actions > Resize.
  5. Select the kraft host group.
  6. Set the desired size of the host group.
    You can enter a number or use the plus and minus buttons to adjust the number of nodes. Ensure that you set an odd number of nodes.
  7. Optional: If you are upscaling the cluster, select an Adjustment type based on your requirements.
  8. Optional: If you are downscaling the cluster, select Force downscale to forcibly remove hosts.
  9. Click Resize.
  10. Wait until the new nodes become available.
  11. Access the Cloudera Manager instance managing the cluster.
  12. Restart all services that have stale configurations.
The cluster is resized.