Deploying Cruise Control

Learn how to deploy Cruise Control alongside your Kafka Cluster using cruiseControl properties in the Kafka resource. Deploying Cruise Control is optional but strongly recommended as it automates the partition rebalancing in the cluster.

You can deploy Cruise Control alongside a Kafka cluster by adding cruiseControl properties to your Kafka resource. Deploying Cruise Control creates a Cruise Control deployment that contains a Cruise Control pod.

If you specify an empty object (cruiseControl: {}), Cruise Control is deployed with the upstream recommended default configuration. You can customize the configuration of Cruise Control by specifying the required options in the cruiseControl property.

Cruise Control requires at least two Kafka brokers. If you try to add Cruise Control while there is only a single Kafka broker in the cluster, the deployment fails. Increase your broker replica count if necessary.

  1. Add a cruiseControl property to your Kafka resource.
    #...
    kind: Kafka
    spec:
      cruiseControl: {}
    
  2. Create or update your resource.
    kubectl apply --filename [***YAML CONFIG***] --namespace [***NAMESPACE***]
  3. Verify the status of the deployment.
    kubectl get deployments --namespace [***NAMESPACE***]

    If deployment is successful, you should see a Cruise Control deployment in the output.

    NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
    my-cluster-cruise-control    1/1     1            1           5m1s
    

    The READY column shows the number of replicas that are ready/expected. The deployment is successful when the AVAILABLE output shows 1.

After Cruise Control is deployed, you can use KafkaRebalance resources to rebalance your cluster. Typically you initiate a rebalance process when scaling your cluster, but rebalances can be carried out at any time.