Configuring goals

After setting the capacity estimates, you can specify which goals need to be used for the rebalancing process in Cloudera Manager. The provided goals are used for the optimization proposal of your Kafka cluster.

  1. Access the Configuration page of Cruise Control.
    1. Go to your cluster in Cloudera Manager.
    2. Select Cruise Control from the list of services.
    3. Click Configuration.
  2. Search for goals using the search bar.
    The list of goals are displayed based on the goal sets.
  3. Add goals using the property name to the Default, Supported, Hard, Self-healing and Anomaly detection lists based on your requirements, and click Save Changes.
    The following table lists the goals that can be used:
    Goal Property name Description
    RackAwareDistributionGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareDistributionGoal As long as replicas of each partition can achieve a perfectly even distribution across the racks, this goal lets placement of multiple replicas of a partition into a single rack.
    ReplicaCapacityGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal Attempt to make all the brokers in a cluster to have less than a given number of replicas.
    CapacityGoals com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal Goals that ensure the broker resource utilization is below a given threshold for the corresponding resource.
    com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal
    com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal
    com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal
    ReplicaDistributionGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal Attempt to make all the brokers in a cluster to have a similar number of replicas.
    PotentialNwOutGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.PotentialNwOutGoal A goal that ensures the potential network output (when all the replicas become leaders) on each of the brokers do not exceed the broker’s network outbound bandwidth capacity.
    ResourceDistributionGoals com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskUsageDistributionGoal Attempt to make the resource utilization variance among all the brokers are within a certain range. This goal does not do anything if the cluster is in a low utilization mode (when all the resource utilization of each broker is below a configured percentage.) This is not a single goal, but consists of the following separate goals for each of the resources.
    com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundUsageDistributionGoal
    com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundUsageDistributionGoal
    com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuUsageDistributionGoal
    TopicReplicaDistributionGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal Attempt to make the replicas of the same topic evenly distributed across the entire cluster.
    LeaderReplicaDistributionGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderReplicaDistributionGoal Attempt to make all the brokers in a cluster to have the similar number of leader replicas.
    LeaderBytesInDistributionGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderBytesInDistributionGoal Attempt to make the leader bytes in rate on each host to be balanced.
    PreferredLeaderElectionGoal com.linkedin.kafka.cruisecontrol.analyzer.goals.PreferredLeaderElectionGoal Attempt to make the first replica in the replica list leader replica of the partition for all topic partitions.
    KafkaAssignerGoals1 com.linkedin.kafka.cruisecontrol.analyzer.kafkaassigner.KafkaAssignerDiskUsageDistributionGoal A goal that ensures all the replicas of each partition are assigned in a rack aware manner.
    com.linkedin.kafka.cruisecontrol.analyzer.kafkaassigner.KafkaAssignerEvenRackAwareGoal Attempt to make all the brokers in a cluster to have the similar number of replicas
1 These goals are used to make Cruise Control behave like a Kafka assigner tool. These goals will be picked up if kafka_assigner parameter is set to true in the corresponding request (for example, with the rebalance request as shown in the Cruise Control documentation).