Configuring capacity estimations and goals
Cruise Control rebalancing works using capacity estimations and goals. You need to configure the capacity estimates based on your resources, and set the goals for Cruise Control to achieve the Kafka partition rebalancing that meets your requirements.
Capacity | Description |
---|---|
capacity.default.cpu |
100 by default |
capacity.default.network-in |
Given by the internet provider |
capacity.default.network-out |
The optimizers in Cruise Control use the network incoming and outgoing capacities to define a
boundary for optimization. The capacity estimates are generated and read by Cruise Control. A
capacity.json
file is generated when Cruise Control is started. When a new
broker is added, Cruise Control uses the default broker capacity values. However, in case disk
related goals are used, Cruise Control must be restarted to load the actual disk capacity metrics
of the new broker.
After setting the capacity estimates, you can provide different goals that define the
optimization proposals given by Cruise Control. Default.goals
are used to
pre-compute optimization proposals that can be applied regardless of any anomalies. These
default.goals
on a healthy cluster can optimize resource utilization. Supported
goals are also available to assist the optimized rebalancing process. When these goals are
fulfilled, the rebalancing is successful. When the goals are violated, self-healing can be used
and rebalancing must be carried out.
Cruise Control has an anomaly detection feature where goal violations can also be set. The
anomaly.detection.goals
configuration defines when the goals are not met, thus
causing a violation. These anomalies can be fixed by the proposal generated from the
self.healing.goals
configuration. In case there is no self-healing goal
specified, Cruise Control uses the default.goals
setting. Hard goals can also be
set to guarantee the fulfilment of any optimization or self-healing process. You can set the
anomaly-detection goals, self-healing goals, and hard goals in Cloudera Manager under the Cruise
Control Server Advanced Configuration Snippet (Safety Valve) for
cruisecontrol.properties
setting.
You can find the capacity estimate and goal configurations at
.The following table lists all the configurations that are needed to configure Cruise Control specifically to your environment. For the list of goals, see the upstream Cruise Control documentation.
Configuration | Description |
---|---|
num.metric.fetchers |
Parallel threads for fetching metrics from the Cloudera Manager database |
default.goals |
List of default goals |
goals |
List of subset goals |
partition.metric.sample.store.topic |
Storing Cruise Control metrics |
broker.metric.sample.store.topic |
Storing Cruise Control metircs |
partition.metrics.window.ms |
Time window size for partition metrics |
broker.metrics.window.ms |
Time window size for broker metrics |
num.partition.metrics.windows |
Number of stored partition windows |
num.broker.metrics.windows |
Number of stored broker windows |