Cloudera Docs

Enable Flow metrics based auto-scaling

Flow metrics based scaling is an optional setting which can be enabled in addition to CPU based auto-scaling.

Before you begin

When deploying a flow definition, continue to Step 4: Sizing & Scaling and pick your NiFi Node size

Steps

  1. Activate the Auto Scaling toggle to turn on CPU based auto-scaling.

  2. Use the slider or text fields to adjust the minimum and maximum number of nodes. Cloudera DataFlow automatically determines the best number of nodes within the specified boundaries.

  3. Select Flow Metrics Scaling to enable flow metrics based auto-scaling for your deployment.

To configure auto-scaling using the Cloudera CLI, execute the cdp df create-deployment command with the following configuration:

cdp df create-deployment \
  --service-crn <your DFService CRN> \
  --flow-version-crn "<your flow version CRN>" \
  --deployment-name "<your flow deployment name>" \
  --cfm-nifi-version <your desired NiFi version> \
  --auto-start-flow \
  --cluster-size-name <your desired NiFi Node Size> \
  --auto-scaling-enabled \
  --auto-scale-min-nodes 1 \
  --auto-scale-max-nodes 3
  --flow-metrics-scaling-enabled