Managing ClustersPDF version

Vertically scaling instance types

If necessary, you can select a larger or smaller instance type for a Cloudera Data Hub or Data Lake cluster after it has been deployed in AWS, Azure, and GCP.

You must stop the Data Lake or Cloudera Data Hub cluster before you vertically scale any of the instances. See Change the instance type in AWS documentation for more information.
Selecting a larger instance type adds more vCPU and/or RAM to your instances. Instances can be scaled both up and down, but scaling down to a smaller size requires 4 CPU and a minimum of 4 GB memory.

Limitations:

  • You cannot scale down to a target instance type that has fewer ephemeral volumes than the current instance type.
  • Vertical scaling to Azure v5 instances is not supported and results in the following error: 
    Unable to resize since changing from resource disk to non-resource disk VM size and vice-versa is not allowed.
  • Vertical scaling is supported on AWS, Azure, and GCP.
  • If you are using an instance without ephemeral disks, you can scale up or down to a new instance with ephemeral disks; however, the reverse is not supported. You cannot start with an instance with ephemeral disks and move to an instance without ephemeral disks.

For information on vertically scaling FreeIPA, see Vertically scale FreeIPA instances.

  1. In the Cloudera main navigation menu, click Data Hub or Data Lakes and select the cluster that requires a larger instance type.
  2. Scroll to the bottom of the page and click the Nodes tab.
  3. Click the Vertical Scaling icon on the top right of the host group that you want to scale.
  4. Select a larger instance type from the drop-down menu of suggested instance types.
  5. Click Scale. You can monitor the action from the Event History tab.
    Alternatively, you can use the CDP CLI to select a new instance for the Data Lake or Cloudera Data Hub cluster:

    Data Lake cluster:

    cdp datalake start-datalake-vertical-scaling 
--datalake <your-data-lake-name-or-its-crn> 
--group <master> 
--instance-template instanceType="<m5.4xlarge>"

    Cloudera Data Hub cluster:

    cdp datahub start-cluster-vertical-scaling 
--datahub <your-data-hub-name-or-its-crn> 
--group <master> 
--instance-template instanceType="<m5.4xlarge>"
After you have vertically scaled the cluster, configure the services on the cluster to use the additional or reduced resources/memory.

We want your opinion

How can we improve this page?

What kind of feedback do you have?