Cloudera Docs

Dedicating Cloudera Embedded Container Service nodes for specific workloads

You use Cloudera Manager to dedicate Cloudera Embedded Container Service cluster nodes for specific workloads. You can dedicate GPU nodes for Cloudera AI Workbenches, and NVME nodes for Cloudera Data Warehouse workloads.

Dedicating Cloudera Embedded Container Service nodes when creating a new cluster

  1. Check the ECS installation requirements.
  2. Add the new hosts to Cloudera Manager.
  3. In Cloudera Manager UI, click Hosts > Hosts Configuration.
  4. Use the Search box to locate the node_taint configuration property and select Add Host Overrides.
  5. Apply the node_taint at the host override level, and not globally.
  6. Under the node_taint property on the Cloudera Embedded Container Service Host Configuration page:
    • Select Dedicated GPU Node to dedicate the node for Cloudera AI workloads.
    • Select Dedicated NVME node to dedicate the node for Cloudera Data Warehouse workloads.
    • Select Dedicated CDE Node for Data Engineering services (available only if Cloudera Embedded Container Service version is 1.5.5 SP2 or later). If selected, the host is reserved exclusively for all CDE-related services and workloads.
    • Select Dedicated CAI Infra Node for CAI infrastructure services (available only if Cloudera Embedded Container Service version is 1.5.5 SP2 or later). If selected, the host is allocated exclusively for CAI infrastructure services.

    When either of these options are selected, no other workload pods will be allowed to run on the dedicated node.

  7. Click Save Changes.
  8. Run the following command to label the dedicated NVME node for Cloudera Data Warehouse workloads:
    kubectl label node <node name> cdw.cloudera.com/dedicated-executor-host=<any value>

    Run the following command to label the Dedicated CDE Node:

    kubectl label node <node name> cloudera.com/node-role=cde

    Run the following command to label the Dedicated CAI Infra Node:

    kubectl label node <node name> cloudera.com/node-role=cai-infra
  9. Repeat the previous steps to add the other Cloudera Embedded Container Service hosts to Cloudera Manager and assign workload types.
  10. Follow the ECS installation procedure. When you reach the Specify Hosts page in the installation wizard, the hosts you added to Cloudera Manager appear. Select the hosts, click Continue, then proceed through the rest of the installation wizard.
  11. After the installation is complete, the applicable workloads will only run on the specified dedicated nodes.

Dedicating Cloudera Embedded Container Service nodes in an existing cluster

  1. Open the Cloudera Manager Admin Console.
  2. On the Home page, click the ECS Cluster.
  3. Click Hosts, go to the ECS host to be tainted, then go to the Configuration tab.
  4. Click the Configuration tab, then use the Search box to locate the node_taint configuration property.
  5. Under the node_taint property on the Cloudera Embedded Container Service Host Configuration page:
    • Select Dedicated GPU Node to dedicate the node for Cloudera AI workloads.
    • Select Dedicated NVME node to dedicate the node for Cloudera Data Warehouse workloads.
    • Select Dedicated CDE Node for Data Engineering services (available only if Cloudera Embedded Container Service version is 1.5.5 SP2 or later). If selected, the host is reserved exclusively for all CDE-related services and workloads.
    • Select Dedicated CAI Infra Node for CAI infrastructure services (available only if Cloudera Embedded Container Service version is 1.5.5 SP2 or later). If selected, the host is allocated exclusively for CAI infrastructure services.
  6. Click Save Changes.
  7. Run the following command to label the dedicated NVME node for Cloudera Data Warehouse workloads:
    kubectl label node <node name> cdw.cloudera.com/dedicated-executor-host=<any value>

    Run the following command to label the Dedicated CDE Node:

    kubectl label node <node name> cloudera.com/node-role=cde

    Run the following command to label the Dedicated CAI Infra Node:

    kubectl label node <node name> cloudera.com/node-role=cai-infra
  8. Repeat the previous steps to assign workload types to the other ECS hosts.
  9. On the ECS Cluster landing page, click Actions > Refresh Cluster.
  10. After the Refresh is complete, click Actions > Rolling Restart.