Scheduling executor pods on dedicated worker nodes in CDW Private Cloud
To optimize resource utilization on your cluster and improve query performance, Cloudera Data Warehouse (CDW) allows you to schedule Hive and Impala executor and coordinator pods on dedicated executor nodes that are tainted for CDW and equipped with more local storage, either using SSDs or NVMEs.
To block CDW pods, other than executors and coordinators to be scheduled on a dedicated worker node with higher local storage capacity, you must enable the Use dedicated nodes for executors option while activating an environment in CDW or by editing an existing environment. This allows nodes with available local storage to be reserved for CDW query executor pods over all other CDW or other data services pods marking them exclusively for CDW executor pods.
By default, the Use dedicated nodes for executors option is disabled. When this option is disabled, all CDW pods, including Hive MetaStore (HMS), Data Visualization, Hive and Impala executors and coordinators can be scheduled on the tainted worker nodes. When you enable this option, only Hive and Impala executor and coordinator pods can be scheduled on the tainted worker nodes.
To enable the Use dedicated nodes for executors option while activating the environment, see Activating OpenShift environments or Activating Embedded Container Service environments. This task describes how to enable the Use dedicated nodes for executors option on an existing environment.
On Embedded Container Service (ECS), you must dedicate the Embedded Container Service cluster nodes for specific workloads using Cloudera Manager, as described in the Dedicating ECS nodes for specific workloads section.
On OpenShift Container Platform (OCP), you must dedicate the OCP cluster nodes for specific workloads using the kubectl taint command, as described in the Dedicate an SSD node for CDW workloads section.
-
Locate the environment on which you want to enable this feature and click
> Edit.
