Cloudera Docs

Creating custom pod configurations for Impala Virtual Warehouses

You can configure the resources used by Impala Virtual Warehouses in Cloudera Data Warehouse (CDW) Private Cloud environments to optimize Impala performance or to control resource usage in the environment.

When you create a Virtual Warehouse, CDW allocates standard resources to the Warehouses that are suitable for most workloads. You can control the size of the Virtual Warehouse at the time of creation by choosing the number of nodes to be used. By using custom pod configurations, you can also change the resources used by the critical Impala components, such as the coordinators, executors, and catalog daemons to pack a particular number of pods into a Kubernetes node or to create extra-large daemons to handle specific workloads.
Ensure that the Customize pod sizes option is selected on the Advanced Configuration > Advanced Settings page.

You must have already activated an environment in CDW.

  1. Log in to the Data Warehouse service as a DWAdmin.
  2. Click on the Environments tab.
  3. Locate your environment and click > Edit.
    The Environment Details page is displayed.
  4. Click the POD CONFIGURATIONS tab.
    The following two read-only options are displayed: Cdw Defaults and 1 x Node. You can create a copy of these options and customize them.
    Pod configuration tab in CDW showing the prebuilt pod configurations available for customization.
    • The Cdw Defaults option is selected by default. CDW uses default values for the pods if a specific pod configuration is not used.
    • The 1 x Node option calculates and shows the resources based on the attributes of a random node from a cluster, assuming all of them are the same.
    A pod configuration is a named resource that is configured at the environment level.
  5. Click Actions corresponding to the configuration you want to copy and customize and click Make A Copy.
  6. Specify a name for the new configuration in the Cloned Config Name field. Optionally, specify a description and click Create New Config.
    Image showing the option to make a copy of an existing pod configuration.
    A new pod configuration is created, which you can now customize.
  7. Specify the values for the following parameters under the Coordinator section:
    • Memory
    • CPUs
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
    • AC Slots (admission_control_slots flag)
    • Cache size (size of the data cache)
    • Scratch size (limit of Impala scratch space)
    • Overhead size (size for resources used by tools run by the containers)
    Image showing sample values for the coordinator pod configuration.
  8. Specify the values for the following parameters under the Executor section:
    • Memory
    • CPUS
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
    • AC Slots (admission_control_slots flag)
    • Cache size (size of the data cache)
    • Scratch size (limit of Impala scratch space)
    • Overhead size (size for resources used by tools run by the containers)
    Image showing sample values for the executor pod configuration.
  9. Specify the values for the following parameters under the Catalog section:
    • Memory
    • CPUS
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
    Image showing sample values for the catalog pod configuration.
  10. Specify the values for the following parameters under the Default Settings section:
    • MaxQueryMemLimit
    • MinQueryMemLimit
    • mt_dop
    Image showing sample values for the default pod settings
  11. Optional: Make the configuration default by selecting the Set As Default option.
    This makes a pod configuration the default configuration at the environment level.
  12. Click Apply Changes under the POD CONFIGURATION tab to save the custom settings.
    The Configuration update initiated message is displayed.
    The new pod configuration becomes available in the Pod Config drop-down menu as shown in the following image. You can select this Impala pod configuration while creating a new Impala Virtual Warehouse.

    Image showing the newly added pod configuration added to the Pod Config drop-down menu on the Add new Virtual Warehouse page.
While adding a new Impala Virtual Warehouse, you can select the Pod Configuration to be used for resource allocation. The default value is "Cdw Defaults", but you can select other configurations available in your environment that you created using these steps.