Cloudera Docs

Creating custom pod configurations for Impala Virtual Warehouses

You can configure the resources used by Impala Virtual Warehouses in Cloudera Data Warehouse (CDW) Private Cloud environments to optimize Impala performance or to control resource usage in the environment.

When you create a Virtual Warehouse, CDW allocates standard resources to the Warehouses that are suitable for most workloads. You can control the size of the Virtual Warehouse at the time of creation by choosing the number of nodes to be used. By using custom pod configurations, you can also change the resources used by the critical Impala components, such as the coordinators, executors, and catalog daemons to pack a particular number of pods into a Kubernetes node or to create extra-large daemons to handle specific workloads.
  1. Log in to the Data Warehouse service as a DWAdmin.
  2. Go to your environment and click > Edit.
    The Environment Details page is displayed.
  3. Click the EDIT POD CONFIGURATIONS tab.
    A pod configuration is a named resource that is configured at the environment level.
  4. Select one of the following two pod configuration options from the Select Pod Configuration section:
    • The Cdw Defaults option is selected by default. CDW uses default values for the pods if a specific pod configuration is not used.
    • Select the 1 x Node option for the allocation of most node resources found in the environment, to the Impala executors and coordinators.
    Cdw Defaults and 1 x Node are read-only options.
  5. Click Copy Config to create and edit a new configuration with the option that you selected earlier as the basis.
    1. Specify the name for your configuration in the Cloned Config Name field.
    2. Optional: Enter a description for the new configuration in the Description for new config field.
    3. Click Create New Config.
      A new pod configuration is created, which you can now customize.
  6. Specify the values for the following parameters under the Coordinator section:
    • Memory
    • Cpus
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
    • AC Slots (admission_control_slots flag)
    • Cache size (size of the data cache)
    • Scratch size (limit of Impala scratch space)
    • Overhead size (size for resources used by tools run by the containers)
  7. Specify the values for the following parameters under the Executor section:
    • Memory
    • Cpus
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
    • AC Slots (admission_control_slots flag)
    • Cache (size of the data cache)
    • Cache size (size of the data cache)
    • Scratch size (limit of Impala scratch space)
    • Overhead size (size for resources used by tools run by the containers)
  8. Specify the values for the following parameters under the Catalog section:
    • Memory
    • Cpus
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
  9. Specify the values for the following parameters under the Default Settings section:
    • MaxQueryMemLimit
    • MinQueryMemLimit
    • mt_dop
  10. Click Apply under the EDIT POD CONFIGURATION tab to save the custom settings.
    The “Configuration update initiated” message is displayed.
  11. Optional: Click the Set as default configuration toggle button to make this a default pod configuration.
    This makes a pod configuration the default configuration at the environment level.
  12. Click APPLY at the top of the Environment Details page.
    The new pod configuration becomes available in the Pod Config drop-down menu as shown in the following image. You can select this Impala pod configuration while creating a new Impala Virtual Warehouse:

    Image showing the newly added pod configuration added to the Pod Config drop-down menu on the Add new Virtual Warehouse page.
While adding a new Impala Virtual Warehouse, you can select the Pod Configuration to be used for resource allocation. The default value is "Cdw Defaults", but you can select other configurations available in your environment that you created using these steps.