Cloudera Docs

Creating custom pod configurations for Impala Virtual Warehouses

You can configure the resources used by Impala Virtual Warehouses in Cloudera Data Warehouse (CDW) Private Cloud environments to optimize Impala performance or to control resource usage in the environment.

When you create a Virtual Warehouse, CDW allocates standard resources to the Warehouses that are suitable for most workloads. You can control the size of the Virtual Warehouse at the time of creation by choosing the number of nodes to be used. By using custom pod configurations, you can also change the resources used by the critical Impala components, such as the coordinators, executors, and catalog daemons to pack a particular number of pods into a Kubernetes node or to create extra-large daemons to handle specific workloads.
  1. Log in to the Data Warehouse service as a DWAdmin.
  2. Go to your environment and click > Edit.
    The Environment Details page is displayed.
  3. Click the EDIT POD CONFIGURATIONS tab.
    A pod configuration is a named resource that is configured at the environment level.
  4. Select one of the following two pod configuration options from the Select Pod Configuration section:
    • The Cdw Defaults option is selected by default. CDW uses default values for the pods if a specific pod configuration is not used.
    • Select the 1 x Node option for allocation of most node resources found in the environment, to the Impala executors and coordinators.
    The following image shows the two default pod configurations:

    Environment details page showing the pod configuration options.
  5. Click COPY CONFIG to create and edit a new configuration with the option that you selected earlier as the basis.
    1. Specify the name for your configuration in the Cloned Config Name field.
    2. Optional: Enter a description for the new configuration in the Description field.

      Image showing the Pod Configuration section on the Environment Details page where you can clone a pod configuration based on one of the existing configurations to further customize it. It also shows a toggle button that you can use to make a configuration a default configuration for an environment.
    3. Click CREATE NEW CONFIG and then click APPLY under the EDIT POD CONFIGURATION tab.
      A new pod configuration is created, which you can now customize.
  6. Specify the values for the following parameters under the Coordinator section:
    • Memory
    • CPU
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
    • AC Slots (admission_control_slots flag)
    • Cache (size of the data cache)
    • Scratch (limit of Impala scratch space)

    Image showing the Coordinator section on the Environment Details page, where you can specify the memory, CPUs, initial and maximum memory allocation pool for JVMs, AC slots, cache, and scratch space that should be allocated to the Impala coordinator process.
  7. Specify the values for the following parameters under the Executor section:
    • Memory
    • CPU
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)
    • AC Slots (admission_control_slots flag)
    • Cache (size of the data cache)
    • Scratch (limit of Impala scratch space)

    Image showing the Executor section on the Environment Details page, where you can specify the memory, CPUs, initial and maximum memory allocation pool for JVMs, AC slots, cache, and scratch space that should be allocated to the Impala executor process.
  8. Specify the values for the following parameters under the Catalog section:
    • Memory
    • CPU
    • Xmx (maximum memory allocation pool for a Java Virtual Machine)
    • Xms (initial memory allocation pool for a Java Virtual Machine)

    Image showing the Catalog section on the Environment Details page, where you can specify the memory, CPUs, and the initial and maximum memory allocation pool for JVMs that should be allocated to the Impala catalog process.
  9. Specify the values for the following parameters under the Default Settings section:
    • MaxQueryMemLimit
    • MinQueryMemLimit
    • mt_dop

    Image showing the Default Settings section on the Environment Details page, where you can set the minimum and maximum memory limit for a query and the mt_dop value (the degree of intra-node parallelism).
  10. Click APPLY under the EDIT POD CONFIGURATION tab to save the custom settings.
    The “Configuration update initiated” message is displayed.
  11. Optional: Click the Set as default configuration toggle button to make this a default pod configuration.
    This makes a pod configuration the default configuration at the environment level.
  12. Click APPLY at the top of the Environment Details page.
    The new pod configuration becomes available in the Pod Config drop-down menu as shown in the following image. You can select this Impala pod configuration while creating a new Impala Virtual Warehouse:

    Image showing the newly added pod configuration added to the Pod Config drop-down menu on the Add new Virtual Warehouse page.
While adding a new Impala Virtual Warehouse, you can select the Pod Configuration to be used for resource allocation. The default value is "Cdw Defaults", but you can select other configurations available in your environment that you created using these steps.