Configuring Impala coordinator high availability

A single Impala coordinator might not handle the number of concurrent queries you want to run or provide the memory your queries require. You can configure multiple active coordinators to resolve or mitigate these problems. You can change the number of active coordinators later.

You can configure up to five active-active Impala coordinators to run in an Impala Virtual Warehouse. When you create an Impala Virtual Warehouse, you use the same UI dropdown to configure Impala coordinator and Database Catalog high availability, described in the next topic. You can choose one of the following options:
  • Disabled

    Disables Impala coordinator and Database Catalog high availability

  • Active-passive

    Runs multiple coordinators (one active, one passive) and Database Catalogs (one active, one passive)

  • Active-active coordinators

    Runs multiple coordinators (both active) and Database Catalogs (one active, one passive)

Using two coordinators in active-passive mode, one coordinator is active at a time. If the coordinator goes down, the passive coordinator becomes active.

When using active-active coordinators, the client software uses a cookie to keep a virtual connection to a particular coordinator. When a coordinator disappears for some reason, perhaps due to a coordinator shutting down, then the client software may print the error "Invalid session id" before it automatically reconnects to a new coordinator.

Using active-active coordinators, you can have up to five coordinators running concurrently in active-active mode with a cookie-based load-balancing.

An Impala Web UI appears for each coordinator that you can use for troubleshooting.

Clients who connect to your Impala Virtual Warehouse using multiple coordinators must use the latest Impala shell. The following procedure covers these tasks.

  1. Follow instructions for "Adding a new Virtual Warehouse".
  2. In Size, select the number of executors, for example xsmall-2Executors.
    A number of additional options appear, including High availability.
  3. In New Virtual Warehouse, in High availability (HA), select Enabled (Active-Active).
  4. In Number of Active Coordinators, accept the default of 2, or increase the number of coordinators up to 5.
    You can edit an existing Impala Virtual Warehouse to change the number of active coordinators.
  5. Accept default values for other settings, or change the values to suit your use case.
    Click the tooltip for information about settings.
  6. After creating the Virtual Warehouse, click Cloudera Data Warehouse > Overview > Web UI, and then click each Impala Coordinator Web UI link to get information about the coordinator.
  7. In the Impala Virtual Warehouse, click options , and select Copy Impala shell Download command.
    This action copies a command to your clipboard that looks something like this:
    impala-shell --protocol='hs2-http' --ssl -i "tpcds-impala.your_company.com:443"
  8. Provide the command to clients who want to connect to the Impala Virtual Warehouse with multiple coordinators using the Impala shell.
  9. Instruct the client user to update impyla to version compatible with CDW, as listed in Data Warehouse Release Notes in section, “Version mapping”.
    For example, installing/updating impyla 0.18a2, is required to connect to your Virtual Warehouse active-active coordinators in CDW 2021.0.3-b27 or later.
  10. Inform the client that to connect over ODBC to an HA-configured Impala Virtual Warehouse that uses active-active coordinators, you must append impala.session.id to the HTTPAuthCookies connector configuration option of the Cloudera ODBC driver.
    Table 1. HTTPAuthCookies
    Key Name Value Required
    HTTPAuthCookies impala.auth,JESSSIONID,KNOXSESSIONID,impala.session.id Yes