IP address and cloud resource requirements for Virtual Warehouses running on Azure environments

Learn about the estimated number of IP addresses and cloud resources required to run Virtual Warehouses on Azure environments for Cloudera Data Warehouse (CDW) Public Cloud.

Virtual Warehouse requirements:

Each compute node in a Virtual Warehouse that runs on Azure environments requires 11 IP addresses (for pods and for the node itself). Each executor needs one compute node, so the size of your Virtual Warehouse contributes to the number of IP addresses required. To calculate the number of IP addresses required for custom sizes, multiply the number of executors by 11 and add for the shared services as specified in the following sections.

Shared services requirements for Database Catalogs:

Virtual Warehouses also require shared services for Database Catalogs. When an environment is activated with only the default Database Catalog, it consumes 3 shared services nodes. In general, an environment with one default Database Catalog and one additional Database Catalog consumes 4 shared services nodes. Each shared services node requires 31 IP addresses (for pods and the node itself).

Additional shared services requirements:

The usage for other shared services is different for Hive Virtual Warehouses and Impala Virtual Warehouses. Here are the different requirements for Hive versus Impala Virtual Warehouses:

  • Hive Virtual Warehouses add 1 compute node for each executor and 1 shared services node for the HiveServer for each Virtual Warehouse.
  • Impala Virtual Warehouses add 1 compute node for each executor, 1 or 2 compute nodes for the coordinator, depending on the HA configuration, and 1 shared services node per Virtual Warehouse for Impala catalogd.

The following tables summarize the approximate number of IP addresses and cloud resources you should plan for Virtual Warehouses on Azure environments.

Table 1. Hive Virtual Warehouses running on Azure environments with 1 default and 1 additional Database Catalog
Size (# executors) # Compute nodes for executors # Shared services nodes for default DB Cat # Shared services nodes for additional DB Cats # Shared services nodes for HiveServer Total IP addresses required
XSMALL (2) 2 3 ~1 1

Executor nodes:

2 nodes X 11 = 22

Shared services nodes:

3 + ~1 + 1 = ~5 nodes X 31 = ~155

TOTAL = ~177 IP addresses

SMALL (10) 10 3 ~1 1

Executor nodes:

10 nodes X 11 = 110

Shared services nodes:

3 + ~1 + 1 = ~5 nodes X 31 = ~155

TOTAL = ~265 IP addresses

MED (20) 20 3 ~1 1

Executor nodes:

20 nodes X 11 = 220

Shared services nodes:

3 + ~1 + 1 = ~5 nodes X 31 = ~155

TOTAL = ~375 IP addresses

LARGE (40) 40 3 ~1 1

Executor nodes:

40 nodes X 11 = 440

Shared services nodes:

3 + ~1 + 1 = ~5 nodes X 31 = ~155

TOTAL = ~595 IP addresses

Table 2. Impala Virtual Warehouses running on Azure environments with 1 default and 1 additional Database Catalog
Size (# executors) # Compute nodes for executors # Compute nodes for coordinator # Shared services nodes for Impala catalogd # Shared services nodes for default DB Cat # Shared services nodes for additional DB Cats Total IP addresses required
XSMALL (2) 2 1-2 1 3 ~1

Executor/ coordinator nodes:

3-4 nodes X 11 = 33-44

Shared services nodes:

1 + 3 + ~1 = ~5 nodes X 31 = ~155

TOTAL = ~188 to ~199 IP addresses

SMALL (10) 10 1-2 1 3 ~1

Executor/ coordinator nodes:

11-12 nodes X 11 = 121-132

Shared services nodes:

1 + 3 + ~1 = ~5 nodes X 31 = ~155

TOTAL = ~276 to ~287 IP addresses

MED (20) 20 1-2 1 3 ~1

Executor/ coordinator nodes:

21-22 nodes X 11 = 231-242

Shared services nodes:

1 + 3 + ~1 = ~5 nodes X 31 = ~155

TOTAL = ~386 to ~397 IP addresses

LARGE (40) 40 1-2 1 3 ~1

Executor/ coordinator nodes:

41-42 nodes X 11 = 451-462

Shared services nodes:

1 + 3 + ~1 = ~5 nodes X 31 = ~155

TOTAL = ~606 to ~617 IP addresses