IP address and cloud resource requirements for Virtual Warehouses running on Azure environments
Learn about the estimated number of IP addresses and cloud resources required to run Virtual Warehouses on Azure environments for Cloudera Data Warehouse on cloud.
Virtual Warehouse requirements:
Each compute node in a Virtual Warehouse that runs on Azure environments requires 16 IP addresses (for pods and for the node itself). Each executor needs one compute node, so the size of your Virtual Warehouse contributes to the number of IP addresses required. To calculate the number of IP addresses required for custom sizes, multiply the number of executors by 16 and add for the shared services as specified in the following sections.
Shared services requirements for Database Catalogs:
Virtual Warehouses also require shared services for Database Catalogs.
Additional shared services requirements:
The usage for other shared services is different for Hive Virtual Warehouses and Impala Virtual Warehouses. Here are the different requirements for Hive versus Impala Virtual Warehouses:
- Hive Virtual Warehouses add 1 compute node for each executor and 1 shared services node for the HiveServer for each Virtual Warehouse.
- Impala Virtual Warehouses add 1 compute node for each executor, 1 or 2 compute nodes for the coordinator, depending on the HA configuration, and 1 shared services node per Virtual Warehouse for Impala catalogd.
The following tables summarize the approximate number of IP addresses and cloud resources you should plan for Virtual Warehouses on Azure environments.
| Size (# executors) | # Compute nodes for executors | # Shared services nodes for Database Catalog | # Shared services nodes for HiveServer | Total IP addresses required |
|---|---|---|---|---|
| XSMALL (2) | 2 | 3 | 1 |
Executor nodes: 2 nodes X 16 = 32 Shared services nodes: 3 + 1 = 4 nodes X 31 = 124 TOTAL = 156 IP addresses |
| SMALL (10) | 10 | 3 | 1 |
Executor nodes: 10 nodes X 16 = 160 Shared services nodes: 3 + 1 = 4 nodes X 31 = 124 TOTAL = 284 IP addresses |
| MED (20) | 20 | 3 | 1 |
Executor nodes: 20 nodes X 16 = 320 Shared services nodes: 3 + 1 = 4 nodes X 31 = 124 TOTAL = 444 IP addresses |
| LARGE (40) | 40 | 3 | 1 |
Executor nodes: 40 nodes X 16 = 640 Shared services nodes: 3 + 1 = 4 nodes X 31 = 124 TOTAL = 764 IP addresses |
| Size (# executors) | # Compute nodes for executors | # Compute nodes for coordinator | # Shared services nodes for Impala catalogd | # Shared services nodes for Database Catalog | Total IP addresses required |
|---|---|---|---|---|---|
| XSMALL (2) | 2 | 1-2 | 1 | 3 |
Executor/ coordinator nodes: 3-4 nodes X 16 = 48-64 Shared services nodes: 1 + 3 = 4 nodes X 31 = 124 TOTAL = 172 to 188 IP addresses |
| SMALL (10) | 10 | 1-2 | 1 | 3 |
Executor/ coordinator nodes: 11-12 nodes X 16 = 176-192 Shared services nodes: 1 + 3 = 4 nodes X 31 = 124 TOTAL = 300 to 316 IP addresses |
| MED (20) | 20 | 1-2 | 1 | 3 |
Executor/ coordinator nodes: 21-22 nodes X 16 = 336-352 Shared services nodes: 1 + 3 = 4 nodes X 31 = 124 TOTAL = 460 to 476 IP addresses |
| LARGE (40) | 40 | 1-2 | 1 | 3 |
Executor/ coordinator nodes: 41-42 nodes X 16 = 656-672 Shared services nodes: 1 + 3 = 4 nodes X 31 = 124 TOTAL = 780 to 796 IP addresses |
