Cloudera Docs

Activating an Azure environment from CDW

To use an Azure environment for Cloudera Data Warehouse (CDW) Public Cloud you must first activate it.

When you activate an environment, CDP creates an AKS (Azure Kubernetes Services) cluster to host Kubernetes-based resources. The underlying compute, network resources are managed by Azure, including the following ones:
  • Resource group
  • Compute instances, which are virtual machine scale sets
  • Load balancer(s)
  • Public IP address(es)
  • Network security group
  • Disk(s)
CDP supports the following Azure virtual machines and their corresponding compute instance types (Hive and Impala executors), which you select during environment activation:
Table 1. Compute Instance Types
Azure VM Processor Type Usage Virtual Warehouse Support
Standard_E16pds_v5 ARM Compute Impala

Standard_E16_v3

Intel Compute Hive and Impala
Standard_E16ds_v4 Intel Compute (default) Hive and Impala
Standard_E16ads_v5 AMD Compute Hive and Impala
Standard_E16ds_v5 Intel Compute Hive and Impala
Standard_D8s_v4 Intel Shared services (default) Hive and Impala
Standard_D8as_v5 Intel Shared services, used with AMD compute instance Standard_E16ads_v5 Hive and Impala
Three instances are added to the cluster as needed for shared services (always on components). Three shared nodes are dStandard_E2s_v3 MemoryOptimized using flexserver, for the Amazon Relational Database Service (RDS). These shared nodes are used for Hue and Data Visualization user metadata. For more information, see Always active, shared services.

Deploying a private AKS

If you want to deploy a Private AKS cluster, you must use the CDP CLI as described in AKS deployment configuration options.

  1. In the CDW service, in Environments, locate the environment that you want to activate.
  2. Click Activate.
  3. In Activate Environment, configure the environment or accept the defaults:
    • Select the Compute VM Size based on your workload.
    • Select a Subnet inside the virtual network (VNet) that you want to use for CDW.

      The VNet that you select must have a sufficient number of free IP addresses.

    • Select Enable internal load balancer (ingress) to distribute traffic inside a virtual network.

      For information about load balancers, see "Azure load balancers".

    • Optionally, select Enable availability zones for AKS.
    • Specify a user-assigned, managed identity for the AKS cluster.
    • Select Enable AKS monitoring and then select the workspace from the adjacent drop-down list.
    • Add trusted endpoint IP CIDRs for your load balancer in a comma-separated list to the Enable IP CIDR for the load balancer text box.
    • Accept the default Use CNI overlay networking if IP address exhaustion is a concern for your deployment.
  4. Optional: To use custom repository, select the Use Custom repository option.
    Depending in your custome repository type, input ecr, acr, or docker in the Registry Type field.
    Specify the URL of your custom repository. Fields for your user name and password appear. You do not need to enter these credentials.
  5. Click ACTIVATE.