Cloudera Docs

Setting up the environment for private cluster deployment

In Azure, you perform steps and follow links to Azure documentation to meet prerequisties for enabling a private CDW environment.

  1. In the Microsoft Azure portal, create a resource group for CDP.

    The resource group provides a management layer that enables you to create, update, and delete resources in your Azure account.

  2. (Optional)Create a private storage account and network access rules to block all internet traffic.

    The private endpoints allow clients on a virtual network (VNET) to securely access data over a Private Link.

  3. Create a VNet and add a subnet.

    VNet is the fundamental building block for your private network in Azure. It enables Azure resources, such as Azure Virtual Machines, to securely communicate with each other and with the internet.

  4. Create a delegated subnet for Azure Database for PostgreSQL Flexible Server.
  5. (Optional) Configure the CDP Control Plane Private Link service.

    To ensure that all egress traffic including CDP Control Plane traffic travels only through private networks, you must configure Cluster Connectivity Manager (CCM) (v2) and CDP Control Plane private endpoints properly for the selected VNet.

    Contact your Cloudera account representative to register the CDP Control Plane Private Link service endpoints in your Azure VNet.

  6. Configure custom DNS on the VNet to resolve Azure Private DNS zones.

    To resolve private endpoint DNS records, the VNet DNS servers must be capable of resolving Azure DNS records.

  7. Disable network endpoint policies for private endpoints and Azure Private Link Service.
  8. Configure the following firewall exceptions for CDW and AKS on the egress firewall, and for storage account endpoints, if the account is not Private-Link enable: Outbound network access destinations and Restrict egress traffic in AKS
  9. Configure user-defined routing (UDR) on the VNet to forward all traffic to an egress firewall and link it to the subnet.