Cloudera Docs

Using Cloudera-managed private DNS

Review this documentation if you are planning to use a private endpoint for Azure Postgres with a Cloudera-managed DNS.

Requirements and limitations

The following limitations apply when using a Cloudera-managed private DNS:

  • Only Azure’s private DNS zone is supported. Using an on-premise DNS is not supported.

  • Private endpoints can only be used with a single customer-provided resource group. They cannot be used with Cloudera-created multiple resource groups. When creating an environment from the UI, the “Create Private Endpoints” option is disabled if you select the option for Cloudera to create multiple resource groups. If using CDP CLI, a validation error appears if this option is used with Cloudera creating multiple resource groups.

  • The private DNS zone must be in the single existing resource group, even if the VNet is located elsewhere. This is because:

    • Microsoft requires that the DNS zone has a fixed name.

    • A VNet cannot have links to two different DNS zones with the same name.

    • It is not possible to check if a VNet already has a DNS zone with a given name attached to it.

  • Only one resource group can have private endpoints with a given VNet. This is because:

    • Only one DNS zone with a given name can be linked to a VNet.

    • That DNS zone must be in the single resource group where all the resources are located.

  • The private DNS zone and virtual network links are shared within the single resource group. The first environment ever created in that resource group will create them. They will never be deleted by Cloudera.

  • The private DNS zone creation cannot be turned off. There is no option to create private endpoints but not create a private DNS zone. You can, however, choose to specify your own existing private DNS zone. See Bringing your own private DNS.

Prerequisites

In order to use a Cloudera-managed private DNS, you should meet the following prerequisites:

  1. Disable private endpoint network policies
  2. Review DNS zones existing in your resource group
  3. Ensure that Cloudera has adequate permissions

Disable private endpoint network policies

Only subnets that have private endpoint network policies turned off are eligible for private endpoint creation, because network security groups (NSG) are not supported for private endpoints.

You can use the following Azure CLI command to disable private endpoint network policies for a certain subnet:

az network vnet subnet update \
  --name <subnet-name> \
  --resource-group <resource-group-name> \
  --vnet-name <vnet-name> \
  --disable-private-endpoint-network-policies true

For example: 

az network vnet subnet update \
  --name default-2 \
  --resource-group my-cdp-rg \
  --vnet-name my-azure-vnet \
  --disable-private-endpoint-network-policies true

Review DNS zones existing in your resource group

If you want Cloudera to create and manage the private DNS zone, review the DNS zones that exist in the resource group that you are planning to use for Cloudera and make sure that one of the following is true:

  • No private DNS zone named “privatelink.postgres.database.azure.com“ is connected to the VNnet.

  • If there is a private DNS zone named “privatelink.postgres.database.azure.com“ connected to the VNet, verify that the zone is located in the existing resource group that you are planning to use for Cloudera. If the private DNS zone is already used for one environment, Cloudera can reuse it for another environment.

Ensure that Cloudera has adequate permissions

Ensure that the role that you are using for the Azure credential has the permissions mentioned in Role definition 2: Allows Cloudera to use only a single existing resource group and create private endpoints.