Cloudera Docs

Enabling Flexible Server during Azure environment creation

During environment registration in Cloudera, the Flexible Server in public service mode is used by default. You also have two options to use the Flexible Server in private service mode: Flexible Server with Private Link and Flexible Server with Delegated Subnet. Flexible Server with Private Link is the recommended option, while Flexible Server with Delegated Subnet is still available but deprecated

When Cloudera is deployed in “private service” mode (without public endpoints), during environment creation you need to select the private DNS zone in case of using a Flexible Server with Private Link, or provide the following information when using a Flexible Server with delegated subnet:

  • An ID of the delegated subnet

  • A private DNS zone ID (optional parameter).

The virtual network link does not need to be specified as input. If you do not provide the private DNS zone, Cloudera creates it for you.

The steps below show you how to enable a private Flexible Server in Flexible Server with Private Link mode or Flexible Server with Delegated Subnet mode. If you would like to enable Flexible Server instances with public access, you do not need to do anything special, as this option is used by default.

Prerequisites

See Azure prerequisites for Flexible Server.

Steps

Flexible Server with Private Link (recommended)

  1. In the Database section, select Flexible Server with Private Link from the dropdown.

  2. Select a private DNS zone for the private Flexible Server. If you do not select one, it will be created automatically. The private DNS zone must be named privatelink.postgres.database.azure.com.

  3. Finish registering your Azure environment in Cloudera.

Flexible Server with Delegated Subnet (deprecated)

  1. In the Database section, select Flexible Server with Delegated Subnet from the dropdown.

  2. Select a delegated subnet for the private Flexible Server.

  3. Select a private DNS zone for the Private Flexible Server. If you do not select one, it will be created automatically.

  4. Finish registering your Azure environment in Cloudera.

  1. Register an Azure environment using the cdp create-azure-environment CDP CLI command including --existing-network-params with a reference to the Private DNS Zone ID and, in case of Flexible Server with Delegated Subnet, --flexible-server-subnet-ids with a reference to the delegated subnet ID. The virtual network link does not need to be specified as input.

    For example: 
    cdp environments create-azure-environment 
  --environment-name <env-name> 
...
  --existing-network-params networkId=dp-rg-test-vnet,resourceGroupName=dp-rg,subnetIds=/subscriptions/3ddda1c7-d1f5-4e7b-ac81-0523f483b3b3/resourceGroups/dp-rg/providers/Microsoft.Network/virtualNetworks/dp-rg-vnet/subnets/a,/subscriptions/3ddda1c7-d1f5-4e7b-ac81-0523f483b3b3/resourceGroups/dp-rg/providers/Microsoft.Network/virtualNetworks/dp-rg-vnet/subnets/2,default,databasePrivateDnsZoneId=<dns-zone-id>
  –-flexible-server-subnet-ids <delegated-subnet-id>
    The following table explains the required parameters:
    Parameter name Description Possible values
    existing-network-params.databasePrivateDnsZoneId (string) The ID of an existing private DNS zone used for the database. Full resource reference
    flexible-server-subnet-ids (array)

    Required only for Flexible Server with Delegated Subnet option. If not specified, a Flexible Server with Private Link is launched.

    Comma separated list of the subnet names or full resource IDs delegated for flexible server.

    This can be specified in two formats:

    subnet1,subnet2

    or

    /subscriptions/3ddda1c7-d1f5-4e7b-ac81-0523f483b3b3/resourceGroups/dp-rg/providers/Microsoft.Network/virtualNetworks/dp-rg-vnet/subnets/1,/subscriptions/3ddda1c7-d1f5-4e7b-ac81-0523f483b3b3/resourceGroups/dp-rg/providers/Microsoft.Network/virtualNetworks/dp-rg-vnet/subnets/2

    The parameter takes a list of subnet IDs (or creates the IDs in case subnet names are provided), validates if they are indeed delegated, and takes the subnet with the largest CIDR range.

    List of full resource reference(s).

    Although this parameter accepts a comma-separated list of subnets, the subnet with the largest CIDR range is always used. Therefore, you should provide only one subnet.

  2. Set IDBroker mappings as usual using the cdp environments set-id-broker-mappings command.
  3. Create a Data Lake as usual using the cdp create-azure-datalake CDP CLI command, including a reference to the database HA type and the database PostgreSQL engine version.