Azure environments requirements checklist

To successfully activate Azure environments with Cloudera Data Warehouse (CDW) service, make sure your environment meets the requirements listed in this topic.

1. Specific networking requirements

  • Make sure Azure VNet subnets are large enough to support the CDW load

    When an Azure environment is activated for CDW service, an Azure Kubernetes Service (AKS) cluster is provisioned in your subscription. The AKS cluster uses the Azure Container Networking Interface (CNI) plug-in for Kubernetes. This plug-in assigns IP addresses for every pod running inside the Kubernetes cluster. By default, the maximum number of pods per node is 30. This means that you need approximately 3,200 IP addresses for a 99-node cluster. if you activate an environment for CDW service, make sure that the subnets are large enough on the Azure VNet for the CDW load. Cloudera recommends using a CIDR/20 subnet or larger.

  • Configure service endpoints on CDW subnets

    You must configure service endpoints on the subnets used for the CDW service. This ensures that the network traffic between CDW components and Azure services remain on the Microsoft Azure backbone network. To use with CDW, Microsoft.Storage and Microsoft.SQL must be registered. Without this step CDW service cannot be activated on existing Azure VNets. For more information, see Virtual Network service endpoints in the Azure documentation.

2. Use only app-based credentials

For the Data Warehouse service, you must only use an app-based credential, which requires the Contributor role to create a new service principal. For more information about creating an app-based credential for the environment you want to use for the Data Warehouse service, see Create an app-based credential. If you need to change your environment credential, see Change environment's credential. Both of these references are in the Management Console documentation.

3. App must have the Contributor role at the subscription level

For environments that you plan to use for the Data Warehouse service, you must ensure that the application you create in Azure has the built-in Contributor Azure role at the Azure subscription level. For more information, see the description of app-based credentials in Credential options on Azure in the Management Console documentation.

4. Created Azure app must have access to the storage account used during environment registration

Ensure that the application, which the Azure app-based credentials are attached to, must have access to the ADSL Gen2 storage location that is specified when you register the Azure environment. This is the storage location specified in Step 6 in the Register an Azure environment topic. Also see ADLS Gen2 and managed identities for information about storage accounts for Azure environments. See Minimal setup for cloud storage for further details. These references are in the Management Console documentation.

5. List of required resources for Azure environments

Azure environments used for the Data Warehouse service must have the following resources available in the specific Azure region where the environment is registered. Currently, there is no cross-regional support for Cloudera Data Warehouse service.

6. Azure subscription should be in a similar region as the resources

Ensure that your Azure subscription is in a relatively similar region as the region where your resources are deployed. Particularly, be careful that the regions are governed by the same regulatory laws. For more information, see Azure region requirements in the Management Console documentation. In that topic it specifies that "CDP requires that the ADLS Gen2 storage location provided during environment registration must be in the same region as the region selected for the environment." In addition, please review Azure geographies in the Microsoft documentation.