Azure outbound network access destinations

If you have limited outbound internet access (for example due to using a firewall or proxy), review this content to learn which specific outbound destinations must be available in order to register a CDP environment.

note

On Azure, it is possible to define network security groups (NSGs) based on IP addresses, but not based on hostnames. At the same time, the egress filtering requirements documented here contain not only static IP addresses but also hostnames, where the IP address may change over time. This means that it is not possible to perform egress filtering using NSGs. Instead, if you would like to add outbound network access to your allow list based on hostnames, you should use Azure firewall with FQDN filtering in network rules. For that you can use either Azure hosted DNS or a custom DNS, as explained in Use FQDN filtering in network rules. Note that interfaces configured via Azure Private Link don't need to go through a proxy.

The following list includes general destinations as well as Azure-specific destinations.

General endpoints


Description/Usage	CDP service	Destination	Protocol and Authentication	IP Protocol/Port	Comments
AMPs Applied ML Prototypes	Machine Learning	https://raw.githubusercontent.com https://github.com	HTTPS	TCP/443	Files for AMPs are hosted on GitHub.
Cloudera CCMv1 Persistent Control Plane connection	All services	*.ccm.cdp.cloudera.com 44.234.52.96/27	SSH public/private key authentication	TCP/6000-6049	One connection per cluster configured; persistent
Cloudera CCMv2 Persistent Control Plane connection	All services	US-based Control Plane: .v2.us-west-1.ccm.cdp.cloudera.com 35.80.24.128/27 35.166.86.177/32 52.36.110.208/32 52.40.165.49/32 EU-based Control Plane:* .v2.ccm.eu-1.cdp.cloudera.com 3.65.246.128/27 AP-based Control Plane:* *.v2.ccm.ap-1.cdp.cloudera.com 3.26.127.64/27	HTTPS with mutual authentication	TCP/443	Multiple long-lived/persistent connections
Cloudera Databus Telemetry, billing and metering data	All services	US-based Control Plane: dbusapi.us-west-1.sigma.altus.cloudera.com https://cloudera-dbus-prod.s3.amazonaws.com EU-based Control Plane: api.eu-1.cdp.cloudera.com https://mow-prod-eu-central-1-sigmadbus-dbus.s3.eu-central-1.amazonaws.com AP-based Control Plane: api.ap-1.cdp.cloudera.com https://mow-prod-ap-southeast-2-sigmadbus-dbus.s3.ap-southeast-2.amazonaws.com	HTTPS with Cloudera-generated access key for dbus HTTPS for S3	TCP/443	Regular interval for telemetry, billing, metering services, and used for Cloudera Observability if enabled. Larger payloads are sent to a Cloudera managed S3 bucket.
Cloudera Manager parcels Software distribution	Data Hub Data Lake Data Engineering DataFlow Operational Database	archive.cloudera.com	HTTPS	TCP/443	Cloudera’s public software repository. CDN backed service; IP range not predictable.
Control Plane API	CDP API Data Engineering Machine Learning	US-based Control Plane: api.us-west-1.cdp.cloudera.com EU-based Control Plane: api.eu-1.cdp.cloudera.com AP-based Control Plane: api.ap-1.cdp.cloudera.com	HTTPS with Cloudera-generated access key	TCP/443	Cloudera’s control plane REST API.
Docker Images Software Distribution	Data Engineering Machine Learning	container.repository.cloudera.com docker.repository.cloudera.com	HTTPS	TCP/443	Cloudera’s public docker registry. CDN backed service; IP range not predictable.
Docker Images Software Distribution	Data Engineering Data Warehouse Machine Learning	container.repo.cloudera.com .s3.<DOCKER-REGISTRY-REGION>.amazonaws.com s3-r-w.<DOCKER-REGISTRY-REGION>.amazonaws.com .execute-api.<DOCKER-REGISTRY-REGION>.amazonaws.com Additionally, the following are required only for old/existing Data Warehouse environments: auth.docker.io* cloudera-docker-dev.jfrog.io* docker-images-prod.s3.amazonaws.com* gcr.io* k8s.gcr.io* quay-registry.s3.amazonaws.com* quay.io* quayio-production-s3.s3.amazonaws.com* docker.io* production.cloudflare.docker.com* storage.googleapis.com*	HTTPS	TCP/443	Moved to container.repo.cloudera.com container.repo.cloudera.com uses ECR which requires S3 URLs. note In the listed entries, replace the `<DOCKER-REGISTRY-REGION>` with one of the following (whichever one is closest to where your environment is deployed: eu-west-1, us-west-2, ap-southeast-1.
Flow definitions CDP AWS bucket with flow definitions	DataFlow	US-based Control Plane: .s3.us-west-1.amazonaws.com EU-based Control Plane:* .s3.eu-central-1.amazonaws.com AP-based Control Plane:* *.s3.ap-southeast-2.amazonaws.com	HTTPS (one way) IAM authentication	TCP/443	Outbound internet access to S3 hosts is necessary on all cloud providers when using CDF as the workload needs to query outbound to an S3 location to retrieve the flow definition when creating a deployment.
Public Signing Key Retrieval	Data Engineering DataFlow	US-based Control Plane: consoleauth.altus.cloudera.com console.us-west-1.cdp.cloudera.com EU-based Control Plane: console.eu-1.cdp.cloudera.com AP-based Control Plane: console.ap-1.cdp.cloudera.com	HTTPS	TCP/443	Required to allow authentication to CDE virtual Cluster using a CDP Access Key.
SQL Stream Builder PostgreSQL driver install	Data Hub: Streaming Analytics clusters	pypi.org	HTTPS	TCP/443	SQL Stream Builder depends on the python3 PostgreSQL driver. This is only required for Runtime versions 7.2.11, 7.2.12 and 7.2.13.
Learning Hub	Machine Learning	https://github.com/cloudera/learning-hub-content	HTTPS	TCP/443	Access Learning Hub in air-gapped environments

Azure-specific endpoints


Description/Usage	CDP service	Destination	Protocol and Authentication	IP Protocol/Port	Comments
General Azure guidelines	All services	See Safelist the Azure portal URLs on your firewall or proxy server for Azure egress best practices.
Azure Kubernetes Services (AKS)	Data Engineering DataFlow Data Warehouse Machine Learning	See Outbound network and FQDN rules for Azure Kubernetes Service (AKS) clusters. note CDP uses AKS and has the same requirements. Therefore, your setup must fulfill the Azure EKS requirements mentioned in the linked documentation (such as port 9000). If you skip this step, your deployment will fail.
Azure Data Lake Storage Gen 2	Data Lake Data Hub Data Engineering DataFlow Operational Database	<STORAGE-ACCOUNT-NAME>.dfs.core.windows.net	HTTPS Azure authentication	TCP/443	Azure Storage VPC endpoint is required (Microsoft.Storage). note Replace the <STORAGE-ACCOUNT-NAME> with an actual storage account name.
Azure Database for Postgres	DataFlow Data Hub Data Lake Data Warehouse Machine Learning	*.postgres.database.azure.com	JDBC / Postgres binary protocol	TCP/5432	Azure SQL VPC endpoint is required (Microsoft.Sql).
ARM to manage User Assigned Managed Identities	Data Lake	management.azure.com	HTTPS Azure authentication	TCP/443	This can be allowed by using the AzureResourceManager Azure service tag. Additionally IP addresses to whitelist are available to download.
Microsoft Log Analytics	All services	.agentsvc.azure-automation.net .ods.opinsights.azure.com .oms.opinsights.azure.com .blob.core.windows.net	HTTPS Azure authentication	TCP/443	Optional, but may cause issues with Azure approved images if blocked.
Azure Database for MySQL	Data Engineering	*.mysql.database.azure.com	JDBC / Postgres binary protocol	TCP/3306	Azure Database for MySQL
Azure files	Data Engineering	*.file.core.windows.net	SMB	TCP/445	What is Azure Files?
Digicert CA Certificate	Data Engineering DataFlow	www.digicert.com cacerts.digicert.com	HTTPS Azure authentication	TCP/443	Fetching TLS CA for Azure MySQL DB secure connection