Outbound network access destinations
Review this section to learn which specific outbound destinations must be available in order to register a CDP environment in an environment with limited outbound network access.
If you have limited outbound internet access (for example due to using a firewall or proxy), review this document to learn which specific outbound destinations must be available in order to register a CDP environment. This document lists:
- General endpoints applicable to all CDP environments
- AWS-specific endpoints
- Azure-specific endpoints
General endpoints
| Description/Usage | CDP service | Destination | Protocol and Authentication | IP Protocol/Port | Comments |
|---|---|---|---|---|---|
| Cloudera CCM Persistent Control Plane connection |
All services | IP: 44.234.52.96/27 Ports: 6000-6049 Hostname pattern: *.ccm.cdp.cloudera.com |
SSH public/private key authentication | TCP/6000-6049 | One connection per cluster configured; persistent. |
| Cloudera Databus Telemetry, billing and metering data |
All services | dbusapi.us-west-1.altus.cloudera.com dbusapi.us-west-1.sigma.altus.cloudera.com |
HTTPS with Cloudera-generated access key | TCP/443 | Regular interval for telemetry, billing, metering services, and used for Workload Manager if enabled. |
| Control Plane API | Machine Learning | api.us-west-1.cdp.cloudera.com | HTTPS with Cloudera-generated access key | TCP/443 | Cloudera’s control plane REST API. |
| Cloudera Manager parcels Software distribution |
Data Lake, Data Hub, Operational Database |
archive.cloudera.com | HTTPS | TCP/443 | Cloudera’s public software repository. CDN backed service; IP range not predictable. |
| Docker Images Software Distribution |
Data Engineering Machine Learning |
container.repository.cloudera.com docker.repository.cloudera.com |
HTTPS | TCP/443 | Cloudera’s public docker registry. CDN backed service; IP range not predictable. |
|
Docker Images Software Distribution |
Data Warehouse | container.repo.cloudera.com *.s3.<region>.amazonaws.com s3-r-w.<region>.amazonaws.com *.execute-api.<region>.amazonaws.com Additionally, the following are required only for old/existing DW environments: auth.docker.io* cloudera-docker-dev.jfrog.io* docker-images-prod.s3.amazonaws.com* gcr.io* k8s.gcr.io* quay-registry.s3.amazonaws.com* quay.io* quayio-production-s3.s3.amazonaws.com* docker.io* production.cloudflare.docker.com* storage.googleapis.com* |
HTTPS | TCP/443 | Moved to container.repo.cloudera.com container.repo.cloudera.com uses ECR which requires S3 URLs. |
AWS specific endpoints
| Description/Usage | CDP service | Destination | Protocol and Authentication | IP Protocol/Port | Comments |
|---|---|---|---|---|---|
| AWS STS | Data Lake | sts.amazonaws.com sts.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 | CDP 7.1.1+ required before can be made internal with VPC endpoints. |
| AWS S3 | Data Lake, Data Hub, Data Engineering, Data Warehouse, Machine Learning, Operational Database |
*.s3.amazonaws.com *.s3.*.amazonaws.com s3.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS DynamoDB | Data Lake, Data Hub, Data Engineering, Data Warehouse, Machine Learning, Operational Database |
dynamodb.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS RDS | Data Lake, Data Hub, Data Engineering |
*.*.rds.amazonaws.com |
JDBC / Postgres binary protocol / MySQL | TCP 5432 / 3306 | VPC Internal. Only Data Engineering uses MySQL and requires port 3306 to be open. |
| AWS ECR | Data Warehouse, Machine Learning |
api.ecr.*.amazonaws.com *.dkr.ecr.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS EC2 | Data Warehouse, Machine Learning, Operational Database |
ec2.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS EKS | Data Engineering, Data Warehouse, Machine Learning |
eks.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | AWS does not support EKS VPC endpoints at this time. |
| AWS Cloudformation | Data Warehouse, Machine Learning |
cloudformation.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS Autoscaling | Data Engineering, Data Warehouse, Machine Learning |
autoscaling.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS EFS | Data Engineering, Data Warehouse, Machine Learning |
elasticfilesystem.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS EKS k8s cluster api | Data Warehouse | UNIQUEID.*.eks.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | Optional for new clusters. |
| AWS ELB | Data Engineering, Data Warehouse |
elasticloadbalancing.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | Can be made internal with VPC endpoints. |
| AWS RDS API | Data Warehouse | rds.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 |
AWS does not support RDS API VPC endpoints at this time. This requirement is under further evaluation. Data Warehouse uses Amazon RDS for PostgreSQL. |
| AWS Service Quotas | Data Warehouse | servicequotas.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | AWS does not support Service Quota via VPC endpoints. Used to check limits and warn prior to hitting the limits. |
| AWS Price List Service | Data Warehouse | pricing.*.amazonaws.com | HTTPS (one way) IAM authentication |
TCP/443 | AWS Price List Service uses us-east-1 or ap-south-1 as the region. |
Azure specific endpoints
| Description/Usage | CDP service | Destination | Protocol and Authentication | IP Protocol/Port | Comments |
|---|---|---|---|---|---|
| General Azure guidelines | All | See Safelist the Azure portal URLs on your firewall or proxy server for Azure egress best practices. | |||
| Azure Kubernetes Services (AKS) | Data Warehouse, Machine Learning | See Control egress traffic for cluster nodes in Azure Kubernetes Service (AKS). CDP uses AKS and has the same requirements. | |||
| Azure Data Lake Storage Gen 2 | Data Lake, Data Hub, Operational Database |
<storage account name>.dfs.core.windows.net |
HTTPS Azure authentication |
TCP/443 | Azure Storage VPC endpoint is required (Microsoft.Storage). |
| Azure Database for Postgres | Data Lake, Data Hub, Data Warehouse, Machine Learning |
*.postgres.database.azure.com |
JDBC / Postgres binary protocol | TCP/5432 | Azure SQL VPC endpoint is required (Microsoft.Sql). |
| ARM to manage User Assigned Managed Identities | Data Lake | management.azure.com | HTTPS Azure authentication |
TCP/443 | This can be allowed by using the AzureResourceManager Azure service tag. Additionally IP addresses to whitelist are available to download. |
| Microsoft Log Analytics | All | *.agentsvc.azure-automation.net *.ods.opinsights.azure.com *.oms.opinsights.azure.com *.blob.core.windows.net |
HTTPS Azure authentication |
TCP/443 | Optional; but may cause issues with Azure approved images if blocked. |
