AWS outbound network access destinations
If you have limited outbound internet access (for example due to using a firewall or proxy), review this content to learn which specific outbound destinations must be available in order to register a CDP environment.
The following list includes general destinations as well as AWS-specific destinations.
General endpoints
Description/Usage |
CDP service |
Destination |
Protocol and Authentication |
IP Protocol/Port |
Comments |
|---|---|---|---|---|---|
AMPs Applied ML Prototypes |
Machine Learning |
https://raw.githubusercontent.com https://github.com |
HTTPS |
TCP/443 |
Files for AMPs are hosted on GitHub. |
Cloudera CCMv1 Persistent Control Plane connection |
All services |
*.ccm.cdp.cloudera.com 44.234.52.96/27 |
SSH public/private key authentication |
TCP/6000-6049 |
One connection per cluster configured; persistent |
|
Cloudera CCMv2 Persistent Control Plane connection |
All services |
US-based Control Plane: *.v2.us-west-1.ccm.cdp.cloudera.com 35.80.24.128/27 35.166.86.177/32 52.36.110.208/32 52.40.165.49/32 EU-based Control Plane: *.v2.ccm.eu-1.cdp.cloudera.com 3.65.246.128/27 AP-based Control Plane: *.v2.ccm.ap-1.cdp.cloudera.com 3.26.127.64/27 |
HTTPS with mutual authentication |
TCP/443 |
Multiple long-lived/persistent connections |
Cloudera Databus Telemetry, billing and metering data |
All services |
US-based Control Plane: dbusapi.us-west-1.sigma.altus.cloudera.com *.s3.amazonaws.com EU-based Control Plane: api.eu-1.cdp.cloudera.com *.s3.amazonaws.com AP-based Control Plane:api.ap-1.cdp.cloudera.com *.s3.amazonaws.com |
HTTPS with Cloudera-generated access key for dbus HTTPS for S3 |
TCP/443 |
Regular interval for telemetry, billing, metering services, and used for Workload Manager if enabled. Larger payloads are sent to a Cloudera managed S3 bucket. |
Cloudera Manager parcels Software distribution |
Data Hub Data Lake Data Engineering DataFlow Operational Database |
archive.cloudera.com |
HTTPS |
TCP/443 |
Cloudera’s public software repository. CDN backed service; IP range not predictable. |
Control Plane API |
CDP API |
US-based Control Plane: api.us-west-1.cdp.cloudera.com EU-based Control Plane: api.eu-1.cdp.cloudera.comAP-based Control Plane: api.ap-1.cdp.cloudera.com |
HTTPS with Cloudera-generated access key |
TCP/443 |
Cloudera’s control plane REST API. |
Control Plane API |
Data Engineering DataFlow Machine Learning |
api.us-west-1.cdp.cloudera.com |
HTTPS with Cloudera-generated access key |
TCP/443 |
Cloudera’s control plane REST API. |
Docker Images Software Distribution |
Data Engineering DataFlow Machine Learning |
container.repository.cloudera.com docker.repository.cloudera.com |
HTTPS |
TCP/443 |
Cloudera’s public docker registry. CDN backed service; IP range not predictable. |
Docker Images Software Distribution |
Data Engineering DataFlow Data Warehouse Machine Learning |
container.repo.cloudera.com *.s3.<DOCKER-REGISTRY-REGION>.amazonaws.com s3-r-w.<DOCKER-REGISTRY-REGION>.amazonaws.com *.execute-api.<DOCKER-REGISTRY-REGION>.amazonaws.com Additionally, the following are required only for old/existing Data Warehouse environments: auth.docker.io* cloudera-docker-dev.jfrog.io* docker-images-prod.s3.amazonaws.com* gcr.io* k8s.gcr.io* quay-registry.s3.amazonaws.com* quay.io* quayio-production-s3.s3.amazonaws.com* docker.io* production.cloudflare.docker.com* storage.googleapis.com* |
HTTPS |
TCP/443 |
Moved to container.repo.cloudera.com container.repo.cloudera.com uses ECR which requires S3 URLs. |
Identity Provider Discovery |
DataFlow |
consoleauth.us-west-1.core.altus.cloudera.com |
HTTPS |
TCP/443 |
None |
Public Signing Key Retrieval |
Data Engineering DataFlow |
consoleauth.altus.cloudera.com |
HTTPS |
TCP/443 |
Required to allow authentication to CDE virtual Cluster using a CDP Access Key. |
SQL Stream Builder PostgreSQL driver install |
Data Hub: Streaming Analytics clusters |
pypi.org |
HTTPS |
TCP/443 |
SQL Stream Builder depends on the python3 PostgreSQL driver. This is only required for Runtime versions 7.2.11, 7.2.12 and 7.2.13. |
Learning Hub |
Machine Learning |
https://github.com/cloudera/learning-hub-content |
HTTPS |
TCP/443 |
Access Learning Hub in air-gapped environments |
AWS-specific endpoints
Description/Usage |
CDP service |
Destination |
Protocol and Authentication |
IP Protocol/Port |
Comments |
|---|---|---|---|---|---|
AWS STS |
Data Lake |
sts.amazonaws.com sts.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
CDP 7.1.1+ required before can be made internal with VPC endpoints. |
AWS S3 |
All services |
*.s3.amazonaws.com *.s3.*.amazonaws.com s3.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS DynamoDB |
All services |
dynamodb.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS RDS |
Data Lake Data Hub Data Engineering DataFlow |
*.*.rds.amazonaws.com |
JDBC / Postgres binary protocol / MySQL |
TCP 5432 / 3306 |
VPC Internal. Only Data Engineering uses MySQL and requires port 3306 to be open. |
AWS EC |
DataFlow Data Warehouse Machine Learning |
api.ecr.*.amazonaws.com *.dkr.ecr.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EC2 |
DataFlow Data Warehouse Machine Learning Operational Database |
ec2.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EKS |
Data Engineering DataFlow Data Warehouse Machine Learning |
eks.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS does not support EKS VPC endpoints at this time. |
AWS Cloudformation |
DataFlow Data Warehouse Machine Learning |
cloudformation.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS Autoscaling |
Data Engineering Data Warehouse Machine Learning |
autoscaling.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EFS |
Data Engineering Data Warehouse Machine Learning |
elasticfilesystem.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EKS k8s cluster api |
Data Warehouse |
UNIQUEID.*.eks.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Optional for new clusters. |
AWS ELB |
Data Engineering DataFlow Data Warehouse |
elasticloadbalancing.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS RDS API |
Data Warehouse |
rds.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS does not support RDS API VPC endpoints at this time. This requirement is under further evaluation. Data Warehouse uses Amazon RDS for PostgreSQL. |
AWS Service Quotas |
Data Warehouse |
servicequotas.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS does not support Service Quota via VPC endpoints. Used to check limits and warn prior to hitting the limits. |
AWS Price List Service |
Data Warehouse |
pricing.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS Price List Service uses us-east-1 or ap-south-1 as the region. |
