AWS outbound network access destinations
If you have limited outbound internet access (for example due to using a firewall or proxy), review this content to learn which specific outbound destinations must be available in order to register a CDP environment.
The following list includes general destinations as well as AWS-specific destinations.
General endpoints
Description/Usage |
CDP service |
Destination |
Protocol and Authentication |
IP Protocol/Port |
Comments |
---|---|---|---|---|---|
Cloudera CCMv1 Persistent Control Plane connection |
All services |
*.ccm.cdp.cloudera.com 44.234.52.96/27 |
SSH public/private key authentication |
TCP/6000-6049 |
One connection per cluster configured; persistent |
Cloudera CCMv2 Persistent Control Plane connection |
All services |
US-based Control Plane: *.v2.us-west-1.ccm.cdp.cloudera.com 35.80.24.128/27 35.166.86.177/32 52.36.110.208/32 52.40.165.49/32 EU-based Control Plane: *.v2.ccm.eu-1.cdp.cloudera.com 3.65.246.128/27 AP-based Control Plane: *.v2.ccm.ap-1.cdp.cloudera.com 3.26.127.64/27 |
HTTPS with mutual authentication |
TCP/443 |
Multiple long-lived/persistent connections |
Cloudera Databus Telemetry, billing and metering data |
All services |
US-based Control Plane: dbusapi.us-west-1.sigma.altus.cloudera.com https://cloudera-dbus-prod.s3.amazonaws.com EU-based Control Plane: api.eu-1.cdp.cloudera.com https://mow-prod-eu-central-1-sigmadbus-dbus.s3.eu-central-1.amazonaws.com https://mow-prod-eu-central-1-sigmadbus-dbus.s3.amazonaws.com AP-based Control Plane:api.ap-1.cdp.cloudera.com https://mow-prod-ap-southeast-2-sigmadbus-dbus.s3.ap-southeast-2.amazonaws.com https://mow-prod-ap-southeast-2-sigmadbus-dbus.s3.amazonaws.com |
HTTPS with Cloudera-generated access key for dbus HTTPS for S3 |
TCP/443 |
Regular interval for telemetry, billing, metering services, and used for Cloudera Observability if enabled. Larger payloads are sent to a Cloudera managed S3 bucket. |
Cloudera Manager parcels Software distribution |
All services | archive.cloudera.com |
HTTPS |
TCP/443 |
Cloudera’s public software repository. CDN backed service; IP range not predictable. |
Control Plane API |
All services | US-based Control Plane: api.us-west-1.cdp.cloudera.com EU-based Control Plane: api.eu-1.cdp.cloudera.comAP-based Control Plane: api.ap-1.cdp.cloudera.com |
HTTPS with Cloudera-generated access key |
TCP/443 |
Cloudera’s control plane REST API. |
RPMs
Cloudera RPMs for workload agents |
All services | cloudera-service-delivery-cache.s3.amazonaws.com | HTTPS | TPC/443 | RPM packages for some workload components |
Docker Images Software Distribution |
Data Engineering DataFlow Machine Learning |
container.repository.cloudera.com docker.repository.cloudera.com |
HTTPS |
TCP/443 |
Cloudera’s public docker registry. CDN backed service; IP range not predictable. |
Docker Images Software Distribution |
Data Engineering DataFlow Data Warehouse Machine Learning |
container.repo.cloudera.com US-based Control Plane: prod-us-west-2-starport-layer-bucket.s3.us-west-2.amazonaws.com prod-us-west-2-starport-layer-bucket.s3.amazonaws.com s3-r-w.us-west-2.amazonaws.com *.execute-api.us-west-2.amazonaws.com EU-based Control Plane: prod-eu-west-1-starport-layer-bucket.s3.eu-west-1.amazonaws.com prod-eu-west-1-starport-layer-bucket.s3.amazonaws.com s3-r-w.eu-west-1.amazonaws.com *.execute-api.eu-west-1.amazonaws.com AP-based Control Plane: prod-ap-southeast-1-starport-layer-bucket.s3.ap-southeast-1.amazonaws.com prod-ap-southeast-1-starport-layer-bucket.s3.amazonaws.com s3-r-w.ap-southeast-1.amazonaws.com *.execute-api.ap-southeast-1.amazonaws.com |
HTTPS |
TCP/443 |
Moved to container.repo.cloudera.com container.repo.cloudera.com uses ECR which requires S3 URLs. |
Docker Images Software Distribution |
Data Warehouse |
auth.docker.io* cloudera-docker-dev.jfrog.io* docker-images-prod.s3.amazonaws.com* gcr.io* k8s.gcr.io* quay-registry.s3.amazonaws.com* quay.io* quayio-production-s3.s3.amazonaws.com* docker.io* production.cloudflare.docker.com* storage.googleapis.com* |
HTTPS |
TCP/443 | These endpoints are required only for old/existing Data Warehouse environments. |
Flow definitions CDP AWS bucket with flow definitions |
DataFlow |
US-based Control Plane: *.s3.us-west-1.amazonaws.com EU-based Control Plane: *.s3.eu-central-1.amazonaws.com AP-based Control Plane: *.s3.ap-southeast-2.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Outbound internet access to S3 hosts is necessary on all cloud providers when using CDF as the workload needs to query outbound to an S3 location to retrieve the flow definition when creating a deployment. |
Public Signing Key Retrieval |
Data Engineering DataFlow |
US-based Control Plane: consoleauth.altus.cloudera.com console.us-west-1.cdp.cloudera.com EU-based Control Plane: console.eu-1.cdp.cloudera.comAP-based Control Plane: console.ap-1.cdp.cloudera.com |
HTTPS |
TCP/443 |
Required to allow authentication to CDE virtual Cluster using a CDP Access Key. |
SQL Stream Builder PostgreSQL driver install |
Data Hub: Streaming Analytics clusters |
pypi.org |
HTTPS |
TCP/443 |
SQL Stream Builder depends on the python3 PostgreSQL driver. This is only required for Runtime versions 7.2.11, 7.2.12 and 7.2.13. |
Control plane IAM API |
Machine learning |
US-based Control Plane: iamapi.us-west-1.altus.cloudera.com EU-based Control Plane: console.eu-1.cdp.cloudera.com AP-based Control Plane: console.ap-1.cdp.cloudera.com |
HTTPS |
TCP/443 |
For connecting to the IAMAPI for fetching the entitlement details. |
AMPs Applied ML Prototypes |
Machine Learning |
https://raw.githubusercontent.com https://github.com |
HTTPS |
TCP/443 |
Files for AMPs are hosted on GitHub. |
Learning Hub |
Machine Learning |
https://github.com/cloudera/learning-hub-content |
HTTPS |
TCP/443 |
Access Learning Hub in air-gapped environments |
AWS-specific endpoints
Description/Usage |
CDP service |
Destination |
Protocol and Authentication |
IP Protocol/Port |
Comments |
---|---|---|---|---|---|
AWS STS |
All services |
sts.amazonaws.com sts.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
CDP 7.1.1+ required before can be made internal with VPC endpoints. |
AWS S3 |
All services |
*.s3.amazonaws.com *.s3.*.amazonaws.com s3.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS RDS |
All services | *.*.rds.amazonaws.com |
JDBC / Postgres binary protocol / MySQL / RDS CA certs |
TCP 5432 / 3306 / 443 |
VPC Internal. Only Data Engineering uses MySQL and requires port 3306 to be open. |
AWS EC |
DataFlow Data Warehouse Machine Learning |
api.ecr.*.amazonaws.com *.dkr.ecr.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EC2 |
DataFlow Data Warehouse Machine Learning Operational Database |
ec2.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EKS |
Data Engineering DataFlow Data Warehouse Machine Learning |
eks.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS does not support EKS VPC endpoints at this time. |
AWS Cloudformation |
DataFlow Data Warehouse Machine Learning |
cloudformation.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS Autoscaling |
Data Engineering Data Warehouse Machine Learning |
autoscaling.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EFS |
Data Engineering Data Warehouse Machine Learning |
elasticfilesystem.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS ELB |
Data Engineering DataFlow Data Warehouse |
elasticloadbalancing.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Can be made internal with VPC endpoints. |
AWS EKS k8s cluster API |
Data Warehouse |
UNIQUEID.*.eks.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
Optional for new clusters. |
AWS RDS API |
Data Warehouse |
rds.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS does not support RDS API VPC endpoints at this time. This requirement is under further evaluation. Data Warehouse uses Amazon RDS for PostgreSQL. |
AWS Service Quotas |
Data Warehouse |
servicequotas.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS does not support Service Quota via VPC endpoints. Used to check limits and warn prior to hitting the limits. |
AWS Price List Service |
Data Warehouse |
pricing.*.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |
AWS Price List Service uses us-east-1 or ap-south-1 as the region. |
Flow definitions storage |
DataFlow |
US-based Control Plane: s3.us-west-2.amazonaws.com/dfx-flow-artifacts.mow-prod.mow-prod.cloudera.com EU-based Control Plane: cldr-mow-prod-eu-central-1-dfx-flow-artifacts.s3.eu-central-1.amazonaws.com AP-based Control Plane:cldr-mow-prod-ap-southeast-2-dfx-flow-artifacts.s3.ap-southeast-2.amazonaws.com |
HTTPS (one way) IAM authentication |
TCP/443 |