Cluster Connectivity Manager
Using Cluster Connectivity Manager (CCM), CDP can communicate with Data Lake and Data Hub workload clusters that are on private subnets. This functionality is available for CDP deployments on AWS, Azure, and Google Cloud.
Communication takes place over private IPs without any inbound network access rules required. CDP requires that these clusters have outbound connections to cloud provider NLBs hosted in Cloudera's multi-cloud account. Workload clusters initiate an SSH tunnel to the CDP control plane, which is then used for all communication thereafter.
For example, CDP can communicate with clusters that are on private subnets with only private IPs without any additional network configuration or setup. However, CDP requires that these clusters have outbound connections to cloud provider NLBs hosted in the Cloudera's multi-cloud account.
You can use Replication Manager with your on-premise CDH clusters to assist with data migration and synchronization to cloud storage by first registering your cluster with Classic Cluster registration.
The following two diagrams illustrate CDP connectivity to customer account with and without using the reverse SSH tunnel.
- This is done automatically for new networks created by CDP, so the only CIDRs required during deployment are from the customer’s own network.
- Customer-provided security groups must be configured to whitelist the Cloudera Control Plane CIDRs in addition to the customer’s own network CIDR.
The second diagram illustrates the CDP connectivity to customer account with CCM enabled. When CCM is enabled, the traffic direction is reversed so the environment does not require inbound access from Cloudera’s network. Since in this setup, inbound traffic is only allowed on the private subnets, configuring security groups is not as critical as in the public IP mode outlined in the previous diagram; However, in case of bridged networks it may be useful to restrict access to a certain range of private IPs.