Cluster Connectivity Manager

Using Cluster Connectivity Manager (CCM), CDP can communicate with Data Lake and Data Hub workload clusters that are on private subnets. This functionality is available for CDP deployments on AWS, Azure, and Google Cloud.

Communication takes place over private IPs without any inbound network access rules required. CDP requires that these clusters have outbound connections to cloud provider NLBs hosted in Cloudera's multi-cloud account. Workload clusters initiate an SSH tunnel to the CDP control plane, which is then used for all communication thereafter.

For example, CDP can communicate with clusters that are on private subnets with only private IPs without any additional network configuration or setup. However, CDP requires that these clusters have outbound connections to cloud provider NLBs hosted in the Cloudera's multi-cloud account.

You can use Replication Manager with your on-premise CDH, HDP, and CDP Private Cloud Base clusters to assist with data migration and synchronization to cloud storage by first registering your cluster with classic cluster registration.

The following two diagrams illustrate CDP connectivity to customer account with and without using the reverse SSH tunnel.

The first diagram illustrates the CDP connectivity to customer account without CCM. When CDP is deployed in public mode, security groups (called firewall rules in Google Cloud) must be configured to allow inbound access to the environment from the CDP Control Plane exit IP range, in addition to end-user access rules restricting traffic to only originate from the customer’s own network.
  • This is done automatically for new networks created by CDP, so the only CIDRs required during deployment are from the customer’s own network.
  • Customer-provided security groups must be configured to include the Cloudera Control Plane CIDRs in addition to the customer’s own network CIDR.
Figure 1. Connectivity to customer account with CCM disabled

The second diagram illustrates the CDP connectivity to customer account with CCM enabled. When CCM is enabled, the traffic direction is reversed so the environment does not require inbound access from Cloudera’s network. Since in this setup, inbound traffic is only allowed on the private subnets, configuring security groups is not as critical as in the public IP mode outlined in the previous diagram; However, in case of bridged networks it may be useful to restrict access to a certain range of private IPs.

Figure 2. Connectivity to customer account with CCM enabled