Domain names for the endpoints
The previous sections dealt with how connectivity is established to the workload infrastructure. This section deals with “addressability”.
The workloads launched by CDP contain a few services that need to be accessed by the CDP admins and data consumers. These include services such as Cloudera Manager; metadata services such as the Hive Metastore, Atlas, or Ranger; and data processing or consumption services such as Oozie server, Hue, and so on. Given the nature of the cloud infrastructure, the IP addresses for the nodes running these services may change (for example if the infrastructure is restarted or repaired). However, these should have statically addressable DNS names so that users can access them with the same names.
In order to help with this, CDP assigns DNS names to these nodes. These naming schemes have the following properties:
- The DNS name is of the following format for each Data Lake node and the Data Lake
cluster endpoint:
An example could be:<CLUSTER_NAME>-{<HOST_GROUP><i>}.<ENVIRONMENT-IDENTIFIER>.<CUSTOMER_IDENTIFIER>.cloudera.site
my-dataeng-master0.my-envir.aaaa-1234.cloudera.site
This name has the following components:
- The base domain is cloudera.site. This is a publicly registered
DNS suffix
. It is also a registered Route53 hosted zone in a Cloudera owned AWS account. - The
<CUSTOMER_IDENTIFIER>
is unique to a customer account on CDP made of alphanumeric characters and a “-” (dash). - The
<ENVIRONMENT_IDENTIFIER>
is generated based on the environment name and is truncated to 8 characters. - The
<CLUSTER_NAME>
is the cluster name given to the Data Lake or Data Hub. It is appended with a<HOST_GROUP>
name such as “gateway”, “master”, “worker”, and so on, depending on the role that the node plays in the cluster. If there are more than one of these nodes playing the same role, they are appended with a serial number<i>
.
- The base domain is cloudera.site. This is a publicly registered
- The DNS name of the endpoints of the CDP data services is of the following
format:
- For a Virtual Warehouse in CDW, it is:
<VIRTUAL_WAREHOUSE_NAME>.dw-<CDW_ENVIRONMENT_IDENTIFIER>.<CUSTOMER_IDENTIFIER>.cloudera.site
<VIRTUAL_WAREHOUSE_NAME>
is the name of the Virtual Warehouse created. There could be multiple virtual warehouses for a given CDP environment.<CDW_ENVIRONMENT_IDENTIFIER>
is the identifier for the CDP environment.
- For a Session Terminal in CML workspace, it
is:
<TTY_SESSION_ID>.<CML_WORKSPACE_ID>.<ENVIRONMENT_IDENTIFIER>.<CUSTOMER_IDENTIFIER>.cloudera.site
<TTY_SESSION_ID>
is the ID of the CML Terminal Session ID.<CML_WORKSPACE_ID>
is the ID of the CML workspace created.- The
<ENVIRONMENT_IDENTIFIER>
is generated based on the environment name and is truncated to 8 characters. If the 8th character is a “-” (dash), then it is truncated to 7 characters instead.
- For all the CDP data services listed above, the common portions of the DNS
include:
- The base domain is cloudera.site. This is a publicly registered
DNS Suffix
. It is also a registered Route53 hosted zone in a Cloudera owned AWS account. - The
<CUSTOMER_IDENTIFIER>
is unique to a customer account on CDP made of alphanumeric characters and a “-” (dash).
- The base domain is cloudera.site. This is a publicly registered
- The length of the DNS name is restricted to 64 characters due to some limitations with Hue workloads.
- These names are stored as A records in the Route53 hosted zone in the Cloudera managed CDP Control Plane AWS account. Hence, you can resolve these names from any location outside of the VPC. However, note that they would still resolve to private IP addresses and hence are constrained by the connectivity setup described in preceding sections.
- Within a CDP environment, the DNS resolution happens differently. Every CDP environment has a DNS server that is played by a component called FreeIPA. This server is seeded with the hostnames of the nodes of all workload clusters in the environment. Every node in a Data Lake, Data Hub and data service is configured to look up the FreeIPA DNS service for name resolution within the cluster.
- FreeIPA nodes are internal to the CDP environment both for resolving and use. They don't have public DNS records.
- For a Virtual Warehouse in CDW, it is: