Cloudera Docs

DNS

This topic covers recommended DNS configurations for on AWS.

The previous sections dealt with how connectivity is established to the workload infrastructure. This section deals with ‘addressability’. The workloads launched by contain a few services that need to be accessed by the admins or data consumers. These include services like , metadata services like the Hive Metastore, Atlas or Ranger, data processing or consumption services such as Oozie server, Hue, and so on. Given the nature of the cloud infrastructure, the IP addresses for the nodes running these services may change (for example, if the infrastructure is restarted or repaired). However, these should have statically addressable DNS names so that users can access them with the same names.

In order to help with this, assigns DNS names to these nodes. These naming schemes have the following properties:

  • The DNS name is of the following format for each Data Lake node, node, and the Data Lake/ cluster endpoint: <CLUSTER_NAME>-{<HOST_GROUP><i>}.<ENVIRONMENT_IDENTIFIER>.<CUSTOMER_IDENTIFIER>.cloudera.site

    An example could be my-dataeng-master0.my-envir.aaaa-1234.cloudera.site

    This name has the following components:
    • The base domain is cloudera.site. This is a publicly registered DNS suffix (see Public Suffix List). It is also a registered Route53 hosted zone in a owned AWS account.
    • The <CUSTOMER_IDENTIFIER> is unique to a customer account on made of alphanumeric characters and "-"
    • The <ENVIRONMENT_IDENTIFIER> is generated based on the environment name and is truncated to 8 characters.
    • The <CLUSTER_NAME> is the cluster name given to the Data Lake or . It is appended with a <HOST_GROUP> name such as "gateway", "master", "worker", and so on, depending on the role that the node plays in the cluster. If there are more than one of these nodes playing the same role, they are appended with a serial number, <i>.
  • The DNS name of the endpoints of the data services is of the following format:
    • For a Virtual Warehouse in , it is <VIRTUAL_WAREHOUSE_NAME>.<CDW_ENVIRONMENT_IDENTIFIER>.dw.<CUSTOMER_IDENTIFIER>.cloudera.site
      • The <VIRTUAL_WAREHOUSE_NAME> is the name of the Virtual Warehouse created. There could be multiple virtual warehouses for a given environment.
      • The <CDW_ENVIRONMENT_IDENTIFIER> is the identifier for the environment.
    • For a Session Terminal in a workspace, it is <TTY_SESSION_ID>.<CML_WORKSPACE_ID>.<ENVIRONMENT_IDENTIFIER>.<CUSTOMER_IDENTIFIER>.cloudera.site
      • The <TTY_SESSION_ID> is the ID of the Terminal Session ID.
      • The <CML_WORKSPACE_ID> is the ID of the workspace created.
      • The <ENVIRONMENT_IDENTIFIER> is generated based on the environment name and is truncated to 8 characters. If the 8th character is a "-" (dash), then it is truncated to 7 characters instead.
    • For all the data services listed above, the common portions of the DNS include.
      • The base domain is cloudera.site. This is a publicly registered DNS suffix (see Public Suffix List). It is also a registered Route53 hosted zone in a owned AWS account.
      • The <CUSTOMER_IDENTIFIER> is unique to a customer account on made of alphanumeric characters and a "-" (dash).
    • For a virtual cluster in , it is <VIRTUAL_CLUSTER_ID>.<CDE_SERVICE_ID>.<ENVIRONMENT_IDENTIFIER>.<CUSTOMER_IDENTIFIER>.cloudera.site
      • The <VIRTUAL_CLUSTER_ID> is the 8-character ID of the virtual cluster, for example, afg57p98.
      • The <CDE_SERVICE_ID> is the ID of the service containing the virtual cluster, for example, cde-g6th4kjv.
      • The <ENVIRONMENT_IDENTIFIER> is generated based on the environment name and is truncated to 8 characters. if the 8th character is a "-" (dash), then it is truncated to 7 characters instead.
    • For a DataFlow service in , it is

      dfx.<CDF_WORKLOAD_ENDPOINT_ID>.<CUSTOMER_IDENTIFIER>.cloudera.site

      • The <CDF_WORKLOAD_ENDPOINT_ID> is the 8-character ID of the Service Workload Endpoint, for example, 1bxt50kk.
    • For a database in , it is <COD_WORKLOAD_NAME>-{<HOST-GROUP><i>}.<ENVIRONMENT_IDENTIFIER>.<CUSTOMER_IDENTIFIER>.cloudera.site
      • <COD_Workload_Name> is the ID of the , for example, cod-1m6yz9uwqhrg2.
      • The user provides a database name and the environment where they want to create the database. These two entities are hashed together to create the internal <COD_WORKLOAD_NAME>, which is set as the cluster.
      • Except for the <COD_WORKLOAD_NAME>, the rest of the DNS name of the endpoint is implemented as per DNS format as mentioned above.
    • For all the data services listed above, the common portions of the DNS include:
      • The base domain is cloudera.site. This is a publicly registered DNS suffix. It is also a registered Route53 hosted zone in a owned AWS account.
      • The <CUSTOMER_IDENTIFIER> is unique to a customer account on made of alphanumeric characters and a "-" (dash).
  • The length of the DNS name is restricted to 64 characters due to some limitations with Hue workloads.
  • These names are stored as A records in the Route53 hosted zone in the managed AWS account. Hence, you can resolve these names from any location outside of the VPC. However, note that they would still resolve to private IP addresses and hence are constrained by the connectivity setup described in preceding sections.
  • Within a environment, the DNS resolution happens differently. Every environment has a DNS server that is played by a component called FreeIPA. This server is seeded with the hostnames of the nodes of all workload clusters in the environment. Every node in a Data Lake, , and a data service is configured to look up the FreeIPA DNS service for name resolution within the cluster.