HTTPS Communication in Cloudera Manager

Both the Cloudera Manager Agent and the roles that make up the Cloudera Management Service use HTTPS to communicate with Cloudera Manager and CDH services. This topic aims to explain how the various aspects of HTTPS communication are handled by the Cloudera Manager Agents and the Cloudera Management Service roles.

Cloudera Manager Agents use HTTPS to communicate with HBase, HDFS,Impala, MapReduce, and YARN to collect monitoring data.

Cloudera Manager Agent

Configuring TLS communication between the Cloudera Manager Server and Agents is outlined in Configuring TLS Security for Cloudera Manager. You can configure the certificates available for server certificate verification using the verify_cert_dir parameter in the Agent config.ini file. See the comments in the config.ini file for a detailed explanation of this property. You can also use the existing value for the verify_cert_file parameter.

When the Cloudera Manager Agent communicates with CDH services using HTTPS:

  • If verify_cert_file or verify_cert_dir are configured in the Agent config.ini, the Agent uses these settings to verify the server certificates. If these settings are not configured, no certificate verification occurs. If certificate verification is performed for the Cloudera Manager Server, it must also be performed for CDH daemons.
  • An Agent never participates in mutual TLS authentication with any CDH service. Instead, each service has its own authentication scheme. Most services use Kerberos authentication, but Impala uses HTTP digest.

User Impact

This depends on how you use certificates.

  • If you do not need certificate verification, do not configure verify_cert_file or verify_cert_dir. However, this leaves you vulnerable to man-in-the-middle attacks.
  • If you are using a CA-signed certificate, configure the Agent accordingly. Adding new services or enabling SSL/TLS on a service requires no changes to the Agent configuration because the CA verifies the certificates used by any new servers brought online.
  • If you are using self-signed certificates, the certificate for each new service that uses HTTPS must be available to the Agent. Modify the file pointed to by verify_cert_file (Agent restart required), or the directory pointed to by verify_cert_dir, to contain the new certificate.

Cloudera Management Services

Some Cloudera Management Service roles act as HTTPS clients when communicating with Cloudera Manager entities and CDH services.

You can verify server certificates in two ways:
  • Configure a truststore through Cloudera Manager to perform certificate verification on the certificates of the servers with which it communicates. If this truststore is configured, it is used to verify server certificates.

    OR

  • If no truststore is configured through Cloudera Manager, the default Java truststore (cacerts) is used to verify certificates.
The following table shows Cloudera Management Service roles that act as HTTPS clients as Cloudera Manager entities, and CDH services that communicate with them as HTTPS servers.This table does not depict the entirety of the roles' communication, only communications over HTTPS.
HTTPS Communication Between Cloudera Management Service Roles and Cloudera Manager Entities
Roles as HTTPS Clients Communicating HTTPS Servers
Activity Monitor
  • Cloudera Manager Server
  • JobTracker Web Server
  • Oozie server (may involve the load balancer in an HA configuration)
Host Monitor
  • Cloudera Manager Server
Service Monitor
  • Cloudera Manager Server
  • NameNode(s) Web Server(s)
  • Impala StateStore Web Server
  • YARN ResourceManager(s) Web Server(s)
  • YARN JobHistory Web Server
  • Oozie server (directly, not through the load balancer)
Event Server
  • Cloudera Manager Server
Reports Manager
  • Cloudera Manager Server
  • NameNode(s) Web Servers

The Cloudera Management Service roles communicate using HTTPS as follows:

  • If the Cloudera Management Service SSL Client Truststore File Location parameter is configured, the roles use this truststore to verify server certificates. If this parameter is not set, the default Java truststore is used to verify certificates. Without using safety valves, you cannot verify certificates for some Cloudera Management Service roles but not for others. Nor can you verify certificates for only a subset of the HTTPS communication by a role.
  • The Cloudera Management Service roles never participate in mutual TLS authentication with any CDH service or with the Cloudera Manager Server. Instead, each service has its own authentication scheme: Kerberos for most services, HTTP digest for Impala. For the Cloudera Manager Server, this authentication is session-based.

User Impact

This depends on how you use certificates:

  • If you use a CA-signed certificate, configure the Cloudera Management Service SSL Client Truststore File Location parameter to point to a truststore that contains the CA certificate. Adding a new service or enabling TLS on an existing service requires no changes to the Cloudera Management Service configuration because the CA certificate verifies the certificates used by any new servers brought online. Alternatively, this CA-signed certificate can be added to the default Java truststore.
  • If you are using self-signed certificates, the certificate for each new service that uses HTTPS must be available to the Agent.. You must modify the truststore pointed to by the Cloudera Management Service SSL Client Truststore File Location parameter. Truststore changes are required on each host on which a Cloudera Management Service daemon is running. Changes to the truststore do not require a role restart, and should be picked up within 10 seconds by default.

    If the Cloudera Management Service SSL Client Truststore File Location is not used, the certificate must be made available in the default Java truststore. The Cloudera Management Service role must be restarted for this change to take effect.