This is the documentation for Cloudera Manager 5.1.x. Documentation for other versions is available at Cloudera Documentation.

HTTPS Communication in Cloudera Manager

Both the Cloudera Manager Agent and the daemons that make up the Cloudera Management Service participate in HTTPS communication with Cloudera Manager and CDH components. This topic aims to explain how the how the various aspects of HTTPS communication are handled by the Cloudera Manager agents and the Cloudera Management Services daemons.

Cloudera Manager Agents communicate with a number of the local running CDH services over HTTPS to collect monitoring data. As of Cloudera Manager 5.1.x, monitoring data for HBase, HDFS, MapReduce and YARN can also be collected over HTTPS. Previous releases only supported monitoring Impala daemons using HTTPS.

Cloudera Manager Agent

The process of configuring TLS communication between the Cloudera Manager Server and agents is outlined here. The process is identical to that for previous releases except for one new parameter. You can now configure the certificates available for server certificate verification using the verify_cert_dir parameter in the Agent's config.ini file. See the comments in the config.ini file for a detailed explanation of this property. This change is strictly additive. The existing verify_cert_file parameter can still be used.

The following points outline the Cloudera Manager Agent's behavior when it communicates with CDH services using HTTPS.

  • If verify_cert_file and/or verify_cert_dir are configured in the Agent's config.ini then the Agent will use these settings to perform certificate verification on the server certificates. If these settings are not configured, no certificate verification will be performed. That is, it is not possible to perform certificate verification for the Cloudera Manager Server but not for CDH daemons.
  • An Agent never participates in mutual TLS authentication with any CDH service. Instead each service has it's own authentication scheme. For most services this is Kerberos authentication, while Impala uses HTTP digest.

User Impact

This depends on how you are using certificates.

  • If you are not interested in certificate verification, do not configure verify_cert_file or verify_cert_dir. Remember that this leaves you vulnerable to man-in-the-middle attacks.
  • If you are using a CA-signed certificate, configure the Agent accordingly. Adding new services or enabling SSL/TLS on a service will not require any changes to the Agent configuration since the CA should be able to verify the certificates used by any new servers you bring online.
  • If you are using self-signed certificates, then the addition of each new service that uses HTTPS will require making the certificate available to the Agent. This will involve modifying the file pointed to by verify_cert_file (agent restart required), or the directory pointed to by verify_cert_dir, to contain the new certificate.

Cloudera Management Services

A number of Cloudera Management Service roles act as HTTPS clients in communications with the Cloudera Management Service and other CDH entities.

As of Cloudera Manager 5.1.x, there are two ways to configure verification of server certificates.
  • You can configure a truststore through Cloudera Manager to perform certificate verification on the certificates of the various servers it communicates with. If this truststore is configured, it is used to verify server certificates.

    OR

  • If no truststore is configured through Cloudera Manager, the default Java truststore (cacerts) will be used to verify certificates.
The following table shows Cloudera Manager roles that act as HTTPS servers as other roles communicate with them as HTTPS clients. This table does not depict the entirety of the roles' communication, just communications over HTTPS.
Table 1. HTTPS Communication Between Cloudera Manager Roles
Roles as HTTPS Servers Communicating HTTPS Client(s) Roles
Activity Monitor
  • Cloudera Manager Server
  • JobTracker Web Server
  • Oozie server (may involve the Load Balancer in an HA configuration)
Host Monitor
  • Cloudera Manager Server
Service Monitor
  • Cloudera Manager Server
  • NameNode(s) Web Server
  • Impala StateStore Web Server
  • YARN ResourceManager(s) Web Server(s)
  • YARN JobHistory Web Server
  • Oozie server (directly, never through the Load Balancer)
Event Server
  • Cloudera Manager Server
Reports Manager
  • Cloudera Manager Server
  • NameNode(s) Web Servers
  Note: The Cloudera Navigator roles also act as HTTPS clients, but are outside the scope of this document.

The Cloudera Manager roles behave as follows when communicating using HTTPS:

  • If the Cloudera Management Service's SSL Client Truststore File Location parameter is configured, the roles will use this truststore to perform certificate verification on the server certificates. If this parameter is not set, certificate verification will be performed using the default Java truststore. This means that it is not possible, without the use of safety valves, to perform certificate verification for some Cloudera Management Service roles and not for others. Nor is it possible to perform certificate verification for only a subset of the HTTPS communication by a role.
  • The Cloudera Management Service roles never participate in mutual TLS authentication with any CDH service or with the Cloudera Manager Server. Instead each service has it's own authentication scheme. For most services this is Kerberos authentication, while Impala uses HTTP digest. For the Cloudera Manager Server, this is session-based authentication.

User Impact

This depends on how you are using certificates:

  • If you are using a CA-signed certificate, configure the Cloudera Manager Service's SSL Client Truststore File Location parameter to point to a truststore that contains the CA certificate. Adding a new service or enabling TLS on an existing service will not require any changes to the Cloudera Management Service configuration since the CA certificate should be able to verify the certificates used by any new servers you bring online. Alternatively, this CA-signed certificate may be added to the default Java truststore.
  • If you are using self-signed certificates, the addition of each new server using HTTPS will require making the certificate available. This will involve modifying the truststore pointed to by the Cloudera Manager Service's SSL Client Truststore File Location parameter. Truststore changes will be needed on each host on which a Cloudera Management Service daemon is running. Changes to the truststore will not require a role restart, and should be picked up within 10 seconds by default.

    If the Cloudera Manager Service's SSL Client Truststore File Location is not in use, the certificate must be made available in the default Java truststore. The Cloudera Management Service daemon will need to be restarted for this change to take effect.

Page generated September 3, 2015.