HTTPS Communication in Cloudera Manager
Both the Cloudera Manager Agent and the roles that make up the Cloudera Management Service participate in HTTPS communication with Cloudera Manager and CDH services. This topic aims to explain how the how the various aspects of HTTPS communication are handled by the Cloudera Manager Agents and the Cloudera Management Service roles.
Cloudera Manager Agents communicate with a number of the local running CDH services over HTTPS to collect monitoring data. As of Cloudera Manager 5.1, monitoring data for HBase, HDFS, MapReduce and YARN can also be collected over HTTPS. Previous releases only supported monitoring Impala daemons using HTTPS.
Cloudera Manager Agent
The process of configuring TLS communication between the Cloudera Manager Server and Agents is outlined in Configuring TLS Security for Cloudera Manager. The process is identical to that for previous releases except for one new parameter. You can now configure the certificates available for server certificate verification using the verify_cert_dir parameter in the Agent's config.ini file. See the comments in the config.ini file for a detailed explanation of this property. This change is strictly additive. The existing verify_cert_file parameter can still be used.
The following points outline the Cloudera Manager Agent's behavior when it communicates with CDH services using HTTPS.
- If verify_cert_file or verify_cert_dir are configured in the Agent config.ini, the Agent uses these settings to verify the server certificates. If these settings are not configured, no certificate verification occurs. If certificate verification is performed for the Cloudera Manager Server, it must also be performed for CDH daemons.
- An Agent never participates in mutual TLS authentication with any CDH service. Instead each service has it's own authentication scheme. For most services this is Kerberos authentication, while Impala uses HTTP digest.
User Impact
This depends on how you are using certificates.
- If you are not interested in certificate verification, do not configure verify_cert_file or verify_cert_dir. Remember that this leaves you vulnerable to man-in-the-middle attacks.
- If you are using a CA-signed certificate, configure the Agent accordingly. Adding new services or enabling SSL/TLS on a service will not require any changes to the Agent configuration since the CA should be able to verify the certificates used by any new servers you bring online.
- If you are using self-signed certificates, then the addition of each new service that uses HTTPS will require making the certificate available to the Agent. This will involve modifying the file pointed to by verify_cert_file (agent restart required), or the directory pointed to by verify_cert_dir, to contain the new certificate.
Cloudera Management Services
A number of Cloudera Management Service roles act as HTTPS clients in communications with Cloudera Manager entities and CDH services.
- You can configure a truststore through Cloudera Manager to perform certificate verification on the certificates of the various servers it communicates with. If this truststore is
configured, it is used to verify server certificates.
OR
- If no truststore is configured through Cloudera Manager, the default Java truststore (cacerts) will be used to verify certificates.
| Roles as HTTPS Clients | Communicating HTTPS Servers |
|---|---|
| Activity Monitor |
|
| Host Monitor |
|
| Service Monitor |
|
| Event Server |
|
| Reports Manager |
|
The Cloudera Management Service roles behave as follows when communicating using HTTPS:
- If the Cloudera Management Service's SSL Client Truststore File Location parameter is configured, the roles will use this truststore to perform certificate verification on the server certificates. If this parameter is not set, certificate verification will be performed using the default Java truststore. This means that it is not possible, without the use of safety valves, to perform certificate verification for some Cloudera Management Service roles and not for others. Nor is it possible to perform certificate verification for only a subset of the HTTPS communication by a role.
- The Cloudera Management Service roles never participate in mutual TLS authentication with any CDH service or with the Cloudera Manager Server. Instead each service has it's own authentication scheme. For most services this is Kerberos authentication, while Impala uses HTTP digest. For the Cloudera Manager Server, this is session-based authentication.
User Impact
This depends on how you are using certificates:
- If you are using a CA-signed certificate, configure the Cloudera Management Service's SSL Client Truststore File Location parameter to point to a truststore that contains the CA certificate. Adding a new service or enabling TLS on an existing service will not require any changes to the Cloudera Management Service configuration since the CA certificate should be able to verify the certificates used by any new servers you bring online. Alternatively, this CA-signed certificate may be added to the default Java truststore.
- If you are using self-signed certificates, the addition of each new server using HTTPS will require making the certificate available. This will involve modifying the truststore pointed
to by the Cloudera Management Service's SSL Client Truststore File Location parameter. Truststore changes will be needed on each host on which a Cloudera Management
Service daemon is running. Changes to the truststore will not require a role restart, and should be picked up within 10 seconds by default.
If the Cloudera Management Service's SSL Client Truststore File Location is not in use, the certificate must be made available in the default Java truststore. The Cloudera Management Service role will need to be restarted for this change to take effect.