Step 4: Configure Kafka brokers

Learn how to configure TLS/SSL communication for Kafka brokers.

Kafka Brokers support listening for connections on multiple ports. If SSL is enabled for inter-broker communication, both PLAINTEXT and SSL ports are required.

To configure the listeners from Cloudera Manager, perform the following steps:

  1. In Cloudera Manager, go to Kafka > Instances.
  2. Go to Kafka Broker > Configurations.
  3. In the Kafka Broker Advanced Configuration Snippet (Safety Valve) for Kafka Properties, enter the following information:

    where kafka-broker-host-name is the FQDN of the broker that you selected from the Instances page in Cloudera Manager. In the above sample configurations we used PLAINTEXT and SSL protocols for the SSL enabled brokers.

    For information about other supported security protocols, seeUsing Kafka's inter-broker security.

  4. Repeat the previous step for each broker.

    The advertised.listeners configuration is needed to connect the brokers from external clients.

  5. Deploy the above client configurations and rolling restart the Kafka service from Cloudera Manager.

Kafka CSD auto-generates listeners for Kafka brokers, depending on your SSL and Kerberos configuration. To enable SSL for Kafka installations, do the following:

  1. Turn on SSL for the Kafka service by turning on the ssl_enabled configuration for the Kafka CSD.
  2. Set as SSL, if Kerberos is disabled; otherwise, set it as SASL_SSL.

The following SSL configurations are required on each broker. Each of these values can be set in Cloudera Manager. Be sure to replace this example with the truststore password.

For instructions, see Changing the Configuration of a Service or Role Instance.


Other configuration settings may also be needed, depending on your requirements:

  • ssl.client.auth=none: Other options for client authentication are required, or requested, where clients without certificates can still connect. The use of requested is discouraged, as it provides a false sense of security and misconfigured clients can still connect.
  • ssl.cipher.suites: A cipher suite is a named combination of authentication, encryption, MAC, and a key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. This list is empty by default.
  • ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1: Provide a list of SSL protocols that your brokers accept from clients.
  • ssl.keystore.type=JKS
  • ssl.truststore.type=JKS

Communication between Kafka brokers defaults to PLAINTEXT. To enable secured communication, modify the broker properties file by adding

For a list of the supported communication protocols, see Using Kafka's inter-broker security.

After SSL is configured your broker, logs should show an endpoint for SSL communication:

with addresses: PLAINTEXT -> EndPoint(,9092,PLAINTEXT),SSL -> EndPoint(,9093,SSL)

You can also check the SSL communication to the broker by running the following command:

openssl s_client -debug -connect localhost:9093 -tls1

This check can indicate that the server keystore and truststore are set up properly.

The output of this command should show the server certificate:

{variable sized random bytes}
subject=/C=US/ST=CA/L=Palo Alto/O=org/OU=org/CN=Franz Kafka
issuer=/C=US/ST=CA/L=Palo Alto

If the certificate does not appear, or if there are any other error messages, your keystore is not set up properly.