Configuring multiple listeners

Kafka brokers can simultaneously listen to connection requests on multiple ports with various protocols. By default Cloudera Manager automatically configures the ports and the protocols used by the brokers to listen to requests. However, manual configuration is possible and may be required in an advanced deployment.

Kafka Brokers support listening for connections on multiple ports with different protocols. For example, a broker is capable of listening on port 9092 for PLAINTEXT requests and 9093 using TLS/SSL simultaneously. Which ports a broker listens to is controlled by the listeners Kafka broker property.

In Cloudera Manager, the listeners property is not available directly for configuration. Instead, it is set by Cloudera Manager automatically based on how other security properties are configured.

For example, if TLS/SSL is enabled, Enable TLS/SSL for Kafka Broker is set to true, Cloudera Manager automatically configures the Kafka broker to listen to TLS/SSL requests on port 9093. That is, the listeners property is set to SSL://[***KAFKA BROKER FQDN***]:9093.

It is usually sufficient to rely on Cloudera Manager to automatically configure listeners for you. However, in a deployment where the clients use various protocols and ports to establish a connection with the broker, you need to configure the broker so that it listens on all ports and with all protocols used by the clients. This configuration has to be done manually with the help of an advanced configuration snippet.

Complete the following steps to manually configure listeners for Kafka brokers.

  1. In Cloudera Manager, select the Kafka service.
  2. Go to Configuration.
  3. Find the Kafka Broker Advanced Configuration Snippet (Safety Valve) for property.
  4. Configure listeners.
    For example:
    listeners=SASL_SSL://[***KAFKA BROKER FQDN***]:9093,SSL://[***KAFKA BROKER FQDN***]:9094
    This example shows a configuration where the broker is accepting both SASL_SSL and SSL requests.
  5. Click Save Changes.
  6. Restart the Kafka service.
The Kafka broker now listens for connection requests on the configured ports with the configured protocols.