Inter-broker security

Kafka can expose multiple communication endpoints, each supporting a different protocol. Supporting multiple communication endpoints enables you to use different communication protocols for client-to-broker communications and inter-broker communications.

The security protocol used for inter-broker communication is controlled by the Inter Broker Protocol Cloudera Manager property. By default this property is set to INFERRED, which sets the security protocol based on how other security properties of the broker are configured.

The INFERRED setting configures the protocol according to the following logic:
Kerberos or LDAP enabled TLS/SSL enabled Protocol
YES YES SASL_SSL
YES NO SASL_PLAINTEXT
NO YES SSL
NO NO PLAINTEXT
Cloudera recommends that you use the protocol set by INFERRED. However, you can change this setting and use a specific protocol. This can be done by setting the Inter Broker Protocol property to the protocol that you want to use for inter-broker communication.
Changing the inter-broker protocol from the default is primarily useful for the following reasons:
Improving performance
Enabling TLS/SSL is known to have performance overhead. If your Kafka brokers are behind a firewall and are not susceptible to network snooping, you can consider enabling TLS/SSL for client-to-broker communication, but keep inter-broker communication set as PLAINTEXT. A configuration like this can result in a better performing Kafka cluster.
Securing a non-secure Kafka deployment without downtime
It is possible to migrate from a non-secure Kafka configuration to a secure Kafka configuration without downtime. This can be achieved with a rolling restart and by setting the inter-broker protocol to a protocol that is supported by all brokers until all brokers are updated to support the new protocol. For example, if you have a Kafka cluster that needs to be configured to enable Kerberos without downtime, you would take the following steps:
  1. Set inter-broker protocol to PLAINTEXT.
  2. Update the Kafka service configuration to enable Kerberos.
  3. Perform a rolling restart.
  4. Set inter-broker protocol to SASL_PLAINTEXT.