Enable Kerberos authentication for Kafka

Learn how to enable Kerberos Authentication for Kafka.

Apache Kafka supports Kerberos authentication, but it is supported only for the new Kafka Producer and Consumer APIs.

If you already have a Kerberos server, you can add Kafka to your current configuration. If you do not have a Kerberos server, install it before proceeding. See Enabling Kerberos Authentication for Runtime for detailed instructions

If you already have configured the mapping from Kerberos principals to short names using the hadoop.security.auth_to_local HDFS configuration property, configure the same rules for Kafka by adding the sasl.kerberos.principal.to.local.rules property to the Advanced Configuration Snippet for Kafka Broker Advanced Configuration Snippet using Cloudera Manager. Specify the rules as a comma separated list.

In Cloudera Manager, navigate to Kafka > Configuration.
Set SSL Client Authentication to none.
Set Inter Broker Protocol to SASL_PLAINTEXT.
Click Save Changes.
Restart the Kafka service, select Action > Restart.
Make sure that listeners = SASL_PLAINTEXT is present in the Kafka broker logs, by default in /var/log/kafka/server.log.
Create a jaas.conf file with either cached credentials or keytabs.
- To use cached Kerberos credentials, where you use kinit first, use this configuration:
```
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=true;
};
```
- If you use a keytab, use this configuration:
```
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/etc/security/keytabs/mykafkaclient.keytab"
principal="mykafkaclient/clients.hostname.com@EXAMPLE.COM";
};
```
  For more information on generating keytabs, see Get or create a Kerberos principal for each user account.
Create the client.properties file containing the following properties.
```
security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka
```
Test with the Kafka console producer and consumer.
To obtain a Kerberos ticket-granting ticket (TGT):
```
kinit user
```

Verify that the jaas.conf file is used by setting the environment.

export KAFKA_OPTS="-Djava.security.auth.login.config=/home/user/jaas.conf"

Verify that your topic exists.

kafka-topics --list --bootstrap-server anybroker:9092 --command-config client.properties

Run a Kafka console producer.

kafka-console-producer --broker-list anybroker:9092 --topic test1 --producer.config client.properties

Run a Kafka console consumer.

kafka-console-consumer --topic test1 --from-beginning --bootstrap-server anybroker:9092 --consumer.config client.properties