Configuring KafkaSpout to Connect to a Secure Kafka Cluster
To connect to a Kerberized Kafka topic:
Code: Add
spoutConfig.securityProtocol=PLAINTEXTSASL
to your Kafka Spout configuration.Configuration: Add a
KafkaClient
section (excerpted from/usr/hdp/current/kafka-broker/config/kafka_jaas.conf
) to/usr/hdp/current/storm-supervisor/conf/storm_jaas.conf
:KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/stormusr.service.keytab" storeKey=true useTicketCache=false serviceName="kafka" principal="stormusr/host.name@EXAMPLE.COM"; };
Setup: Add a Kafka ACL for the topic. For example:
bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal user:stormusr --allow-hosts * --operations Read --topic TEST