Configuring KafkaSpout to Connect to a Secure Kafka Cluster
To connect to a Kerberized Kafka topic:
Code: Add
spoutConfig.securityProtocol=PLAINTEXTSASL
to your Kafka Spout configuration.Configuration: Add a KafkaClient section (excerpted from
/usr/hdp/current/kafka-broker/config/kafka_jaas.conf
) to/usr/hdp/current/storm-supervisor/conf/storm_jaas.conf
:KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true renewTicket=true serviceName="kafka"; } ; ...
Setup: Add a Kafka ACL for the topic:
/usr/hdp/current/kafka-broker/bin/kafka-acls.sh --topic TEST --add allowhosts * --allowprincipals "User:stormusr" --operations DESCRIBE,READ --config /usr/hdp/current/kafka-broker/config/server.properties