Running MirrorMaker on Kerberos-Enabled Clusters
To run MirrorMaker on a Kerberos/SASL-enabled cluster, configure producer and consumer properties as follows:
Choose or add a new principal for MirrorMaker. Do not use
kafka
or any other service accounts. The following example uses principalmirrormaker
.Create client-side Kerberos keytabs for your MirrorMaker principal. For example:
sudo kadmin.local -q "ktadd -k /tmp/mirrormaker.keytab mirrormaker/HOSTNAME@EXAMPLE.COM"
Add a new Jaas configuration file to the node where you plan to run MirrorMaker:
-Djava.security.auth.login.config=/usr/hdp/current/kafka-broker/config/kafka_mirrormaker_jaas.conf
Add the following settings to the KafkaClient section of the new Jaas configuration file. Make sure the principal has permissions on both the source cluster and the target cluster.
KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/tmp/mirrormaker.keytab" storeKey=true useTicketCache=false serviceName="kafka" principal="mirrormaker/HOSTNAME@EXAMPLE.COM"; };
Run the following ACL command on the source and destination Kafka clusters:
bin/kafka-acls.sh --topic test-topic --add --allow-principal user:mirrormaker --operation ALL --config /usr/hdp/current/kafka-broker/config/server.properties
In your MirrorMaker
consumer.config
andproducer.config
files, specifysecurity.protocol=SASL_PLAINTEXT
.Start MirrorMaker. Specify the
new.consumer
option in addition to your other options. For example:/usr/hdp/current/kafka-broker/bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config consumer.properties --producer.config target-cluster-producer.properties --whitelist my-topic --new.consumer