Apache Kafka Component Guide
Also available as:

Running MirrorMaker on Kerberos-Enabled Clusters

To run MirrorMaker on a Kerberos/SASL-enabled cluster, configure producer and consumer properties as follows:

  1. Choose or add a new principal for MirrorMaker. Do not use kafka or any other service accounts. The following example uses principal mirrormaker.

  2. Create client-side Kerberos keytabs for your MirrorMaker principal. For example:

    sudo kadmin.local -q "ktadd -k /tmp/mirrormaker.keytab mirrormaker/HOSTNAME@EXAMPLE.COM"
  3. Add a new Jaas configuration file to the node where you plan to run MirrorMaker:

  4. Add the following settings to the KafkaClient section of the new Jaas configuration file. Make sure the principal has permissions on both the source cluster and the target cluster.

    KafkaClient {
         com.sun.security.auth.module.Krb5LoginModule required
  5. Run the following ACL command on the source and destination Kafka clusters:

    bin/kafka-acls.sh --topic test-topic --add --allow-principal user:mirrormaker --operation ALL --config /usr/hdp/current/kafka-broker/config/server.properties
  6. In your MirrorMaker consumer.config and producer.config files, specify security.protocol=SASL_PLAINTEXT.

  7. Start MirrorMaker. Specify the new.consumer option in addition to your other options. For example:

    /usr/hdp/current/kafka-broker/bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config consumer.properties --producer.config target-cluster-producer.properties --whitelist my-topic --new.consumer