ZooKeeper Authentication

ZooKeeper supports mutual server-to-server (quorum peer) authentication using SASL (Simple Authentication and Security Layer), which provides a layer around Kerberos authentication. Server to server authentication among ZooKeeper servers in an ensemble mitigates the risk of spoofing by a rogue server on an unsecured network. For more information about quorum peer authentication and how the feature leverages ZooKeeper's SASL support, see the Cloudera Engineering Blog post, Hardening Apache ZooKeeper Security.

Follow the steps in Configuring ZooKeeper Server for Kerberos Authentication and Configuring ZooKeeper Client Shell for Kerberos Authentication to configure ZooKeeper to use client-to-server SASL-based authentication.

To configure client-server or server-server authentication for ZooKeeper, follow the appropriate steps below:

Requirements

Configuring ZooKeeper to use Kerberos for client-server or server-server authentication requires that your organization's Kerberos instance (MIT Kerberos, Microsoft Active Directory) be up and running, and reachable by the ZooKeeper server or client during the configuration processes detailed below.

Before enabling mutual authentication, the ZooKeeper servers in the cluster must be configured to authenticate using Kerberos.

Configuring ZooKeeper Server for Kerberos Authentication

You can configure the ZooKeeper server for Kerberos authentication in Cloudera Manager.

Using Cloudera Manager to Configure ZooKeeper Server for Kerberos Authentication

To set up the ZooKeeper server for Kerberos authentication in Cloudera Manager, complete the following steps:

  1. In Cloudera Manager, open the ZooKeeper service.
  2. Click the Configuration tab.
  3. Enter Kerberos in the in the Search bar.
  4. Find the Enable Kerberos Authentication property and select the check-box next to the ZooKeeper services that you want to configure for Kerberos authentication.

Configuring ZooKeeper Client Shell for Kerberos Authentication

In addition to configuring ZooKeeper Server hosts to use Kerberos for authentication, you should also configure the ZooKeeper client shell (the ZooKeeper CLI) to authenticate to the ZooKeeper service using Kerberos credentials. As with the ZooKeeper Server, you must create a Kerberos principal for the client, as detailed below:

  1. Create a Kerberos principal for the zookeeper-client, zkcli@YOUR-REALM, replacing YOUR-REALM with the name of your organization's Kerberos realm: 
    kadmin: addprinc -randkey zkcli@YOUR-REALM
  2. Create a keytab file for the ZooKeeper client shell using the -norandkey option. 
    $ kadmin
kadmin: xst -norandkey -k zkcli.keytab zkcli@YOUR-REALM
  3. On the host running the ZooKeeper client shell, set up JAAS (Java Authentication and Authorization Service) in the configuration directory: /etc/zookeeper/conf/.
  4. Create a jaas.conf file containing the following settings: 
    Client {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="/path/to/zkcli.keytab"
  storeKey=true
  useTicketCache=false
  principal="zkcli@YOUR-REALM";
};
  5. In this same configuration directory, add the following setting to the java.env file, creating the file if necessary. 
    export JVMFLAGS="-Djava.security.auth.login.config=/etc/zookeeper/conf/jaas.conf"

Verifying the Configuration

After enabling Kerberos authentication and restarting the ZooKeeper cluster, you can verify that the authentication is working correctly by following these steps:
  1. Start the ZooKeeper client, passing to it the name of a ZooKeeper server: 
    zookeeper-client -server fqdn.example.com:port
  2. From the ZooKeeper CLI, create a protected znode using your ZooKeeper client principal: 
    create /znode1 znode1data sasl:zkcli@{{YOUR-REALM}}:cdwra
  3. Verify the znode is created and the ACL is set correctly: 
    getAcl /znode1
The getAcl command returns the znode's scheme and permissions values. Verify that these are as expected.

Enabling Server-Server Mutual Authentication

Support for mutual authentication between ZooKeeper Servers can be enabled through the Cloudera Manager Admin Console. For secured networks, server-to-server authentication is considered an optional security enhancement, so the capability is disabled by default:


Server-to-server SASL authentication requires all servers in the ZooKeeper ensemble to authenticate using Kerberos, as detailed in Configuring ZooKeeper Server for Kerberos Authentication.

Assuming your cluster is already configured to authenticate using Kerberos, you can enable mutual authentication as follows:
  1. Log into the Cloudera Manager Admin Console.
  2. Select Clusters > ZOOKEEPER-n.
  3. Click the Configuration tab.
  4. Select Category > Security under the Filters menu to display the security properties.
  5. Click the Enable Server to Server SASL Authentication box to select it.
  6. Click Save.
  7. Select Restart from the Actions drop-down to restart the cluster with this setting.

To disable server-to-server SASL authentication, simply return to the Cloudera Manager Admin Console page shown above, de-select Enable Server to Server SASL Authentication (by clicking the checked box), and restart the cluster.