Configuring Kerberos Authentication for HBase

Configuring Kerberos Authentication for HBase Using Cloudera Manager

Cloudera Manager simplifies the task of configuring Kerberos authentication for HBase.

Configure HBase Servers to Authenticate with a Secure HDFS Cluster Using Cloudera Manager

Minimum Required Role: Security Administrator (also provided by Full Administrator)

  1. If not done already, enable Kerberos authentication on your cluster using the Cloudera Manager wizard. This procedure will create principals for the Cloudera Manager server and associated CDH components such as HBase.
  2. Go to the HBase service.
  3. Click the Configuration tab.
  4. Select HBase Service-Wide.
  5. Select Category > Security.
  6. Ensure the Kerberos principal for the HBase service was generated. See Managing Kerberos Credentials Using Cloudera Manager.
  7. Locate the HBase Secure Authentication property or search for it by typing its name in the Search box.
  8. Select the kerberos option.
  9. Click Save Changes to commit the changes.
  10. Restart the role.
  11. Restart the service.

Configure HBase Servers and Clients to Authenticate with a Secure ZooKeeper

To run a secure HBase, you must also use a secure ZooKeeper. To use a secure ZooKeeper, each HBase host machine (Master, RegionServer, and client) must have a principal that allows it to authenticate with your secure ZooKeeper ensemble.

Cloudera Manager automatically configures the following features when you enable Kerberos authentication, and no additional configuration is required.
  • Authentication between HBase and ZooKeeper
  • Authentication for the HBase REST and Thrift gateways
  • doAs impersonation for the HBase Thrift gateway

Configuring Kerberos Authentication for HBase Using the Command Line

Configure HBase Servers to Authenticate with a Secure HDFS Cluster Using the Command Line

To configure HBase servers to authenticate with a secure HDFS cluster, you must do the following:

Enabling HBase Authentication

Set the hbase.security.authentication property to kerberos in hbase-site.xml on every host acting as an HBase master, RegionServer, or client. In CDH 5, hbase.rpc.engine is automatically detected and does not need to be set.

<property>
    <name>hbase.security.authentication</name>
    <value>kerberos</value> 
</property>

Configuring HBase Kerberos Principals

To run on a secure HDFS cluster, HBase must authenticate itself to the HDFS services. HBase acts as a Kerberos principal and needs Kerberos credentials to interact with the Kerberos-enabled HDFS daemons. You can authenticate a service by using a keytab file, which contains a key that allows the service to authenticate to the Kerberos Key Distribution Center (KDC).

  1. Create a service principal for the HBase server using the following syntax. This principal is used to authenticate the HBase server with the HDFS services. Cloudera recommends using hbase as the username. 
    $ kadmin
kadmin: addprinc -randkey hbase/fully.qualified.domain.name@YOUR-REALM.COM

    fully.qualified.domain.name is the host where the HBase server is running, and YOUR-REALM is the name of your Kerberos realm.

  2. Create a keytab file for the HBase server. 
    $ kadmin
kadmin: xst -k hbase.keytab hbase/fully.qualified.domain.name
  3. Copy the hbase.keytab file to the /etc/hbase/conf directory on the HBase server host. The owner of the hbase.keytab file should be the hbase user, and the file should have owner-only read permissions—that is, assign the file 0400 permissions and make it owned by hbase:hbase. 
    -r--------  1 hbase   hbase     1343 2012-01-09 10:39  hbase.keytab
  4. To test that the keytab file was created properly, try to obtain Kerberos credentials as the HBase principal using only the keytab file. Substitute your fully.qualified.domain.name and realm in the following command: 
    $ kinit -k -t /etc/hbase/conf/hbase.keytab hbase/fully.qualified.domain.name@YOUR-REALM.COM
  5. In the /etc/hbase/conf/hbase-site.xml configuration file on all cluster hosts running the HBase daemon, add the following lines: 
    <property>
  <name>hbase.regionserver.kerberos.principal</name> 
  <value>hbase/_HOST@YOUR-REALM.COM</value> 
</property> 

<property> 
  <name>hbase.regionserver.keytab.file</name> 
  <value>/etc/hbase/conf/hbase.keytab</value> 
</property>

<property> 
  <name>hbase.master.kerberos.principal</name> 
  <value>hbase/_HOST@YOUR-REALM.COM</value> 
</property> 

<property> 
<name>hbase.master.keytab.file</name> 
<value>/etc/hbase/conf/hbase.keytab</value> 
</property>

Configure HBase Servers and Clients to Authenticate with a Secure ZooKeeper

To run a secure HBase, you must also use a secure ZooKeeper. To use a secure ZooKeeper, each HBase host machine (Master, RegionServer, and client) must have a principal that allows it to authenticate with your secure ZooKeeper ensemble.

Configure HBase JVMs (all Masters, RegionServers, and clients) to Use JAAS

  1. On each host, set up a Java Authentication and Authorization Service (JAAS) by creating a /etc/hbase/conf/zk-jaas.conf file that contains the following: 
       Client {
      com.sun.security.auth.module.Krb5LoginModule required
      useKeyTab=true
      useTicketCache=false
      keyTab="/etc/hbase/conf/hbase.keytab"
      principal="hbase/fully.qualified.domain.name@<YOUR-REALM>";
   };
  2. Modify the hbase-env.sh file on HBase server and client hosts to include the following: 
       export HBASE_OPTS="$HBASE_OPTS -Djava.security.auth.login.config=/etc/hbase/conf/zk-jaas.conf"
   export HBASE_MANAGES_ZK=false
  3. Restart the HBase cluster.

Configure the HBase Servers (Masters and RegionServers) to Use Authentication to Connect to ZooKeeper

  1. Update your hbase-site.xml on each HBase server host with the following properties: 
    <configuration>
   <property>
      <name>hbase.zookeeper.quorum</name>
      <value>$ZK_NODES</value>
   </property>
   <property>
      <name>hbase.cluster.distributed</name>
      <value>true</value>
   </property>
</configuration>

    $ZK_NODES is the comma-separated list of hostnames of the ZooKeeper Quorum hosts that you configured according to the instructions in ZooKeeper Security Configuration.

  2. Add the following lines to the ZooKeeper configuration file zoo.cfg: 
    kerberos.removeHostFromPrincipal=true
kerberos.removeRealmFromPrincipal=true
  3. Restart ZooKeeper.

Configure Authentication for the HBase REST and Thrift Gateways

By default, the REST gateway does not support impersonation, but accesses HBase as a statically configured user. The actual user who initiated the request is not tracked. With impersonation, the REST gateway user is a proxy user. The HBase server records the actual user who initiates each request and uses this information to apply authorization.
  1. Enable support for proxy users by adding the following properties to hbase-site.xml. Substitute the REST gateway proxy user for $USER, and the allowed group list for $GROUPS. 
    <property>
  <name>hbase.security.authorization</name>
  <value>true</value>
</property>
<property>
  <name>hadoop.proxyuser.$USER.groups</name>
  <value>$GROUPS</value>
</property>
<property>
  <name>hadoop.proxyuser.$USER.hosts</name>
  <value>$GROUPS</value>
</property>
  2. Enable REST gateway impersonation by adding the following to the hbase-site.xml file for every REST gateway: 
    <property>
  <name>hbase.rest.authentication.type</name>
  <value>kerberos</value>
</property>
<property>
  <name>hbase.rest.authentication.kerberos.principal</name>
  <value>HTTP/fully.qualified.domain.name@<YOUR-REALM/value>
</property>
<property>
  <name>hbase.rest.authentication.kerberos.keytab</name>
  <value>/etc/hbase/conf/hbase.keytab</value>
</property>
  3. Add the following properties to hbase-site.xml for each Thrift gateway, replacing the Kerberos principal with a valid value: 
    <property>
  <name>hbase.thrift.keytab.file</name>
  <value>/etc/hbase/conf/hbase.keytab</value>
</property>
<property>
  <name>hbase.thrift.kerberos.principal</name>
  <value>hbase/fully.qualified.domain.name@<YOUR-REALM</value>
</property>
<property>
  <name>hbase.thrift.security.qop</name>
  <value>auth</value>
</property>
    The value for the property hbase.thrift.security.qop can be one of the following:

    • auth-conf—Authentication, integrity, and confidentiality checking

    • auth-int—Authentication and integrity checking

    • auth—Authentication checking only

  4. To use the Thrift API principal to interact with HBase, add the hbase.thrift.kerberos.principal to the acl table. For example, to provide administrative access to the Thrift API principal thrift_server, run an HBase Shell command like the following: 
    hbase> grant'thrift_server', 'RWCA'
  5. Optional: Configure HTTPS transport for Thrift by configuring the following parameters, substituting the placeholders with actual values: 
    <property>
  <name>hbase.thrift.ssl.enabled</name>
  <value>true</value>
</property>
<property>
  <name>hbase.thrift.ssl.keystore.store</name>
  <value>LOCATION_OF_KEYSTORE</value>
</property>
<property>
  <name>hbase.thrift.ssl.keystore.password</name>
  <value>KEYSTORE_PASSWORD</value>
</property>
<property>
  <name>hbase.thrift.ssl.keystore.keypassword</name>
  <value>LOCATION_OF_KEYSTORE_KEY_PASSWORD</value>
</property>

    The Thrift gateway authenticates with HBase using the supplied credential. No authentication is performed by the Thrift gateway itself. All client access through the Thrift gateway uses the gateway’s credential, and all clients have its privileges.

Configure doAs Impersonation for the HBase Thrift Gateway

doAs Impersonation provides a flexible way to use the same client to impersonate multiple principals. doAs is supported only in Thrift 1, not Thrift 2.

Enable doAs support by adding the following properties to hbase-site.xml on each Thrift gateway: 
<property>
  <name>hbase.regionserver.thrift.http</name>
  <value>true</value>
</property>
<property>
  <name>hbase.thrift.support.proxyuser</name>
  <value>true/value>
</property>

See the demo client for information on using doAs impersonation in your client applications.

Start HBase

If the configuration worked, you see something similar to the following in the HBase Master and RegionServer logs when you start the cluster:

INFO zookeeper.ZooKeeper: Initiating client connection, connectString=ZK_QUORUM_SERVER:2181 sessionTimeout=180000 watcher=master:60000
INFO zookeeper.ClientCnxn: Opening socket connection to server /ZK_QUORUM_SERVER:2181
INFO zookeeper.RecoverableZooKeeper: The identifier of this process is PID@ZK_QUORUM_SERVER
INFO zookeeper.Login: successfully logged in.
INFO client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
INFO zookeeper.Login: TGT refresh thread started.
INFO zookeeper.ClientCnxn: Socket connection established to ZK_QUORUM_SERVER:2181, initiating session
INFO zookeeper.Login: TGT valid starting at:        Sun Apr 08 22:43:59 UTC 2012
INFO zookeeper.Login: TGT expires:                  Mon Apr 09 22:43:59 UTC 2012
INFO zookeeper.Login: TGT refresh sleeping until: Mon Apr 09 18:30:37 UTC 2012
INFO zookeeper.ClientCnxn: Session establishment complete on server ZK_QUORUM_SERVER:2181, sessionid = 0x134106594320000, negotiated timeout = 180000