Writing to a Secure HBase cluster

If you want to write to a secure HBase cluster, be aware of the following:

• Flume must be configured to use Kerberos security as documented above, and HBase must be configured to use Kerberos security as documented in HBase Security Configuration.
• The hbase-site.xml file, which must be configured to use Kerberos security, must be in Flume's classpath or HBASE_HOME/conf.
• HBaseSink org.apache.flume.sink.hbase.HBaseSink supports secure HBase, but AsyncHBaseSink org.apache.flume.sink.hbase.AsyncHBaseSink does not.
• The Flume HBase Sink takes these two parameters:
• kerberosPrincipal – specifies the Kerberos principal to be used
• kerberosKeytab – specifies the path to the Kerberos keytab These are defined as:

  agent.sinks.hbaseSink.kerberosPrincipal = flume/fully.qualified.domain.name@YOUR-REALM.COM
  agent.sinks.hbaseSink.kerberosKeytab = /etc/flume-ng/conf/flume.keytab

• If HBase is running with the AccessController coprocessor, the flume user (or whichever user the agent is running as) must have permissions to write to the same table and the column family that the sink is configured to write to. You can grant permissions using the grant command from HBase shell as explained in HBase Security Configuration.
• The Flume HBase Sink does not currently support impersonation; it will write to HBase as the user the agent is being run as.
• If you want to use HDFS Sink and HBase Sink to write to HDFS and HBase from the same agent respectively, both sinks have to use the same principal and keytab. If you want to use different credentials, the sinks have to be on different agents.
• Each Flume agent machine that writes to HBase (using a configured HBase sink) needs a Kerberos principal of the form:

  flume/fully.qualified.domain.name@YOUR-REALM.COM

  where fully.qualified.domain.name is the fully qualified domain name of the given Flume agent host machine, and YOUR-REALM.COM is the Kerberos realm.