Flume Authentication
Flume agents have the ability to store data on an HDFS filesystem configured with Hadoop security. The Kerberos system and protocols authenticate communications between clients and services. Hadoop clients include users and MapReduce jobs on behalf of users, and the services include HDFS and MapReduce. Flume acts as a Kerberos principal (user) and needs Kerberos credentials to interact with the Kerberos security-enabled service. Authenticating a user or a service can be done using a Kerberos keytab file. This file contains a key that is used to obtain a ticket-granting ticket (TGT). The TGT is used to mutually authenticate the client and the service using the Kerberos KDC.
The following sections describe how to use Flume 1.3.x and CDH 5 with Kerberos security on your Hadoop cluster:
- Configuring Flume's Security Properties
- Configuring Kerberos for Flume Thrift Source and Sink Using Cloudera Manager
- Configuring Kerberos for Flume Thrift Source and Sink Using the Command Line
- Flume Account Requirements
- Testing the Flume HDFS Sink Configuration
- Writing to a Secure HBase Cluster
- Step 10: (Flume Only) Use Substitution Variables for the Kerberos Principal and Keytab