Integrating Hadoop Security with Alternate Authentication

One of the ramifications of enabling security on a Hadoop cluster is that every user who interacts with the cluster must have a Kerberos principal configured. For some of the services, specifically Oozie and Hadoop (for example, JobTracker and TaskTracker), it can be convenient to run a mixed form of authentication where Kerberos authentication is used for API or command line access while some other form of authentication (for example, SSO and LDAP) is used for accessing Web UIs. Using an alternate authentication deployment is considered an advanced topic because only a partial implementation is provided in this release: you will have to implement some of the code yourself.

Proceed as follows:

See also the Example Implementation for Oozie.

Configuring the AuthenticationFilter to use Kerberos

First, you must do all of the steps in the Server Side Configuration section of the Hadoop Auth, Java HTTP SPNEGO Documentation to configure AuthenticationFilter to use Kerberos. You must configure AuthenticationFilter to use Kerberos before doing the steps below.

Creating an AltKerberosAuthenticationHandler Subclass

An AuthenticationHandler is installed on the server-side to handle authenticating clients and creating an AuthenticationToken.

  1. Subclass the org.apache.hadoop.security.authentication.server.AltKerberosAuthenticationHandler class (in the hadoop-auth package).
  2. When a client sends a request, the authenticate method will be called. For browsers, AltKerberosAuthenticationHandler will call the alternateAuthenticate method, which is what you need to implement to interact with the desired authentication mechanism. For non-browsers, AltKerberosAuthenticationHandler will follow the Kerberos SPNEGO sequence (this is provided for you).
  3. The alternateAuthenticate(HttpServletRequest request, HttpServletResponse response) method in your subclass should following these rules:
  4. Return null if the authentication is still in progress; the response object can be used to interact with the client.
  5. Throw an AuthenticationException if the authentication failed.
  6. Return an AuthenticationToken if the authentication completed successfully.

Enabling Your AltKerberosAuthenticationHandler Subclass

You can enable the alternate authentication on Hadoop Web UIs, Oozie Web UIs, or both. You will need to include a JAR containing your subclass on the classpath of Hadoop or Oozie. All Kerberos-related configuration properties will still apply.

Enabling Your AltKerberosAuthenticationHandler Subclass on Hadoop Web UIs

  1. Stop Hadoop by running the following command on every node in your cluster (as root): 
    $ for x in `cd /etc/init.d ; ls hadoop-*` ; do sudo service $x stop ; done
  2. Set the following property in core-site.xml, where org.my.subclass.of.AltKerberosAuthenticationHandler is the classname of your subclass: 
    <property>
  <name>hadoop.http.authentication.type</name>
  <value>org.my.subclass.of.AltKerberosAuthenticationHandler</value>
</property>
  3. (Optional) You can also specify which user-agents you do not want to be considered as browsers by setting the following property as required (default value is shown). Note that all Java-based programs (such as Hadoop client) will use java as their user-agent. 
    <property>
  <name>hadoop.http.authentication.alt-kerberos.non-browser.user-agents</name>
  <value>java,curl,wget,perl</value>
</property>
  4. Copy the JAR containing your subclass into /usr/lib/hadoop/lib/.
  5. Start Hadoop by running the following command: 
    $ for x in `cd /etc/init.d ; ls hadoop-*` ; do sudo service $x start ; done

Enabling Your AltKerberosAuthenticationHandler Subclass on Oozie Web UI

  1. Stop the Oozie Server: 
    sudo /sbin/service oozie stop
  2. Set the following property in oozie-site.xml, where org.my.subclass.of.AltKerberosAuthenticationHandler is the classname of your subclass: 
    <property>
  <name>oozie.authentication.type</name>
  <value>org.my.subclass.of.AltKerberosAuthenticationHandler</value>
</property>
  3. (Optional) You can also specify which user-agents you do not want to be considered as browsers by setting the following property as required (default value is shown). Note that all Java-based programs (such as Hadoop client) will use java as their user-agent. 
    <property>
  <name>oozie.authentication.alt-kerberos.non-browser.user-agents</name>
  <value>java,curl,wget,perl</value>
</property>
  4. Copy the JAR containing your subclass into /var/lib/oozie.
  5. Start the Oozie Server: 
    sudo /sbin/service oozie start

Example Implementation for Oozie

An example implementation of AltKerberosAuthenticationHandler is included (though not built by default) with Oozie. Also included is a simple Login Server with two implementations. The first one will authenticate any user who is using a username and password that are identical, such as foo:foo. The second one can be configured against an LDAP server to use LDAP for authentication.

You can read comprehensive documentation on the example at Creating Custom Authentication.