This is the documentation for CDH 5.1.x. Documentation for other versions is available at Cloudera Documentation.

HttpFS Security Configuration

This section describes how to configure HttpFS CDH 5 with Kerberos security on a Hadoop cluster:

For more information about HttpFS, see http://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-hdfs-httpfs/index.html.

Important:

To enable HttpFS to work with Kerberos security on your Hadoop cluster, make sure you perform the installation and configuration steps in Configuring Hadoop Security in CDH 5.

Important:

If the NameNode, Secondary NameNode, DataNode, JobTracker, TaskTrackers, ResourceManager, NodeManagers, HttpFS, or Oozie services are configured to use Kerberos HTTP SPNEGO authentication, and two or more of these services are running on the same host, then all of the running services must use the same HTTP principal and keytab file used for their HTTP endpoints.

Configuring the HttpFS Server to Support Kerberos Security

Create an HttpFS service user principal that is used to authenticate with the Hadoop cluster. The syntax of the principal is: httpfs/<fully.qualified.domain.name>@<YOUR-REALM> where: fully.qualified.domain.name is the host where the HttpFS server is running YOUR-REALM is the name of your Kerberos realm
```
kadmin: addprinc -randkey httpfs/fully.qualified.domain.name@YOUR-REALM.COM
```
Create a HTTP service user principal that is used to authenticate user requests coming to the HttpFS HTTP web-services. The syntax of the principal is: HTTP/<fully.qualified.domain.name>@<YOUR-REALM> where: 'fully.qualified.domain.name' is the host where the HttpFS server is running YOUR-REALM is the name of your Kerberos realm
```
kadmin: addprinc -randkey HTTP/fully.qualified.domain.name@YOUR-REALM.COM
```
Important:
The HTTP/ component of the HTTP service user principal must be upper case as shown in the syntax and example above.

Create keytab files with both principals.

$ kadmin
kadmin: xst -k httpfs.keytab httpfs/fully.qualified.domain.name
kadmin: xst -k http.keytab HTTP/fully.qualified.domain.name

Merge the two keytab files into a single keytab file:

$ ktutil
ktutil: rkt httpfs.keytab
ktutil: rkt http.keytab
ktutil: wkt httpfs-http.keytab

Test that credentials in the merged keytab file work. For example:
```
$ klist -e -k -t httpfs-http.keytab
```
Copy the httpfs-http.keytab file to the HttpFS configuration directory. The owner of the httpfs-http.keytab file should be the httpfs user and the file should have owner-only read permissions.

Edit the HttpFS server httpfs-site.xml configuration file in the HttpFS configuration directory by setting the following properties:

Property	Value
httpfs.authentication.type	kerberos
httpfs.hadoop.authentication.type	kerberos
httpfs.authentication.kerberos.principal	HTTP/<HTTPFS-HOSTNAME>@<YOUR-REALM.COM>
httpfs.authentication.kerberos.keytab	/etc/hadoop-httpfs/conf/httpfs-http.keytab
httpfs.hadoop.authentication.kerberos.principal	httpfs/<HTTPFS-HOSTNAME>@<YOUR-REALM.COM>
httpfs.hadoop.authentication.kerberos.keytab	/etc/hadoop-httpfs/conf/httpfs-http.keytab
httpfs.authentication.kerberos.name.rules	Use the value configured for 'hadoop.security.auth_to_local' in 'core-site.xml'

Important:

You must restart the HttpFS server to have the configuration changes take effect.

Using curl to access an URL Protected by Kerberos HTTP SPNEGO

Important:

Your version of curl must support GSS and be capable of running curl -V.

To configure curl to access an URL protected by Kerberos HTTP SPNEGO:

Run curl -V:

$ curl -V
curl 7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l
zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz

$ kinit
Please enter the password for tucu@LOCALHOST:

Use curl to fetch the protected URL:
```
$ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt http://localhost:14000/webhdfs/v1/?op=liststatus
```
where: The --negotiate option enables SPNEGO in curl. The -u : option is required but the user name is ignored (the principal that has been specified for kinit is used). The -b and -c options are used to store and send HTTP cookies.

Page generated September 3, 2015.