Configuring Storm on YARN on Secure Clusters

1. Generate and Distribute Keytab Files for Storm

On the KDC host:

Create a Storm server principal for each NodeManager host:

kadmin.local -q "addprinc -randkey <server_principal_name>/<node_manager_host_name>@EXAMPLE.COM"

Create a Storm client principal:

kadmin.local -q "addprinc -randkey <client_principal_name>@EXAMPLE.COM"

Export the principals to a keytab file:

For each NodeManager host:

kadmin.local -q "xst -norandkey -k /etc/security/keytabs/nimbus.keytab <server_principal_name>/<node_manager_host_name>@EXAMPLE.COM"

For the Storm client principal:

kadmin.local -q "xst -norandkey -k /etc/security/keytabs/storm.keytab <client_principal_name>@EXAMPLE.COM"

Distribute the keytab file to the NodeManager hosts on which the application components will be launched. Be sure to set the permissions so that the runtime elements are allowed to access the keytab files, for example:
```
sudo su -
chown root:users <keytab_file>
chmod 440 <keytab_file>
```

2. Add an OS User for HDFS Access

You need to add an OS user for proper HDFS access (user and group availability) to the secure Storm deployment:

Create system users with the same short names as the generated server principal and client principal:
```
useradd -n <storm server or client principal short name>
passwd <storm server or client principal short name>
```
You must specify a password in order to activate the user account.

Associate the users to the appropriate user groups:

usermod -a -G hadoop <storm server or client principal short name>

Edit the Secure Version of the Application Configuration Files

As previously mentioned, the Storm-on-Slider application package includes both non-secure (appConfig-default.json) and secure (appConfig-secured-default.json) versions of the application specification.

On secure clusters, you should use the secure version of the application specification. The security-related entries in the appConfig-secured.json file are listed below.

"site.storm-site.nimbus.authorizer": "backtype.storm.security.auth.authorizer.SimpleACLAuthorizer", "site.storm-site.storm.thrift.transport": "backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin", "site.storm-site.java.security.auth.login.config": "${AGENT_WORK_ROOT}/app/install/apache-storm-0.9.3.2.2.9.0-$BUILD/conf/storm_jaas.conf", "site.storm-site.storm.principal.tolocal": "backtype.storm.security.auth.KerberosPrincipalToLocal", "site.storm-site.storm.zookeeper.superACL": "sasl:storm", "site.storm-site.nimbus.admins": "['jon', 'storm']", "site.storm-site.nimbus.supervisor.users": "['storm']", "site.storm-site.nimubs.authorizer": "backtype.storm.security.auth.authorizer.SimpleACLAuthorizer", "site.storm-site.storm.thrift.transport": "backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin", "site.storm-site.storm.principal.tolocal": "backtype.storm.security.auth.KerberosPrincipalToLocal", "site.storm-site.ui.filter": "org.apache.hadoop.security.authentication.server.AuthenticationFilter", "site.storm-site.ui.filter.params": "{'type': 'kerberos', 'kerberos.principal': 'HTTP/_HOST', 'kerberos.keytab': '/etc/security/keytabs/spnego.service.keytab', 'kerberos.name.rules': 'RULE:[2:$1@$0]([jt]t@.*EXAMPLE.COM)s/.*/$MAPRED_USER/ RULE:[2:$1@$0]([nd]n@.*EXAMPLE.COM)s/.*/$HDFS_USER/DEFAULT'}", "site.storm-env.kerberos_domain": "EXAMPLE.COM", "site.storm-env.storm_client_principal_name": "storm@EXAMPLE.COM", "site.storm-env.storm_server_principal_name": "storm_server/_HOST@EXAMPLE.COM", "site.storm-env.storm_client_keytab": "/etc/security/keytabs/storm.keytab", "site.storm-env.storm_server_keytab": "/etc/security/keytabs/nimbus.keytab"

some key points regarding these configuration properties:

The properties assume the use of the Kerberos domain "EXAMPLE.COM". Change the domain name to match the name configured for your environment.
The JAAS configuration (storm_jaas.conf) path will be dependent on the version of the Storm distribution you are using (for example, version apache-storm-0.9.3.2.2.9.0-$BUILD as shown above).
The superACL property should point to the client principal short name.
The nimbus.admins property values should include both the Storm client principal short name and the principal associated with the Slider user who launches the application.
The supervisor.users property should be set to the short name of the Storm client principal.
The ui.filter.params property requires an HTTP/Web principal. This principle can be found in the spnego.service.keytab file.
The storm-env properties are fairly straightforward -- simply provide the server principal, client principal, and keytab file locations.

At this point you should be ready to launch a Storm cluster using the Slider create command. You will need to authenticate against Kerberos and obtain a TGT using the kinit command prior to invoking the Slider create command:

kinit <user name>

	Note
	For more information on configuring Storm-on-Slider on secure clusters, including information about keytab-associated properties and the available keytab distribution options, see Apache Slider Security.

​Configuring Storm on YARN on Secure Clusters

Configuring Storm on YARN on Secure Clusters