6. Configuring for Secure Clusters

Verify that hadoop.security.auth_to_local in core-site.xml is consistent across all clusters.

	Important
	Inconsistent rules for hadoop.security.auth_to_local can lead to issues with delegation token renewals.

For working with secure clusters that use hive and hcatalog, the cluster.xml entity should have hadoop.rpc.protection set to the value of the hadoop cluster's hadoop.rpc.protection. For example:

<property name="hadoop.rpc.protection" value="authentication"/>

Note: value cannot be hardcoded to authentication. It has to match the authentication value the hadoop cluster uses.

Set dfs.namenode.kerberos.principal for the cluster NameNode. For example:

<property name="dfs.namenode.kerberos.principal" value="nn/ip-172-31-47-87.ec2.internal@EXAMPLE.COM"/>

For the hcatalog retention/replication/process to work with secure clusters, set hive.metastore.sasl.enabled to true in the cluster entitiy. For example:

<property name="hive.metastore.sasl.enabled" value="true"/>

Set hive.metastore.kerberos.principal and hive.metastore.uris. For example:

<property name="hive.metastore.kerberos.principal" value="hive/ip-172-31-47-87.ec2.internal@EXAMPLE.COM"/>
<property name="hive.metastore.uris" value="thrift://ip-172-31-47-87.ec2.internal:9083"/>

For wasb replication to work, the target cluster's core-site.xml must have wasb credentials. For example:

<property>
    <name>fs.azure.account.key.testuser.blob.core.windows.net</name>
    <value>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</value>
</property>

Create the following property definitions in your cluster entity or entities. In the following example, replace $my.internal@EXAMPLE.COM and $my.internal with your own values.

<properties>
    <property name="dfs.namenode.kerberos.principal" value="nn/$my.internal@EXAMPLE.COM"/> 
    <property name="hive.metastore.kerberos.principal" value="hive/$my.internal@EXAMPLE.COM"/> 
    <property name="hive.metastore.uris" value="thrift://$my.internal:9083"/>
    <property name="hive.metastore.sasl.enabled" value="true"/>
</properties>