Command Line Installation
Also available as:
PDF
loading table of contents...

Configuring for Secure Clusters

If you are using secure clusters, complete the following steps.

  1. Verify that hadoop.security.auth_to_local in core-site.xml is consistent across all clusters.

    [Important]Important

    Inconsistent rules for hadoop.security.auth_to_local can lead to issues with delegation token renewals.

  2. For working with secure clusters that use hive and hcatalog, the cluster.xml entity should have hadoop.rpc.protection set to the value of the hadoop cluster's hadoop.rpc.protection. For example:

    <property name="hadoop.rpc.protection" value="authentication"/>

    [Note]Note

    Value cannot be hard coded to authentication. It has to match the authentication value the hadoop cluster uses.

  3. Set dfs.namenode.kerberos.principal for the cluster NameNode. For example:

    <property name="dfs.namenode.kerberos.principal" value="nn/ip-172-31-47-87.ec2.internal@EXAMPLE.COM"/>

  4. For the hcatalog retention/replication/process to work with secure clusters, set hive.metastore.sasl.enabled to true in the cluster entity. For example:

    <property name="hive.metastore.sasl.enabled" value="true"/>

  5. Set hive.metastore.kerberos.principal and hive.metastore.uris. For example:

    <property name="hive.metastore.kerberos.principal" value="hive/ip-172-31-47-87.ec2.internal@EXAMPLE.COM"/>
    <property name="hive.metastore.uris" value="thrift://ip-172-31-47-87.ec2.internal:9083"/>
  6. For Windows Azure Storage Blob (WASB) replication to work, the target cluster's core-site.xml must have wasb credentials. For example:

    <property>
        <name>fs.azure.account.key.testuser.blob.core.windows.net</name>
        <value>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</value>
    </property>
  7. Create the following property definitions in your cluster entity or entities. In the following example, replace $my.internal@EXAMPLE.COM and $my.internal with your own values.

    <properties>
        <property name="dfs.namenode.kerberos.principal" value="nn/$my.internal@EXAMPLE.COM"/> 
        <property name="hive.metastore.kerberos.principal" value="hive/$my.internal@EXAMPLE.COM"/> 
        <property name="hive.metastore.uris" value="thrift://$my.internal:9083"/>
        <property name="hive.metastore.sasl.enabled" value="true"/>
    </properties>