6. Secure-to-Secure: ResourceManager Mapping Rules

When copying between two HDP2 secure clusters, or from HDP1 secure to HDP2 secure, further ResourceManager (RM) configuration is required if the two clusters have different realms. In order for DistCP to succeed, the same RM mapping rule must be used in both clusters.

For example, if secure Cluster 1 has the following RM mapping rule:

<property>
    <name>hadoop.security.auth_to_local</name>
    <value>
      RULE:[2:$1@$0](rm@.*SEC1.SUP1.COM)s/.*/yarn/
      DEFAULT
    </value>
</property>

And secure Cluster 2 has the following RM mapping rule:

<property>
    <name>hadoop.security.auth_to_local</name>
    <value>
      RULE:[2:$1@$0](rm@.*BA.YISEC3.COM)s/.*/yarn/
      DEFAULT
    </value>
</property>

The DistCp job from Cluster 1 to Cluster 2 will fail because Cluster 2 cannot resolve the RM principle of Cluster 1 correctly to the yarn user, because the RM mapping rule in Cluster 2 is different than the RM mapping rule in Cluster 1.

The solution is to use the same RM mapping rule in both Cluster 1 and Cluster 2:

<property>
    <name>hadoop.security.auth_to_local</name>
    <value>
      RULE:[2:$1@$0](rm@.*SEC1.SUP1.COM)s/.*/yarn/
      RULE:[2:$1@$0](rm@.*BA.YISEC3.COM)s/.*/yarn/
      DEFAULT
   </value>
 </property>