Chapter 5. Configuring Rack Awareness On HDP

Use the following instructions to configure rack awareness on an HDP cluster.

Create a Rack Topology Script

Topology scripts are used by Hadoop to determine the rack location of nodes. This information is used by Hadoop to replicate block data to redundant racks.

Create a topology script and data file. The topology script must be executable.

Sample Topology Script Named rack-topology.sh

#!/bin/bash

# Adjust/Add the property "net.topology.script.file.name"
# to core-site.xml with the "absolute" path the this
# file. ENSURE the file is "executable".

# Supply appropriate rack prefix
RACK_PREFIX=default

# To test, supply a hostname as script input:
if [ $# -gt 0 ]; then

CTL_FILE=${CTL_FILE:-"rack_topology.data"}

HADOOP_CONF=${HADOOP_CONF:-"/etc/hadoop/conf"} 

if [ ! -f ${HADOOP_CONF}/${CTL_FILE} ]; then
 echo -n "/$RACK_PREFIX/rack "
 exit 0
fi

while [ $# -gt 0 ] ; do
 nodeArg=$1
 exec< ${HADOOP_CONF}/${CTL_FILE}
 result=""
 while read line ; do
 ar=( $line )
 if [ "${ar[0]}" = "$nodeArg" ] ; then
 result="${ar[1]}"
 fi
 done
 shift
 if [ -z "$result" ] ; then
 echo -n "/$RACK_PREFIX/rack "
 else
 echo -n "/$RACK_PREFIX/rack_$result "
 fi
done

else
 echo -n "/$RACK_PREFIX/rack "
fi

Sample Topology Data File Named rack_topology.data

# This file should be:
# - Placed in the /etc/hadoop/conf directory
# - On the Namenode (and backups IE: HA, Failover, etc)
# - On the Job Tracker OR Resource Manager (and any Failover JT's/RM's) 
# This file should be placed in the /etc/hadoop/conf directory.

# Add Hostnames to this file. Format <host ip> <rack_location>
192.168.2.10 01
192.168.2.11 02
192.168.2.12 03

Copy both of these files to the /etc/hadoop/conf directory on all cluster nodes.
Run the rack-topology.sh script to ensure that it returns the correct rack information for each host.

Add the Topology Script Property to core-site.xml

Stop HDFS using the applicable commands in the "Controlling HDP Services Manually" section of Installing HDP Manually

Add the following property to core-site.xml:

<property>
  <name>net.topology.script.file.name</name> 
  <value>/etc/hadoop/conf/rack-topology.sh</value>
</property>

By default the topology script will process up to 100 requests per invocation. You can also specify a different number of requests with the net.topology.script.number.args property. For example:

<property> 
  <name>net.topology.script.number.args</name> 
  <value>75</value>
</property>

Restart HDFS and MapReduce

Restart HDFS and MapReduce using the applicable commands in Controlling HDP Services Manually.

Verify Rack Awareness

After the services have started, you can use the following methods to verify that rack awareness has been activated:

Look in the NameNode logs located in /var/log/hadoop/hdfs/. For example: hadoop-hdfs-namenode-sandbox.log. You should see an entry like this:
```
014-01-13 15:58:08,495 INFO org.apache.hadoop.net.NetworkTopology: Adding
                        a new node: /rack01/<ipaddress>
```

The Hadoop fsck command should return something like the following (if there are two racks):

 Status: HEALTHY  Total size: 123456789 B  Total dirs: 0  Total files: 1  Total blocks (validated): 1 (avg. block size 123456789 B)
 Minimally replicated blocks: 1 (100.0 %)  Over-replicated blocks: 0 (0.0 %)  Under-replicated blocks: 0 (0.0 %)  Mis-replicated blocks: 0 (0.0 %)
 Default replication factor: 3  Average block replication: 3.0  Corrupt blocks: 0  Missing replicas: 0 (0.0 %)  Number of data-nodes: 40  Number of racks: 2  FSCK ended at Mon Jan 13 17:10:51 UTC 2014 in 1 milliseconds

The Hadoop dfsadmin -report command will return a report that includes the rack name next to each machine. The report should look something like the following excerpted example:

 [bsmith@hadoop01 ~]$ sudo -u hdfs hadoop dfsadmin -report
 Configured Capacity: 19010409390080 (17.29 TB) Present Capacity: 18228294160384 (16.58 TB) DFS Remaining: 5514620928000 (5.02 TB) DFS
 Used: 12713673232384 (11.56 TB) DFS Used%: 69.75% Under replicated blocks: 181 Blocks with corrupt replicas: 0  Missing blocks: 0
 -------------------------------------------------  Datanodes available:
 5 (5 total, 0 dead)  Name: 192.168.90.231:50010 (h2d1.hdp.local) Hostname: h2d1.hdp.local Rack: /default/rack_02 Decommission Status : Normal Configured Capacity: 15696052224 (14.62 GB) DFS Used: 314380288
(299.82 MB) Non DFS Used: 3238612992 (3.02 GB) DFS Remaining: 12143058944 (11.31 GB) DFS Used%: 2.00% DFS Remaining%: 77.36%
 Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Last contact: Thu Jun 12 11:39:51 EDT 2014