Use the following instructions to configure rack awareness on a HDP cluster:
Topology scripts are used by Hadoop to determine the rack location of nodes. This information is used by Hadoop to replicate block data to redundant racks.
Create a topology script and data file. The topology script MUST be executable.
Sample Topology Script
File name:
rack-topology.sh
#!/bin/bash # Adjust/Add the property "net.topology.script.file.name" # to core-site.xml with the "absolute" path the this # file. ENSURE the file is "executable". # Supply appropriate rack prefix RACK_PREFIX=default # To test, supply a hostname as script input: if [ $# -gt 0 ]; then CTL_FILE=${CTL_FILE:-"rack_topology.data"} HADOOP_CONF=${HADOOP_CONF:-"/etc/hadoop/conf"} if [ ! -f ${HADOOP_CONF}/${CTL_FILE} ]; then echo -n "/$RACK_PREFIX/rack " exit 0 fi while [ $# -gt 0 ] ; do nodeArg=$1 exec< ${HADOOP_CONF}/${CTL_FILE} result="" while read line ; do ar=( $line ) if [ "${ar[0]}" = "$nodeArg" ] ; then result="${ar[1]}" fi done shift if [ -z "$result" ] ; then echo -n "/$RACK_PREFIX/rack " else echo -n "/$RACK_PREFIX/rack_$result " fi done else echo -n "/$RACK_PREFIX/rack " fi
Sample Topology Data File
File name:
rack_topology.data
# This file should be: # - Placed in the /etc/hadoop/conf directory # - On the Namenode (and backups IE: HA, Failover, etc) # - On the Job Tracker OR Resource Manager (and any Failover JT's/RM's) # This file should be placed in the /etc/hadoop/conf directory. # Add Hostnames to this file. Format <host ip> <rack_location> 192.168.2.10 01 192.168.2.11 02 192.168.2.12 03
Copy both of these files to the
/etc/hadoop/conf
directory on all cluster nodes.Run the
rack-topology.sh
script to ensure that it returns the correct rack information for each host.
Stop HDFS using the instructions on this page.
Add the following property to
core-site.xml
:<property> <name>net.topology.script.file.name</name> <value>/etc/hadoop/conf/rack-topology.sh</value> </property>
By default the topology script will process up to 100 requests per invocation. You can also specify a different number of requests with the
net.topology.script.number.args
property. For example:<property> <name>net.topology.script.number.args</name> <value>75</value> </property>
Restart HDFS and MapReduce using the instructions on this page.
After the services have started, you can use the following methods to verify that rack awareness has been activated:
Look in the NameNode logs located in
/var/log/hadoop/hdfs/
(for example:hadoop-hdfs-namenode-sandbox.log
). You should see an entry like this:014-01-13 15:58:08,495 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /rack01/<ipaddress>
The Hadoop
fsck
command should return something like the following (if there are two racks):Status: HEALTHY Total size: 123456789 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 123456789 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 3 Average block replication: 3.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 40 Number of racks: 2 FSCK ended at Mon Jan 13 17:10:51 UTC 2014 in 1 milliseconds
The Hadoop
dfsadmin -report
command will return a report that includes the rack name next to each machine. The report should look something like the following (partial) example:[bsmith@hadoop01 ~]$ sudo -u hdfs hadoop dfsadmin -report Configured Capacity: 19010409390080 (17.29 TB) Present Capacity: 18228294160384 (16.58 TB) DFS Remaining: 5514620928000 (5.02 TB) DFS Used: 12713673232384 (11.56 TB) DFS Used%: 69.75% Under replicated blocks: 181 Blocks with corrupt replicas: 0 Missing blocks: 0 ------------------------------------------------- Datanodes available: 5 (5 total, 0 dead) Name: 192.168.90.231:50010 (h2d1.hdp.local) Hostname: h2d1.hdp.local Rack: /default/rack_02 Decommission Status : Normal Configured Capacity: 15696052224 (14.62 GB) DFS Used: 314380288 (299.82 MB) Non DFS Used: 3238612992 (3.02 GB) DFS Remaining: 12143058944 (11.31 GB) DFS Used%: 2.00% DFS Remaining%: 77.36% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Last contact: Thu Jun 12 11:39:51 EDT 2014