4. Verify Rack Awareness

After the services have started, you can use the following methods to verify that rack awareness has been activated:

  1. Look in the NameNode logs located in /var/log/hadoop/hdfs/. For example: hadoop-hdfs-namenode-sandbox.log. You should see an entry like this:

    014-01-13 15:58:08,495 INFO org.apache.hadoop.net.NetworkTopology: Adding
                            a new node: /rack01/<ipaddress>
  2. The Hadoop fsck command should return something like the following (if there are two racks):

     Status: HEALTHY  Total size: 123456789 B  Total dirs: 0  Total files: 1  Total blocks (validated): 1 (avg. block size 123456789 B)
     Minimally replicated blocks: 1 (100.0 %)  Over-replicated blocks: 0 (0.0 %)  Under-replicated blocks: 0 (0.0 %)  Mis-replicated blocks: 0 (0.0 %)
     Default replication factor: 3  Average block replication: 3.0  Corrupt blocks: 0  Missing replicas: 0 (0.0 %)  Number of data-nodes: 40  Number of racks: 2  FSCK ended at Mon Jan 13 17:10:51 UTC 2014 in 1 milliseconds
  3. The Hadoop dfsadmin -report command will return a report that includes the rack name next to each machine. The report should look something like the following excerpted example:

     [bsmith@hadoop01 ~]$ sudo -u hdfs hadoop dfsadmin -report
     Configured Capacity: 19010409390080 (17.29 TB) Present Capacity: 18228294160384 (16.58 TB) DFS Remaining: 5514620928000 (5.02 TB) DFS
     Used: 12713673232384 (11.56 TB) DFS Used%: 69.75% Under replicated blocks: 181 Blocks with corrupt replicas: 0  Missing blocks: 0
     -------------------------------------------------  Datanodes available:
     5 (5 total, 0 dead)  Name: 192.168.90.231:50010 (h2d1.hdp.local) Hostname: h2d1.hdp.local Rack: /default/rack_02 Decommission Status : Normal Configured Capacity: 15696052224 (14.62 GB) DFS Used: 314380288
    (299.82 MB) Non DFS Used: 3238612992 (3.02 GB) DFS Remaining: 12143058944 (11.31 GB) DFS Used%: 2.00% DFS Remaining%: 77.36%
     Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Last contact: Thu Jun 12 11:39:51 EDT 2014