Verify rack awareness

You must perform a series of checks to verify if rack awareness is activated on the cluster.

Check the NameNode logs located in /var/log/hadoop/hdfs/ for the addition of nodes.

014-01-13 15:58:08,495 INFO org.apache.hadoop.net.NetworkTopology: Adding
                        a new node: /rack01/<ipaddress>

Run the hdfs fsck command to ensure that there are no inconsistencies.

For a cluster with two racks, the fsck command returns a status similar to the following:

Status: HEALTHY  Total size: 123456789 B  Total dirs: 0  Total files: 1  Total blocks (validated): 1 (avg. block size 123456789 B)
 Minimally replicated blocks: 1 (100.0 %)  Over-replicated blocks: 0 (0.0 %)  Under-replicated blocks: 0 (0.0 %)  Mis-replicated blocks: 0 (0.0 %)
 Default replication factor: 3  Average block replication: 3.0  Corrupt blocks: 0  Missing replicas: 0 (0.0 %)  Number of data-nodes: 40  Number of racks: 2  FSCK ended at Mon Jan 13 17:10:51 UTC 2014 in 1 milliseconds

Run the dfsadmin -report command for a report that includes the rack name next to each DataNode.

The dfsadmin -report command returns a report similar to the following excerpted example:

Configured Capacity: 19010409390080 (17.29 TB) Present Capacity: 18228294160384 (16.58 TB) DFS Remaining: 5514620928000 (5.02 TB) DFS
 Used: 12713673232384 (11.56 TB) DFS Used%: 69.75% Under replicated blocks: 181 Blocks with corrupt replicas: 0  Missing blocks: 0
 -------------------------------------------------  Datanodes available:
 5 (5 total, 0 dead)  Name: 192.168.90.231:50010 (h2d1.hdp.local) Hostname: h2d1.hdp.local Rack: /default/rack_02 Decommission Status : Normal Configured Capacity: 15696052224 (14.62 GB) DFS Used: 314380288
(299.82 MB) Non DFS Used: 3238612992 (3.02 GB) DFS Remaining: 12143058944 (11.31 GB) DFS Used%: 2.00% DFS Remaining%: 77.36%
 Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Last contact: Thu Jun 12 11:39:51 EDT 2014