1. For All Nodes

Use the following as a base configuration for all nodes in your cluster:

  • Root parttion: OS and core program files

  • Swap: Size 2X system memory

  • Hadoop partitions: Hadoop should have its own partitions for Hadoop files and logs. All Hadoop partitions should be mounted individually from drives as "/grid/[0-n]". Drives should be partitioned using XFS, ext4, or ext3 in that order of preference. Don't use LVM; it adds latency and causes a bottleneck.

A sample layout based on this configuration:

  • /swap - 96 GB (for a 48GB memory system)

  • /root - 20GB (ample room for existing files, future log file growth, and OS upgrades)

  • /grid/0/ - [full disk GB] first partition for Hadoop to use for local storage

  • /grid/1/ - second partition for Hadoop to use

  • /grid/2/ - ...