The following recommendations can help you determine appropriate memory configurations based on your usage scenario:
Make sure that there is enough memory for all the processes. Remember that system processes take around 10% of the available memory.
For co-deploying an HBase RegionServer and MapReduce service on the same node, reduce the RegionServer's heap size (use the HBase Settings: RegionServer:
HBase Region Servers maximum Java heap sizeproperty to modify the RegionServer heap size).For co-deploying an HBase RegionServer and the MapReduce service on the same node, or for memory intensive MapReduce applications, modify the map and reduce slots as suggested in the following example:
EXAMPLE: For co-deploying an HBase RegionServer and the MapReduce service on a machine with 16GB of available memory, the following would be a recommended configuration:
2 GB: system processes
8 GB: MapReduce slots. 6 Map + 2 Reduce slots per 1 GB task
4 GB: HBase RegionServer
1 GB: TaskTracker
1 GB: DataNode
To change the number of Map and Reduce slots based on the memory requirements of your application, use the following properties:
MapReduce Settings: TaskTracker: Number of Map slots per
node
MapReduce Settings: TaskTracker: Number of Reduce slots
per node
