Configure memory settings

The memory configuration for YARN and MapReduce memory is important to get the best performance from your cluster.

Several different settings are involved. The table below shows the default settings, as well as the settings that Cloudera recommends, for each configuration option.

Table 1. YARN and MapReduce Memory Configuration
Cloudera Manager Property Name Cloudera Runtime Property Name Default Configuration Cloudera Tuning Guidelines
Container Memory Minimum yarn.scheduler.minimum-allocation-mb 1 GB 0
Container Memory Maximum yarn.scheduler.maximum-allocation-mb 64 GB amount of memory on largest host
Container Memory Increment yarn.scheduler.increment-allocation-mb 512 MB Use a fairly large value, such as 128 MB
Container Memory yarn.nodemanager.resource.memory-mb 8 GB 8 GB
Map Task Memory mapreduce.map.memory.mb 1 GB 1 GB
Reduce Task Memory mapreduce.reduce.memory.mb 1 GB 1 GB
Map Task Java Opts Base mapreduce.map.java.opts -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Xmx768m
Reduce Task Java Opts Base mapreduce.reduce.java.opts -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Xmx768m
ApplicationMaster Memory yarn.app.mapreduce.am.resource.mb 1 GB 1 GB
ApplicationMaster Java Opts Base yarn.app.mapreduce.am.command-opt -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Xmx768m