Configure Hadoop
RHEL/CentOS/Oracle Linux
Use the HDP Utility script to calculate memory configuration settings. You must update the memory/CPU settings in yarn-site.xml and mapred-site.xml
Paths have changed in HDP 2.3. make sure you remove old path specifications from hadoop-env.sh, such as:
export JAVA_LIBRARY_PATH=/usr/lib/hadoop/lib/native/Linux-amd64-64
If you leave these paths in your hadoop-env.sh file, the lzo compression code will not load, as this is not where lzo is installed.
SLES
Use the HDP Utility script to calculate memory configuration settings. You must update the memory/CPU settings in yarn-site.xml and mapred-site.xml
Paths have changed in HDP 2.3. make sure you remove old path specifications from hadoop-env.sh, such as:
export JAVA_LIBRARY_PATH=/usr/lib/hadoop/lib/native/Linux-amd64-64
If you leave these paths in your hadoop-env.sh file, the lzo compression code will not load, as this is not where lzo is installed.
Ubuntu/Debian
Use the HDP Utility script to calculate memory configuration settings. You must update the memory/CPU settings in yarn-site.xml and mapred-site.xml
Paths have changed in HDP 2.3. make sure you remove lines from hadoop-env.sh such as:
export JAVA_LIBRARY_PATH=/usr/lib/hadoop/lib/native/Linux-amd64-64
If you leave these paths in your hadoop-env.sh file, the lzo compression code will not load as this is not where lzo is installed.