Warning | |
---|---|
These instructions are now obsolete and no longer provide a Supported version of Tez. Use the information found in HDP 2.1 regarding Installing and Configuring Tez. |
Use the following instructions to enable Tez AM:
On all the client nodes and Tez Service host machine, edit
/etc/tez/conf/tez-env.sh
file and modify the following environment variables:export HADOOP_HOME="$HADOOP_HOME" export JAVA_HOME="$JAVA_HOME"
where
$HADOOP_HOME
is the location of the directory that contains all core Hadoop JAR files. For example,/usr/lib/hadoop
.$JAVA_HOME
is the location of the directory that contains JDK.
Ensure that the
/
file exists on the Tez Service host machine.$HADOOP_HOME
/bin/hadoopOn all the client nodes and Tez host machine, edit
mapred-site.xml
and modify the following properties:Enable Tez AM:
<property> <name>mapreduce.framework.name</name> <value>yarn-tez</value> <description>Name of the MapReduce framework. Default value is yarn.</description> </property>
Set MapReduce CLASSPATH to a CLASSPATH that contains all the Tez JAR files:
<property> <name>mapreduce.application.classpath</name> <value>$TEZ_HOME/*,$TEZ_HOME/lib/*</value> <description>Classpath for MapReduce applications.</description> </property>
where
$TEZ_HOME
is the location of the directory that contains all the Tez JAR files. By default,$TEZ_HOME
is set to/usr/lib/tez
.Enable container reuse across task attempts:
<property> <name>yarn.app.mapreduce.am.scheduler.reuse.enable</name> <value>true</value> <description>Enable container reuse across task attempts. Default is set to false.</description> </property>
Define number of task attempts to be run on a single container before the container is released. Use -1 to disable this limit.
<property> <name>yarn.app.mapreduce.am.scheduler.reuse.max-attempts-per-container</name> <value>-1</value> <description>Defines number of task attempts to be run on a single container before the container is released. To disable this limit, set the value of this property to -1.</description> </property>
Note For certain workloads, some jobs tend to have memory leaks and so we recommend that you set the container reuse property to a manageable value (for example 5 or 10).
On all the client nodes and Tez hostmachine, edit
hadoop-env.sh
and setHADOOP_CLASSPATH
as shown below:export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$TEZ_HOME/*:$TEZ_HOME/lib/*
where,
$TEZ_HOME
is the location of the directory that contains all the Tez JAR files. By default,$TEZ_HOME
is set to/usr/lib/tez
.