Command Line Installation
Also available as:
PDF
loading table of contents...

Configuring Tez with the Capacity Scheduler

You can use the tez.queue.name property to specify which queue is used for Tez jobs. Currently the Capacity Scheduler is the default scheduler in HDP. In general, this is not limited to the Capacity Scheduler, but applies to any YARN queue.

If no queues are configured, the default queue is used, which means that 100% of the cluster capacity is used to run Tez jobs. If queues are configured, a queue name must be configured for each YARN application.

Setting tez.queue.name in tez-site.xml applies to Tez applications that use that configuration file. To assign separate queues for each application, separate tez-site.xml files are required, or the application can pass this configuration to Tez while submitting the Tez DAG.

For example, in Hive you can use the tez.queue.name property in hive-site.xml to specify the queue to use for Hive-on-Tez jobs. To assign Hive-on-Tez jobs to use the "engineering" queue, add the following property to hive-site.xml:

<property>
     <name>tez.queue.name</name>
     <value>engineering</value>
</property> 

Setting this configuration property in hive-site.xml affects all Hive queries that read that configuration file.

To assign Hive-on-Tez jobs to use the "engineering" queue in a Hive query, use the following commands in the Hive shell or in a Hive script:

bin/hive --hiveconf tez.queue.name=engineering