Configuring Tez with the Capacity Scheduler
You can use the tez.queue.name property to specify which queue is used for Tez jobs. Currently the Capacity Scheduler is the default scheduler in HDP. In general, this is not limited to the Capacity Scheduler, but applies to any YARN queue.
If no queues are configured, the default queue is used, which means that 100% of the cluster capacity is used to run Tez jobs. If queues are configured, a queue name must be configured for each YARN application.
Setting tez.queue.name in tez-site.xml
applies to Tez applications
that use that configuration file. To assign separate queues for each application,
separate tez-site.xml
files are required, or the application can pass
this configuration to Tez while submitting the Tez DAG.
For example, in Hive you can use the tez.queue.name property in
hive-site.xml
to specify the queue to use for Hive-on-Tez jobs.
To assign Hive-on-Tez jobs to use the "engineering" queue, add the following
property to hive-site.xml:
<property> <name>tez.queue.name</name> <value>engineering</value> </property>
Setting this configuration property in hive-site.xml
affects all
Hive queries that read that configuration file.
To assign Hive-on-Tez jobs to use the "engineering" queue in a Hive query, use the following commands in the Hive shell or in a Hive script:
bin/hive --hiveconf tez.queue.name=engineering