Non-Ambari Cluster Installation Guide
Also available as:
loading table of contents...

Configuring Tez with the Capacity Scheduler

You can use the property to specify which queue will be used for Tez jobs. Currently the Capacity Scheduler is the default scheduler in HDP. In general, this is not limited to the Capacity Scheduler, but applies to any YARN queue.

If no queues are configured, the default queue is used, which means that 100% of the cluster capacity is used to run Tez jobs. If queues are configured, a queue name must be configured for each YARN application.

Setting in tez-site.xml applies to Tez applications that use that configuration file. To assign separate queues for each application, separate tez-site.xml files are required, or the application can pass this configuration to Tez while submitting the Tez DAG.

For example, in Hive you can use the property in hive-site.xml to specify the queue to use for Hive-on-Tez jobs. To assign Hive-on-Tez jobs to use the "engineering" queue, add the following property to hive-site.xml:


Setting this configuration property in hive-site.xml affects all Hive queries that read that configuration file.

To assign Hive-on-Tez jobs to use the "engineering" queue in a Hive query, use the following commands in the Hive shell or in a Hive script:

bin/hive --hiveconf