Setting YARN Queue Priorities
About This Task
Even with preemption enabled, there are some use cases where applications might not have access to cluster resources without setting priorities:
Long-running applications – Without setting priorities, long-running applications in queues that are under capacity and with lower relative resource usage may not release cluster resources until they finish running.
Applications that require large containers – The issue with long-running applications is exacerbated for applications that require large containers. With short-running applications, previous containers may eventually finish running and free cluster resources for applications with large containers. But with long-running services in the cluster, the large containers may never get sufficiently large resources on any nodes.
Hive LLAP – Hive LLAP (Low-Latency Analytical Processing) enables you to run Hive queries with low-latency in near real-time. To ensure low-latency, you should set the priority of the queue used for LLAP to a higher priority, especially if your cluster includes long-running applications.
![[Note]](../common/images/admon/note.png)
Note To set the queue used for Hive LLAP, select Hive > Config > Settings on the Ambari dashboard, then select a queue using the Interactive Query Queue drop-down. For more information, see the Hive Performance Tuning guide.
For example, the following figure shows a 3-node cluster with long-running 20 GB containers. The LLAP daemons require 90 GB of cluster resources, but preemption does not occur because the available queues are under capacity with lower relative resource usage. With only 80 GB available on any of the nodes, LLAP must wait for the long-running applications to finish before it can access cluster resources.
Prerequisites
![[Important]](../common/images/admon/important.png) | Important |
|---|---|
In order for YARN Queue Priorities to be applied, you must enable preemption. |
Steps
On the YARN Queue Manager view, select a queue, then enter a priority in the Priority box under Resources. All queues are set to a priority of 0 by default. Higher numbers indicate higher priority.
Select Actions > Save and Refresh Queues to save the priority setting.
