Cloudera Docs

Setting queue priorities

By setting queue priorities you can ensure that applications can access cluster resources.

Although the preemption feature, which is enabled by default, improves cluster resources use, there are some use cases where applications might not have access to cluster resources without setting priorities:
Long-running applications:
Without setting queue priorities, long-running applications in queue that are under capacity and with lower relative resource use may not release cluster resources until they finish running.
Applications that require large containers:
The issue with long-running applications is exacerbated for applications that require large containers. With short-running applications, previous containers may eventually finish running and free the cluster resources for applications with large containers. But with long-running services in the cluster, the large containers may never get sufficiently large resources on any nodes.
Hive LLAP
Hive LLAP (Low-Latency Analytical Processing) enables you to run Hive queries with low-latency in near real-time. To ensure low-latency, you should set the priority of the queue used for LLAP to a higher priority, especially if your cluster includes long-running applications.

For more information, see Configuring preemption and Enabling Intra-Queue preemption

Configure preemption as described in Configuring preemption.

  1. In Cloudera Manager navigate to Cluster > YARN Queue Manager UI.
    The Overview tab is opened, displaying a hierarchical view of the queues.
  2. Find the queue for which you want to set queue priority and click the three vertical dots next to its name.
  3. Select Edit Queue Properties.
  4. In the Resource Allocation section find the Queue Priority property.
  5. Set the queue priority.
    By default all queues are set to a priority of 0. Higher numbers indicate higher priority.
  6. Click Save.