Setting queue priorities
By setting queue priorities you can ensure that applications can access cluster resources.
Although the preemption feature, which is enabled by default, improves
cluster resources use, there are some use cases where applications might not have
access to cluster resources without setting priorities:
- Long-running applications:
- Without setting queue priorities, long-running applications in queue that are under capacity and with lower relative resource use may not release cluster resources until they finish running.
- Applications that require large containers:
- The issue with long-running applications is exacerbated for applications that require large containers. With short-running applications, previous containers may eventually finish running and free the cluster resources for applications with large containers. But with long-running services in the cluster, the large containers may never get sufficiently large resources on any nodes.
- Hive LLAP
- Hive LLAP (Low-Latency Analytical Processing) enables you to run Hive queries with low-latency in near real-time. To ensure low-latency, you should set the priority of the queue used for LLAP to a higher priority, especially if your cluster includes long-running applications.
For more information, see Configuring preemption and Enabling Intra-Queue preemption
Configure preemption as described in Configuring preemption.