Intra-Queue preemption based on application priorities
Enabling preemption on a queue depending on application priorities ensures that higher-priority applications can preempt resources from lower-priority applications when required.
Example of resource consumption on a queue without preemption
Consider a queue qA with
root.qA.capacity configured at 70%.
Consider applications submitted in the following order:- A user submits an application app1 with priority p1. Because no other application is running on the queue, app1 uses a majority of the resources available on the queue.
- Shortly after app1 starts running, the user submits another application app2 with the same priority as app1. In this situation, app2 uses the resources that remain on the queue.
- The user submits a third application app3 with a higher priority p3.
If preemption is not enabled on the queue, the lower priority applications app1 and app2 consume all of the queue's available capacity leaving the higher priority application app3 starved of resources.
The following table explains the resource distribution between the three
applications:
| Application | Priority | Consumed Resources | Pending Resources |
|---|---|---|---|
| app1 | p1 | 50 | 20 |
| app2 | p1 | 20 | 20 |
| app3 | p3 | 0 | 80 |
Example of resource consumption on a queue with preemption
Consider the same queue and applications with priorities as the previous example.
If preemption based on application priority is enabled on the queue, resources are preempted from app1 and app2 for the higher-priority app3 to run.
The following table explains the resource distribution between the three applications
when preemption is enabled:
| Application | Priority | Preempted Resources | Consumed Resources | Pending Resources |
|---|---|---|---|---|
| app1 | p1 | 16 | 34 | 36 |
| app2 | p1 | 19 | 1 | 39 |
| app3 | p3 | 0 | 35 | 45 |
