Managing Data Operating System
Also available as:
PDF
loading table of contents...

Intra-Queue Preemption based on User Limits

Enabling preemption on a queue based on user limits ensures that resources are uniformly distributed among all users who submit applications to the particular queue.

Example of resource consumption on a queue without preemption

Consider a queue qA with root.qA.capacity configured at 100% and minimum-user-limit-percent configured at 33%. This implies that the first three users submitting applications to the queue can each use a minimum of 33% of the queue's resources. If the three users are already consuming the queue's resources as specified, then any additional user must wait for resources to be allocated before submitting applications.

Consider applications submitted in the following order:
  1. The user u1 submits an application app1 with priority p1. Because no other application is running on the queue, app1 uses a majority of the resources available on the queue.
  2. Shortly after app1 starts running, users u2 and u3 respectively submit applications app2 and app3 around the same time with priority p1. In this situation, app2 and app3 use the resources that remain on the queue.

    If preemption is not enabled on the queue, app2 and app3 cannot consume their share of resources on the queue in spite of having the same priority as app1.

The following table explains the resource distribution between the three applications:
Application Users Consumed Resources Pending Resources
app1 u1 60 30
app2 u2 20 25
app3 u3 20 25

Example of resource consumption on a queue with preemption

Consider the same queue and applications with priorities as the previous example.

If preemption based on user limits is enabled on the queue, resources are preempted from app1 for app2 and app3 to run.

The following table explains the resource distribution between the three applications when preemption is enabled:
Application Priority Preempted Resources Consumed Resources Pending Resources
app1 u1 26 34 56
app2 u2 0 33 12
app3 u3 0 33 12