Setting User Limits
The minimum-user-limit-percent property can be used to set the minimum percentage of resources allocated to each leaf queue user. For example, to enable equal sharing of the "services" leaf queue capacity among five users, you would set the minimum- user-limit property to 20%:
Property: yarn.scheduler.capacity.root.support.services.minimum-user- limit-percent
Value: 20
This setting determines the minimum limit that any user’s share of the queue capacity can shrink to. Irrespective of this limit, any user can come into the queue and take more than his or her allocated share if there are idle resources available.
The following table shows how the queue resources are adjusted as users submit jobs to a queue with a minimum-user-limit-percent value of 20%:
The Capacity Scheduler also manages resources for decreasing numbers of users. As users’ applications finish running, other existing users with outstanding requirements begin to reclaim that share.
Note that despite this sharing among users, the FIFO application scheduling order of Capacity Scheduler does not change. This guarantees that users cannot monopolize queues by submitting new applications continuously. Applications (and thus the corresponding users) that are submitted first always get a higher priority than applications that are submitted later.
Capacity Scheduler’s leaf queues can also use the user-limit-factor
property to control user resource allocations. This property denotes the fraction of queue
capacity that any single user can consume up to a maximum value, regardless of whether or
not there are idle resources in the cluster.
Property:
yarn.scheduler.capacity.root.support.user-limit-factor
Value: 1 The default value of "1"
means that any single user in the queue can at maximum only occupy the queue’s configured
capacity. This prevents users in a single queue from monopolizing resources across all
queues in a cluster. Setting the value to "2" would restrict the queue's users
to twice the queue’s configured capacity. Setting it to a value of 0.5 would restrict any user
from using resources beyond half of the queue capacity.
These settings can also be dynamically changed at run-time using
yarn rmadmin - refreshQueues.