What's New in Apache Hadoop YARN

Fair sharing intra-queue preemption support (Technical Preview)

You can now enable intra-queue preemption for queues that are configured with fairness-based ordering. If the user has configured ordering-policies for queues (yarn.scheduler.capacity.<queue-path>.ordering-policy) to be Fair and if this new feature intra-queue preemption is enabled using YARN Queue Manager UI, then all the applications from the same user get a fair amount of resources. Thus, fair-ordering preemption now ensures that each application under a queue gets its fair-share, whether from a single user or several.

While that is for applications from a single user, for resource allocation across users, you can enforce a limit on each user’s resource-usage by setting the user limits (yarn.scheduler.capacity.<queue-path>.user-limit-factor).

For more information about ordering policies, see Set Ordering policies within a specific queue.