Configuring preemption

Preemption allows applications of higher priority to preempt applications of lower priority.

A scenario can occur in which a queue has a guaranteed level of cluster resources, but must wait to run applications because other queues are utilizing all of the available resources. If preemption is enabled, applications of higher priority do not have to wait because applications of lower priority have taken up the available capacity. When you enable Preemption (yarn.resourcemanager.scheduler.monitor.enable) underserved queues can begin to claim their allocated cluster resources almost immediately, without having to wait for other queues' applications to finish running.

You can use Priority Scheduling to run YARN applications at a higher priority, regardless of other applications that are already running in the cluster . For more information, see Setting global maximum application priority.

To configure the Queue Preemption options on all queues, perform the following:

  1. In Cloudera Manager, navigate to YARN > Configuration.
  2. Search for preemption.
  3. Find the Capacity Scheduler Preemption property.
  4. Ensure that it is checked, meaning that Preemption is enabled.
  5. In Cloudera Manager, navigate to Clusters > YARN Queue Manager UI > Configuration.
  6. Configure the required preemption properties:
    • Preemption: Observe Only - Select the checkbox to run the policy, but not affect the cluster with preemption and stop events.
    • Preemption: Monitoring Interval (ms) - The time in milliseconds between invocations of this policy. Setting this value to a longer time interval causes the Capacity Monitor to run less frequently.
    • Preemption: Maximum Wait Before Kill (ms) - The time in milliseconds between requesting a preemption from an application and stopping the container. Setting this to a higher value gives applications more time to respond to preemption requests and gracefully release containers.
    • Preemption: Total Resources Per Round - The maximum percentage of resources preempted in a single round. You can use this value to restrict the pace at which containers are reclaimed from the cluster. After computing the total desired preemption, the policy scales it back to this limit.
    • Preemption: Over Capacity Tolerance - The maximum amount of resources above the target capacity ignored for preemption. The default value is 0.1, which means that the Resource Manager starts preemption for a queue only when it goes 10% above its guaranteed capacity. This avoids resource-rotation and aggressive preemption.
    • Preemption: Maximum Termination Factor - The maximum percentage of each queue's preemption target capacity that is preempted per cycle. You can increase this value to speed up resource reclamation.
  7. Click Save.

    For information about configuring preemption for a specific queue, see Configure preemption for a specific queue.