Configure Preemption

Property: yarn.resourcemanager.scheduler.monitor.enable

Value: true

Description: Setting this property to "true" enables Preemption. It enables a set of periodic monitors that affect the Capacity Scheduler. This default value for this property is "false" (disabled).

Property: yarn.resourcemanager.scheduler.monitor.policies

Value:

Description: The list of SchedulingEditPolicy classes that interact with the scheduler. The only policy currently available for preemption is the “ProportionalCapacityPreemptionPolicy”.

Property: yarn.resourcemanager.monitor.capacity.preemption.monitoring_interval

Value: 3000

Description: The time in milliseconds between invocations of this policy. Setting this value to a longer time interval will cause the Capacity Monitor to run less frequently.

Property: yarn.resourcemanager.monitor.capacity.preemption.max_wait_before_kill

Value: 15000

Description: The time in milliseconds between requesting a preemption from an application and killing the container. Setting this to a higher value will give applications more time to respond to preemption requests and gracefully release Containers.

Property: yarn.resourcemanager.monitor.capacity.preemption.total_preemption_per_round

Value: 0.1

Description: The maximum percentage of resources preempted in a single round. You can use this value to restrict the pace at which Containers are reclaimed from the cluster. After computing the total desired preemption, the policy scales it back to this limit. This should be set to (memory-of-one-NodeManager)/(total-cluster-memory). For example, if one NodeManager has 32 GB, and the total cluster resource is 100 GB, the total_preemption_per_round should set to 32/100 = 0.32. The default value is 0.1 (10%).

Property: yarn.resourcemanager.monitor.capacity.preemption.natural_termination_factor

Value: 1.0

Description: Similar to total_preemption_per_round, you can apply this factor to slow down resource preemption after the preemption target is computed for each queue (for example, “give me 5 GB back from queue-A”). For example, if 5 GB is needed back, in the first cycle preemption takes back 1 GB (20% of 5GB), 0.8 GB (20% of the remaining 4 GB) in the next, 0.64 GB (20% of the remaining 3.2 GB) next, and so on. You can increase this value to speed up resource reclamation. The recommended value for this parameter is 1.0, meaning that 100% of the target capacity is preempted in a cycle.

Property: yarn.resourcemanager.monitor.capacity.preemption.max_ignored_over_capacity

Value: 0.1

Description: The maximum amount of resources above the target capacity ignored for preemption. When configured to a value x >= 0, Resource Manager will wait till a queue uses resources amounting to configured_capacity * (1 + x) before starting to preempt containers from it. By default, it is 0.1, which means Resource Manager will start preemption for a queue only when it goes 10% above its guaranteed capacity. This avoids resource-joggling and aggressive preemption.