15. Preemption Configuration

The following properties in the /etc/hadoop/conf/yarn-site.xml file on the ResourceManager host are used to enable and configure Preemption.

  • Property: yarn.resourcemanager.scheduler.monitor.enable

    Value: true

    Description: Setting this property to "true" enables Preemption. It enables a set of periodic monitors that affect the Capacity Scheduler. This default value for this property is "false" (disabled).

  • Property: yarn.resourcemanager.scheduler.monitor.policies

    Value: org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity. ProportionalCapacityPreemptionPolicy

    Description: The list of SchedulingEditPolicy classes that interact with the scheduler. The only policy currently available for preemption is the “ProportionalCapacityPreemptionPolicy”.

  • Property: yarn.resourcemanager.monitor.capacity.preemption.monitoring_interval


    Description: The time in milliseconds between invocations of this policy. Setting this value to a longer time interval will cause the Capacity Monitor to run less frequently.

  • Property: yarn.resourcemanager.monitor.capacity.preemption.max_wait_before_kill


    Description: The time in milliseconds between requesting a preemption from an application and killing the container. Setting this to a higher value will give applications more time to respond to preemption requests and gracefully release Containers.

  • Property: yarn.resourcemanager.monitor.capacity.preemption.total_preemption_per_round

    Value: 0.1

    Description: The maximum percentage of resources preempted in a single round. You can use this value to restrict the pace at which Containers are reclaimed from the cluster. After computing the total desired preemption, the policy scales it back to this limit.

loading table of contents...