Enabling LazyPreemption

By enabling the LazyPreemption feature you can optimize the selection of containers from queues for preemption. Containers are preempted only if the resources can be used by another application.

Normally when preemption is performed there is no guarantee that preempted resources can be used by pending resource requests from under-satisfied queues. That can lead to excessive preemption as the preemption monitor keeps preempting resources from queue-A, but queue-B cannot use the preempted resources. As a result, such sources are allocated to queue-A again, but then they are preempted again.

The LazyPreemption feature makes resource preemption more efficient. It prevents excessive preemption and ensures that container selection is continuously optimized. It is diabled by default.

Enabling the LazyPreemption feature has the following advantages:

Containers are killed only if the demanding application can use that capacity on the specific nodes.
Application requests that cannot be satisfied at a moment for some reason will not lead to any container kill.
If the application request gets canceled, no containers are killed.

LazyPreemption avoids scenarios when preempted resources cannot be used by the application that needs the resource, and then such preempted resources go back to the over-utilized queue again.

In Cloudera Manager, navigate to YARN > Configuration.
Search for safety valves.
Find the Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve) property.

Set the yarn.scheduler.capacity.lazy-preemption-enabled property to true by adding the following code snipper:

 <property>
    <name>yarn.scheduler.capacity.lazy-preemption-enabled</name>
    <value>true</value>
  </property>
  <property>

Click Save Changes.

LazyPreemption is enabled.