Cloudera Docs

Enabling preemption for a specific queue

Capacity Scheduler Preemption allows higher-priority applications to preempt lower-priority applications.

A scenario can occur in which a queue has a guaranteed level of cluster resources, but must wait to run applications because other queues are utilizing all of the available resources. If Preemption is enabled, higher-priority applications do not have to wait because lower priority applications have taken up the available capacity. With Preemption enabled, under-served queues can begin to claim their allocated cluster resources almost immediately, without having to wait for other queues' applications to finish running.

You can disable the queue preemption (yarn.resourcemanager.scheduler.monitor.enable) for a specific queue.

To enable or disable preemption at the queue level in the Queue Manager UI, you must enable monitoring policies in the global configuration level and define the monitoring policies' class.
  • In the YARN Queue Manager WebUI > Configuration , check Enable Monitoring Policies and set Monitoring Policies to org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy and save to enable/disable preemption for specific queues.

With Enable Monitoring Policies disabled the queue-level preemption option is greyed out and indicates the inherited configuration (enabled by default).

  1. In Cloudera Manager, select Clusters > YARN Queue Manager UI service. A graphical queue hierarchy is displayed in the Overview tab.
  2. Click on the three vertical dots on the queue and select View/Edit Queue Properties option.
  3. In the Queue Properties dialog-box, uncheck the Enable Preemption checkbox.
  4. Click Save.