Managing Data Operating System
Also available as:
PDF
loading table of contents...

Enable Priority Scheduling

You can use Priority Scheduling to run YARN applications at higher priority, regardless of other applications that are already running in the cluster. YARN allocates more resources to applications running at a higher priority over those running at a lower priority. Priority Scheduling enables you to set an application's priority both at the time of submission and dynamically at run time.

Priority Scheduling works only with the FIFO (first-in, first-out) ordering policy. FIFO is the default Capacity Scheduler ordering policy.
  1. Set the cluster maximum and leaf-queue level priorities.
    • Cluster maximum priority: Set the following property in the yarn-site.xml file to define the maximum priority for an application in the cluster:

      yarn.cluster.max-application-priority

      Any application submitted with a priority greater than this setting has its priority reset to the yarn.cluster.max-application-priority value.

    • Leaf queue-level priority: Set the following property in the capacity-scheduler.xml file to define the default application priority in a leaf queue:

      yarn.scheduler.capacity.root.<leaf-queue-path>.default-application-priority
      The default application priority is used for any application submitted without a specified priority.
  2. Use either the yarn application -appID command-line option or the Cluster Application REST API to set the priority for already running applications.