Set up Queues
Capacity Scheduler queues can be set up in a hierarchy that reflects the database structure, resource requirements, and access restrictions required by the various organizations, groups, and users that utilize cluster resources.
The fundamental unit of scheduling in YARN is a queue. The capacity of each queue specifies the percentage of cluster resources that are available for applications submitted to the queue.
Each child queue is tied to its parent queue with the
yarn.scheduler.capacity.<queue-path>.queues
configuration property in the capacity-scheduler.xml file. The top-level
"support", "engineering", and "marketing" queues would be tied to the "root" queue
as follows:
Property:
yarn.scheduler.capacity.root.queues
Value:
support,engineering,marketing
Example:
<property> <name>yarn.scheduler.capacity.root.queues</name> <value>support,engineering,marketing</value> <description>The top-level queues below root.</description> </property>
Similarly, the children of the "support" queue would be defined as follows:
Property:
yarn.scheduler.capacity.support.queues
Value:
training,services
Example:
<property> <name>yarn.scheduler.capacity.support.queues</name> <value>training,services</value> <description>child queues under support</description> </property>
The children of the "engineering" queue would be defined as follows:
Property:
yarn.scheduler.capacity.engineering.queues
Value:
development,qa
Example:
<property> <name>yarn.scheduler.capacity.engineering.queues</name> <value>development,qa</value> <description>child queues under engineering</description> </property>
And the children of the "marketing" queue would be defined as follows:
Property:
yarn.scheduler.capacity.marketing.queues
Value:
sales,advertising
Example:
<property> <name>yarn.scheduler.capacity.marketing.queues</name> <value>sales,advertising</value> <description>child queues under marketing</description> </property>