Managing dynamic queues
Dynamic queues are created automatically during application runtime. They are deleted when the YARN service is restarted.
Dynamic queues are created during runtime automatically. Dynamic queues are not defined in the
capacity-scheduler.xml configuration file. Dynamic queues can be created under
those static parents where auto dynamic child creation is allowed. Auto dynamic child creation
has to be set explicitly using the YARN Queue Manager UI.
- A dynamic queue path is specified by the submitter at the time of the job submission. Dynamic queues will be created if the auto dynamic child creation feature is enabled for the parent queue that was provided in the queue path.
- A placement rule applies to the submitted job that could place it into a dynamic queue. Dynamic queue is created based on the predefined expression of the dynamic placement rule.
Based on your resource allocation mode, dynamic queues are managed differently:
In absolute and relative modes, when you enable the auto dynamic child creation feature for a queue, it becomes a Managed Parent Queue. It cannot have static child queues, queues under it can be created only dynamically. It allows 1-level dynamic queue nesting.
In the weight mode, there is no Managed Parent Queue. When you enable the auto dynamic child creation feature for a queue, it becomes a parent queue that can have both static and dynamic child queues. It allows 2-level dynamic queue nesting.
