Enabling dynamic child creation in weight mode
In the weight resource allocation mode, when you enable dynamic child creation for a queue, it becomes a parent queue that can have both static and dynamic child queues. You can enable that feature through the YARN Queue Manager UI.
In the weight mode, there is no Managed Parent Queue. When you enable the dynamic child creation feature for a queue, it becomes a parent queue that can have both static and dynamic child queues simultaneously. If this feature is not enabled, the queue can only have static child queues.
In contrast to Manage Parent Queue where the dynamic queue nesting level is limited to one, in weight mode, auto dynamic child creation allows you to create 2-level dynamic queues.
By default dynamic child queues are deleted automatically 5 minutes after the last job finished on them. You can disable this feature using the Capacity Scheduler Auto Queue Deletion YARN property. For more information, see Disabling Auto Queue Deletion.
In Cloudera Manager, select the YARN Queue Manager
A graphical queue hierarchy is displayed in the Overview tab.
- Find the queue for which you want to enable the auto dynamic child creation feature.
Select the More Options menu and select
Enable Dynamic Child Creation.
Dynamic Child Queue Capacities window is displayed.
Set the Minimum and Maximum capacities that will be applied for every dynamic
child queue under that particular parent queue.
Save the minimum and maximum capacity with the default values that copy the parent queue, or change them using the Auto Queue Creation template (Technical Preview) feature.
Technical Preview: This is a technical preview feature and considered under development. Do not use this in your production systems. To share your feedback, contact Support by logging a case on our Cloudera Support Portal. Technical preview features are not guaranteed troubleshooting guidance and fixes.
If you want to define a placement rule that could lead to dynamically created child queues, ensure that during placement rule creation you check the Create the target queue if it does not exist? property and provide a parent queue for which dynamic child creation is enabled. For more information, see Manage placement rules.