Placement rules evaluation engine
If you are upgrading to Cloudera Runtime 7.1.6 or higher you need to learn about the new placement rules evaluation engine.
This list is only an overview of the most prominent changes regarding placement rules evaluation. For a more detailed guide, see Managing placement rules.
From Cloudera Runtime 7.1.6 placement rules are described in a JSON-based format. The new engine has full backward compatibility with the old format as well but only if you use safety valves which you must not do. Cloudera recommends managing your placement rules through the YARN Queue Manager UI. The format conversion is automatically done at the first time you use the YARN Queue Manager UI in CDP Private Cloud Base 7.1.6 or higher. For more information about the automatic placement rules format conversion, see Converting old mapping rule format to JSON-based placement rule format.
Enabling dynamic queues
- Enable the dynamic auto child creation feature for the queue that you want to use as a parent queue for the dynamic child queues.
- Ensure that the Create the target queue if it does not exist? property is checked when the applicable placement rule is created.
Fallback action configuration
- Skip: Ignore the current rule and proceed to the next.
- PlaceDefault: Place the application to the default queue
root.default(unless it is overriden to something else). This is how Capacity Scheduler worked with the old mapping rules.
- Reject: Reject the submission.
Placement rule policies
Placement rule policies were introduced as a more user friendly solution for mapping rule creation. Placement rule policies cover the functionality of the placement rules and provide shortcuts for the most common use cases.
All legacy Capacity Scheduler mapping rule options are now available as policy. They provide
the same functionality without having to manually write rules. For example, the
u:%user:%primary_group.%user rule can be achieved using the
primaryGroupUser policy. However, a custom policy option is available that
allows you to define the target queue manually without using any predefined placement rule