Placement rule policies

When creating a placement rule you have to set its policy. There are many predefined policies you can choose from, or you can create a custom policy.

The following table lists the names of the policies, the options displayed in the Placement Rule creation dialog box in the Queue Manager UI, and their detailed description:

Table 1. Placement rule policies
Policy	Queue Manager UI	Description
user	Place the application into a queue named for the user.	Places the application into a queue which matches the username of the submitter.
primaryGroup	Place the application into a queue named for the user’s primary group.	Places the application into a queue which matches the primary group of the submitter.
primaryGroupUser	Place the application into a queue named for the user that is the child of a queue named for the user’s primary group.	Places the application into the queue hierarchy `[parentQueue].<primaryGroup>.<userName>`. The parentQueue is optional.
secondaryGroup	Place the application into a queue named for the user’s secondary group.	Places the application into a queue which matches the secondary group of the submitter.
secondaryGroupUser	Place the application into a queue named for the user that is the child of a queue named for the user’s secondary group.	Places the application into the queue hierarchy `[parentQueue].<secondaryGroup>.<userName>`. The parentQueue is optional.
applicationName	Place the application into a queue named for the application.	Places the application into a queue which matches the name of the application. important it is case-sensitive, white spaces are not removed.
specified	Place the application in the queue specified at the run time.	Places the application to the queue that was defined during submission.
reject	Reject the application.	Rejects the submission.
defaultQueue	Place the application into the default queue.	Places the application into the default queue root.default or to its overwritten value.
setDefaultQueue	Set the default queue to:	Changes the default queue from root.default. This policy does not permanently change the default queue. It is always “root.default” when the evaluation begins when an application is submitted. However, the adjusted default queue remains in effect until the placement rules evaluation is complete. important This policy rule does not place the application in the specified queue. Fallback will not be triggered, no matter what the selection is.
custom	Use the following custom policy:	Enables the user to use custom placement strings.

Custom placement policy

The custom placement policy can describe other policies with the appropriate variable placeholders. For example, primaryGroupUser with the parent queue root.groups can be expressed as: root.groups.%primary_group.%user

The Custom policy variables table described what variables are available if you intend to use the custom policy.

Internally, the tool populates certain variables with appropriate values. These can be used if custom mapping policy is selected. The placement rules evaluation engine does only minimal verification when it comes to replacing them. Therefore it is your responsibility to provide the correct string.

Table 2. Custom policy variables
Variable	Meaning
`%application`	The name of the submitted application.
`%default`	The default queue of the scheduler.
`%primary_group`	Primary group of the submitter.
`%secondary_group`	Secondary (supplementary) group of the submitter.
`%specified`	Contains the queue that the submitter defined. It is a standalone variable, do not combine it with other custom variables or paths. If the specified target queue is `default` this variable will not be set. If the target queue is the `default` queue, it should be specified with the `root.default` parent path.
`%user`	The user who submitted the application.

If you submit a MapReduce application to a queue root.users.mrjobs, the value of %specified will be set to root.users.mrjobs.

Few policies can be achieved with custom. Instead of using the specified policy, you can use custom and set the customPlacement field to %specified.

However, you have more control over it, because you can also append or prepend an extra string to these variables. So the following setting is possible:root.groups.%primary_group.admins.%user.longrunningjobs

The string must be resolved to a valid queue path to have a proper placement.

Differences between legacy modes and weight mode

There are some cases when the legacy resource allocation modes (absolute and relative modes) behave differently than weight mode.

The create flag

Legacy modes: It has no effect if the parent is not managed.
Weight mode: It applies to all parent queues. However, if yarn.scheduler.capacity.<queue-path>.auto-queue-creation-v2.enabled is set to false, then the queue will not be created.

The nested rules primaryGroupUser and secondaryGroupUser

Legacy modes: They expect the parent queues to exist, meaning they cannot be created automatically. More specifically, when you use primaryGroupUser it will result in a queue path similar to root.<primaryGroup>.<userName> and the root.<primaryGroup> parent queue must exist. It can be a managed parent in order to have the userName leaf created automatically, but the parent still has to be created manually.
Weight mode: As long as the parent allows dynamic queues to be created, there is no limitations. The requested queues will be created.