Configure queue mapping to use the user name from the application tag using Cloudera Manager
You can configure queue mapping to use the user name from the application tag instead of the proxy user who submitted the job. You can add only service users like hive using the yarn.resourcemanager.application-tag-based-placement.username.whitelist property and not normal users.
When a user runs Hive queries, HiveServer2 submits the query in the queue mapped from
an end user instead of a hive user. For example, when user alice submits a
Hive query with doAs=false mode, job will run in YARN as
hive user. If application-tag based scheduling is enabled, then the job
will be placed to a target queue based on the queue mapping configuration of user
alice.
How to configure queue mapping
- In Cloudera Manager, select the YARN service.
- Click the Configuration tab.
- Search for ResourceManager. In the Filters pane, under Scope, select ResourceManager.
- In ResourceManager Advanced Configuration Snippet (Safety Valve) for
yarn-site.xml add the following:
- Enable the
application-tag-based-placement
property to enable application placement based on the user ID passed using the application tags.Name: yarn.resourcemanager.application-tag-based-placement.enable Value: true Description: Set to "true" to enable application placement based on the user ID passed using the application tags. When it is enabled, it checks for the userid=<userId> pattern and if found, the application will be placed onto the found user's queue, if the original user has the required rights on the passed user's queue.
- Add the list of users in the allowlist who can use application tag based
placement. The applications when the submitting user is included in the allowlist,
will be placed onto the queue defined in the
yarn.scheduler.capacity.queue-mappings
property defined for the user from the application tag. If there is no user defined, the submitting user will be used.Name: yarn.resourcemanager.application-tag-based-placement.username.whitelist Value: <Hive process user(s)> Description: Comma separated list of users who can use the application tag based placement, if "yarn.resourcemanager.application-tag-based-placement.enable" is enabled.
- Enable the
- Restart the ResourceManager service for the changes to apply.