Configuring queue mapping to use the user name from the application tag using Cloudera Manager

You learn how to add service users to the YARN queue by following a mapping procedure.

You can configure queue mapping to use the user name from the application tag instead of the proxy user who submitted the job. You can add only service users like hive using the yarn.resourcemanager.application-tag-based-placement.username.whitelist property and not normal users.

When a user runs Hive queries, HiveServer2 submits the query in the queue mapped from an end user instead of a hive user. For example, when user alice submits a Hive query with doAs=false mode, job will run in YARN as hive user. If application-tag based scheduling is enabled, then the job will be placed to a target queue based on the queue mapping configuration of user alice.

For more information about queue mapping configurations, see Manage placement rulesManage placement rules. For information about Hive access, see Apache Hive documentation.

How to configure queue mapping

  1. In Cloudera Manager, select the YARN service.
  2. Click the Configuration tab.
  3. Search for ResourceManager. In the Filters pane, under Scope, select ResourceManager.
  4. In ResourceManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml add the following:
    1. Enable the application-tag-based-placement property to enable application placement based on the user ID passed using the application tags.
      Name: yarn.resourcemanager.application-tag-based-placement.enable
      Value: true
      Description: Set to "true" to enable application placement based on the user ID passed using the application tags. When it is enabled, it checks for the userid=<userId> pattern and if found, the application will be placed onto the found user's queue, if the original user has the required rights on the passed user's queue.
    2. Add the list of users in the allowlist who can use application tag based placement. The applications when the submitting user is included in the allowlist, will be placed onto the queue defined in the yarn.scheduler.capacity.queue-mappings property defined for the user from the application tag. If there is no user defined, the submitting user will be used.
      Name: yarn.resourcemanager.application-tag-based-placement.username.whitelist
      Value: <Hive process user(s)>
      Description: Comma separated list of users who can use the application tag based placement, if "yarn.resourcemanager.application-tag-based-placement.enable" is enabled.
  5. Restart the ResourceManager service for the changes to apply.