Configure a YARN queue for workload management

You must add a custom YARN queue for workload management if you enable workload management; otherwise, it is likely that workload management will not start. This queue is in addition to the LLAP queue, named llap by default that is also required for interactive querying and workload management.

The llap queue capacity is reserved for launching LLAP daemons and pre-provisioned YARN Application Masters (AMs), which correspond to Maximum Total Concurrent Queries that you configure during LLAP setup. Typically, the entire llap queue is used when LLAP daemons and AMs are running. There is insufficient capacity for workload management to use the same llap queue because workload management requires more AMs/sessions. Configuring an interactive YARN queue for workload management solves this problem.

Configuration of the workload management queue includes specifying capacity. You need to consider the total cluster capacity, the capacity you have allocated to the llap queue, and the workloads you want to accommodate. For example:

Total cluster capacity = 100 GB
llap queue capacity = 50 GB
Workload management queue capacity = 20 GB

Workload management can launch only 10 AMs/sessions of 2GB container size each. A resource plan that defines query_parallelism greater than 10 does not work (20 * 2GB container = 40GB and only 20GB capacity is available). Plan carefully to create resource plans in accordance with the workload management queue capacity.

In Ambari, select Services > YARN > Configs, and from the hamburger menu Views (top right), select YARN Queue Manager.
Click Add Queue, and enter a queue path (name) wm, for example. Click .
Configure the capacity of the wm queue to greater than zero.
Make adjustments in the capacity and max capacity of other queues as necessary to resolve the error. For example, adjust the default queue from 50 percent to 30 percent:
Select the wm queue under Add Queue, and in Resources that appears on the right, set User Limit Factor to 1, and set Priority to greater than 0 (1 for example).
Select Actions > Save and Refresh Queues.
In Services > YARN > Summary restart any YARN services as prompted.
In Ambari, navigate to Services > Hive > Configs > Advanced > Custom hiveserver2-interactive-site.
Add the following property and set its value to the name of the workload management queue, wm, for example: Key: hive.server2.tez.interactive.queue Value: wm
Save the configuration.