Hive Performance Tuning
Also available as:
PDF

Better Workload Management by Using Queues

Queues are the primary method used to manage multiple workloads. Queues can provide workload isolation and can guarantee that capacity is available for different workloads. Queues can also support meeting Service Level Agreements (SLAs) for different workloads.

Within each queue, you can allow one or more sessions to live simultaneously. Sessions cooperatively share the resources of the queue.

For example, if you have a queue that is assigned 10% of cluster resources, those cluster resources can be allocated anywhere in the cluster, depending on the query and data placement. Where resources are allocated might change as more queries run.

The following figure shows a configuration in which 50% of the cluster capacity is assigned to a "Default" queue for batch jobs, along with two queues for interactive Hive queries. Each Hive queue is assigned 25% of cluster resources. Two sessions are used in the batch queue and three sessions are used in each Hive queue:

Figure 2.4. Cluster Configuration Using Queues