Apache Hive Performance Tuning
Also available as:
PDF

Guidelines for a Mix of Batch and Interactive Queues

When using a mix of batch and interactive queues, Hortonworks recommends that you use multiple HiveServer2 instances with different settings for different applications. The following section explains how to split up queues based on usage.

Usage-Based Queue Capacity Change

Set up two queues--one for batch and one for interactive workloads. Often, one of the queues is under-utilized while the other queue is overwhelmed. In this situation, it is optimum if one queue can use the unused resources of the other queue. To load-balance between queues, set the capacity and maximum capacity properties for queues in the conf/capacity-scheduler.xml file. Then when a queue workload reaches the level specified in the capacity property and there is additional workload and unused capacity in the other queue, the workload can expand to take up to the level specified in the maximum capacity property. However, if the queue needs the capacity back, it is returned if the first queue requests it.

[Note]Note

For information about how to set up a mix of batch and interactive queues, see Setting Up Queues for Mixed Interactive and Batch Workloads.