Configure other LLAP properties
Configuring LLAP involves setting properties in YARN and Hive. After you configure the llap queue, you need to go back to the Hive configuration to continue setting up low-latency analytical processing (LLAP).
In this task, you accept the Ambari configuration of a number of properties or reconfigure the properties. Ambari generally configures the number of nodes that run an LLAP daemon and total concurrent queries depending on the size of the llap queue. Ambari also attempts to correctly configure the following properties for your particular cluster:
- Memory per Daemon
- YARN container size for each daemon (MB)
- In-Memory Cache per Daemon
- Size of the cache in each container (MB)
- Number of executors per LLAP Daemon
- The number of fragments that can execute in parallel on a daemon
Accept or change the Number of Nodes Used By Hive LLAP
num_llap_nodes_for_llap_daemonsproperty). For example, accept using 2 nodes for LLAP.
Accept the Maximum Total Concurrent Queries
hive.server2.tez.sessions.per.default.queueproperty), or make changes.
Check Memory per Daemon (
hive.llap.daemon.yarn.container.mbproperty) and In-Memory Cache per Daemon (
hive.llap.io.memory.sizeproperty).This memory (
hive.llap.daemon.yarn.container.mb) plus the cache (
hive.llap.io.memory.size) must fit within the container size specified for the YARN container. The YARN container configuration setting appears in .
Accept the Number of Executors per LLAP Daemon
hive.llap.daemon.num.executors), or change this setting if you know what you are doing, and check that the
hive.llap.io.threadpool.sizeis the same value.
Save any Hive configuration changes, and in
, check that the Minimum Container Size (Memory) for YARN is
low.The value should rarely exceed 1024 MB.
Set the Maximum Container Size (Memory) to the same value as the
Memory Allocated for All YARN Containers on a Node.
- In Ambari, select .
In Custom yarn-site, add the following properties unless, upon attempting to
add these properties, Ambari indicates the properties are already
yarn.resourcemanager.monitor.capacity.preemption.natural_termination_factor(value = 1) and
yarn.resourcemanager.monitor.capacity.preemption.total_preemption_per_round(as described below).Calculate the value of the total preemption per round by dividing 1 by the number of cluster nodes. Enter the value as a decimal.For example, if your cluster has 20 nodes, then divide 1 by 20 and enter 0.05 as the value of this property setting.
- Save YARN changes, and go back to the Hive configuration.