Installing LLAP on an Unsecured Cluster
On the computer where Hive2 is installed, follow these steps:
Log in as user hive:
sudo su - su $HIVE_USER
Generate the LLAP package:
hive --service llap --name <llap_svc_name> --instances <number_of_cluster_nodes> --cache <cache_size>m --xmx <heap_size>m --size ((<cache_size>+<heap_size>)*1.05)m --executors <number_of_cores> --loglevel <WARN|INFO> --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB"
Table 9.3.
Parameter Recommended Value --instances <cache_size> Number of cluster nodes to use LLAP. --cache <cache_size> <YARN_maximum_container_size> - (<hive.tez.container.size> <number_of_cores>)
<hive.tez.container.size> is the setting for this property, found in the
hive-site.xml
file. Depending on the size of the node, specify a minimum between 1 GB through 4 GB.--xmx <heap_size> For medium-sized nodes: <hive.tex.container.size> * <number of cores> * (0.8 to 0.95)
<hive.tez.container.size> is the setting for this property found in the
hive-site.xml
file.Ensure that the setting for --xmx is 1 GB less than (<hive.tez.container.size> * <number_of_cores>). For smaller nodes, use the same formula, multiplied by 0.8.
--executors <number_of_cores> Set to the number of CPU cores available on nodes running NodeManager. Set this value even if CPU scheduling is enabled in YARN. Set the
--loglevel
parameter toINFO
when you are troubleshooting or testing.The
INFO
option provides verbose output. In a production environment, set the--loglevel
parameter to WARN, which outputs a message to the logs only if there is a warning or error. This makes the logs easier to read and reduces load on the node.Note The recommended values listed in the previous table represent a sample configuration. LLAP also can be configured to use a fraction of node resources.
The following message is returned, confirming that the LLAP package was created:
Prepared llap-slider-<date>/run.sh for running LLAP on Slider
Verify that the following LLAP and HiveServer2 properties are set in the
hive-site.xml
file, which can be located in the/etc/hive/conf/
or the/etc/hive/conf/conf.server/
directory:Table 9.4. LLAP Properties to Set in hive-site.xml
Property Values hive.execution.mode
llap hive.llap.execution.mode all hive.llap.daemon.service.hosts @<llap_service_name> <llap_service_name> is the service name that you specified when you generated the LLAP package in Step 2.
Table 9.5. HiveServer2 Properties to Set in hive-site.xml to Enable Concurrent Queries with LLAP
Property Values hive.server2.tez.default.queues Queue to use for the LLAP sessions. These correspond to YARN queues of the same name. For example, default. hive.server2.tez.initialize.default.sessions True hive.server2.tez.sessions.per.default.queue Set to the number of concurrent queries to run on the queues that are configured by hive.server2.tez.default.queues. This setting launches long-running Tez AMs (sessions). Example of these properties set in the
hive-site.xml
file:<property> <name>hive.execution.mode</name> <value>llap</value> </property> <property> <name>hive.llap.execution.mode</name> <value>ALL</value> </property> <property> <name>hive.llap.daemon.service.hosts</name> <value>@<llap_service_name></value> </property> <property> <name>hive.server2.tez.default.queues</name> <value>default</value> </property> <property> <name>hive.server2.tez.initialize.default.sessions</name> <value>true</value> </property> <property> <name>hive.server2.tez.sessions.per.default.queue</name> <value>1</value> </property>