Configuring Livy
Perform the following steps to configure Livy:
Login as
root
, or use root privilege to create users Livy and Zeppelin:useradd livy -g hadoop useradd zeppelin -g hadoop
Create a log directory for Livy:
mkdir /var/log/livy
Change owner of
/var/log/livy
tolivy:hadoop
.chown livy:hadoop /var/log/livy
/etc/livy/livy.conf
contains information regarding server configuration.Create file
/etc/livy/livy.conf
and add the following to the file:livy.environment production livy.impersonation.enabled true livy.server.csrf_protection.enabled true livy.server.port 8998 livy.server.session.timeout 3600000
/etc/livy/spark-blacklist.conf
defines a list of properties that users are not allowed to override when a Spark session is started.Create an empty file called
/etc/livy/spark-blacklist.conf
.Create file
/etc/livy/livy-env.sh
to define the environmental variables. Add the following to the file:export SPARK_HOME=/usr/hdp/current/spark-client export LIVY_LOG_DIR=/var/log/livy export LIVY_PID_DIR=/var/run/livy export LIVY_SERVER_JAVA_OPTS="-Xmx2g"
If you are not using Kerberos, skip these steps.
If you are using Kerberos, create Livy and Zeppelin principals and keytabs:
kadmin.local -q "addprinc -randkey livy@EXAMPLE.COM" kadmin.local -q "xst -k /etc/security/keytabs/livy.headless.keytab livy@EXAMPLE.COM" kadmin.local -q "addprinc -randkey zeppelin@EXAMPLE.COM" kadmin.local -q "xst -k /etc/security/keytabs/zeppelin.headless.keytab zeppelin@EXAMPLE.COM"
If you are using Kerberos, move the Livy and Zeppelin key tables to the node on which Livy and Zeppelin will run.
chown livy:hadoop /etc/security/keytabs/livy.headless.keytab chown zeppelin:hadoop /etc/security/keytabs/zeppelin.headless.keytab
Ensure that the Livy user can read the contents of the
/etc/livy/conf
directory.