Configuring Apache Spark
Also available as:

Configuring the Livy Server

Use the following steps to configure the Livy server.

Configure the Livy Server

  1. Select Spark > Config on the Ambari dashboard, then select Custom livy-conf.
  2. Add a livy.superusers property and set it to the Zeppelin service account.

For a non-Ambari cluster, see "Installing and Configuring Livy" in the Spark 2 chapter of the Command Line Installation Guide.

Configure SSL for the Livy Server

To enable SSL for Livy, configure the following parameters for the SSL certificate and key, the keystore password, and the key password, respectively:

livy.keystore.password = <storePassword>
livy.key-password = <KeyPassword>

For background information about configuring SSL for Spark, see "Configuring Spark for Wire Encryption" in this guide.

Configure High Availability for the Livy Server

By default, if the Livy server fails, all connected Spark clusters are terminated. This means that all jobs and data will disappear immediately.

For deployments that require high availability, Livy supports session recovery, which ensures that a Spark cluster remains available if the Livy server fails. After a restart, the Livy server can connect to existing sessions and roll back to the state before failing.

Livy uses several property settings for recovery behavior related to high availability. These settings are managed by Ambari.