Configuring Apache Spark
Also available as:
PDF

Configure the Spark history server

On a Kerberos-enabled cluster, the Spark history server daemon must have a Kerberos account and keytab.

When you enable Kerberos for a Hadoop cluster with Ambari, Ambari configures Kerberos for the Spark history server and automatically creates a Kerberos account and keytab for it. Ambari configures the Spark history server permissions such that Admin users can see the history of all Spark jobs, and other users can only see the history of jobs they submitted. To customize these settings, select Spark2 > Configs > Custom spark2-defaults in Ambari.

For example, to add Admin users:

  1. Select Spark2 > Configs > Custom spark2-defaults in Ambari.
  2. Add following properties:
    Table 1. Apache Spark History Server ACL Settings
    Property Value
    spark.history.ui.acls.enable true
    spark.history.ui.admin.acls Comma-separated list of Admin users.
  3. Click Save, then restart Spark and any other services that require a restart.