The Spark History Server is a monitoring tool that lists information about completed Spark applications. By default, applications write history data to a directory on the Application Timeline Server. The History Server pulls the data and presents it in a Web UI at <host>:18080 (by default).

On production systems we recommend configuring the Spark History Server to use HDFS instead of ATS. The next subsection describes how to configure the Spark History Server to use HDFS.

For more information about History Server properties, see the Apache Monitoring and Instrumentation document. For Kerberos considerations, see Configuring Spark for a Kerberos-Enabled Cluster.