Configuring the Spark History Server
The Spark History Server is a monitoring tool that lists information about completed
Spark applications. By default, applications write history data to a directory on the
Application Timeline Server. The History Server pulls the data and presents it in a Web
UI at <host>:18080
(by default).
On production systems we recommend configuring the Spark History Server to use HDFS instead of ATS. The next subsection describes how to configure the Spark History Server to use HDFS.
For more information about History Server properties, see the Apache Monitoring and Instrumentation document. For Kerberos considerations, see Configuring Spark for a Kerberos-Enabled Cluster.