Visualizing Spark Applications Using the Web Application UI

Every Spark application launches a web application UI that displays useful information about the application.

The web application UI includes the following information:

  • An event timeline that displays the relative ordering and interleaving of application events. The timeline view is available on three levels: across all jobs, within one job, and within one stage. The timeline also shows executor allocation and deallocation.
  • A list of stages and tasks.
  • The execution directed acyclic graph (DAG) for each job.
  • A summary of RDD sizes and memory usage.
  • Environment - runtime information, property settings, library paths.
  • Information about Spark SQL jobs.

The web UI is available in different ways depending on whether the application is running or has completed.