Cloudera Docs

Accessing the Web UI of a Running Spark Application

You can access the web UI of a running Spark application from a web browser.

To access the web application UI of a running Spark application, open http://spark_driver_host:4040 in a web browser. If multiple applications are running on the same host, the web application binds to successive ports beginning with 4040 (4041, 4042, and so on). The web application is available only for the duration of the application.

When Knox is configured for YARN, the Tracking URL printed in the command line of an YARN application such as spark-submit shows the direct URL instead of the Knox Gateway URL. Perform the following steps to resolve this issue:
  1. Open the Cloudera Manager Admin Console and go to the Knox service.
  2. Click the Knox Gateway Home URL.
  3. Copy the YARN Resource Manager Web UI V2 URL from the Knox Gateway Home page.

    For example, https://knox-gateway.example.com:8443/gateway/cdp-proxy/yarnuiv2/

  4. Open the Cloudera Manager Admin Console and go to the YARN service.
  5. Click the Configuration tab and search for resourcemanager_config_safety_valve.
  6. Add the Resource Manager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml property, and specify its value by using the YARN Resource Manager Web UI V2 URL, copied earlier, as follows:
    Name: yarn.web-proxy.gateway.url
Value: <YARN Resource Manager Web UI V2 URL>
  7. Enter a Reason for Change and then click Save Changes.
  8. Restart YARN.