Non-Ambari Cluster Installation Guide
Also available as:
PDF
loading table of contents...

Validating Spark

Start Spark, in either yarn-cluster mode or yarn-client mode.

  • yarn-cluster mode:

    /usr/hdp/current/spark-client/bin/spark-submit --class <path.to.your.class> --master yarn-cluster [options] <app jar> [app options]

  • yarn-client mode:

    ./usr/hdp/current/spark-client/bin/spark-shell --master yarn-client

To validate the Spark installation, run the Spark jobs described in Developing Spark Applications in the Spark Guide.