Validating Spark
Start Spark, in either yarn-cluster mode or yarn-client mode.
yarn-cluster mode:
/usr/hdp/current/spark-client/bin/spark-submit --class <path.to.your.class> --master yarn-cluster [options] <app jar> [app options]yarn-client mode:
./usr/hdp/current/spark-client/bin/spark-shell --master yarn-client
To validate the Spark installation, run the Spark jobs described in Developing Spark Applications in the Spark Guide.