Running Spark applications on secure clusters
All CDP clusters are secure by default. Disabling security on CDP clusters is not supported. To run a Spark application on a secure cluster, you must first authenticate using Kerberos.
Users running Spark applications must first authenticate to Kerberos,
kinit, as follows:
After authenticating to Kerberos, users can submit their applications
spark-submit as usual, as shown below. This
command submits one of the default Spark sample jobs using an environment
variable as part of the path, so modify as needed for your own use:
$ spark-submit --class org.apache.spark.examples.SparkPi --master yarn \ --deploy-mode cluster $SPARK_HOME/lib/spark-examples.jar 10
For information on creating user accounts in CDP, see Onboarding Users.