Running raw Scala code in Cloudera Data Engineering

Cloudera Data Engineering (CDE) supports running raw Scala code from the command line, without compiling it into a JAR file. You can use the cde spark submit command to run a .scala file. CDE recognizes the file as Scala code and runs it using spark-shell in batch mode rather than spark-submit.

Run cde spark submit as follows to run a Scala file:

cde spark submit filename.scala --jar <jar_dependency_1> --jar <jar_dependency_2> ...