Using the cde-env tool
You can run the spark-submit command after installing the
cde-env tool. You can use the cde-env tool even without
administrator privileges.
You can activate the specific profile you want to use by running the following command:
$ cde-env.sh activate -p <profile-name>For example, to alternatively run the the same spark-submit command either
against YARN or one of the Cloudera Data Engineering virtual cluster, you can activate the relevant profile:
- Run spark jobs on YARN by activating the
yarnprofile.yarnis a reserved profile name here.$ cde-env.sh activate -p yarn - Run spark jobs on a Cloudera Data Engineering virtual cluster configured by CDE CLI profile named
vc-1:$ cde-env.sh activate -p vc-1
Switching profiles back and forth this way lets you run the same spark-submit
command either against YARN or one of the Cloudera Data Engineering virtual clusters.
