Installing the cde-env tool

You can install the cde-env tool as an Administrator or as a normal user. Cloudera recommends you to install the tool as an Administrator in the /opt/cloudera/bin folder so that all the users in the host can access the tool.

  1. Install the tool by copying the required binary and script files to the /opt/cloudera/bin folder so that the migration tool can run in the current gateway host.
    $ ./ enable-spark-submit-proxy
  2. Enable the installed binary and script files by granting access to those files for all users on the host using one of the following options.
    • Link the existing /usr/bin/spark-submit and /usr/bin/spark3-submit.
      $ ln -s /opt/cloudera/bin/cde /usr/bin/cde
      $ ln -s /opt/cloudera/bin/ /usr/bin/
      $ ln -s -f /opt/cloudera/bin/spark-submit /usr/bin/spark-submit
      $ ln -s -f /opt/cloudera/bin/spark3-submit /usr/bin/spark3-submit


    • Update PATH to point to the new installation location at the host level.
      $ export PATH=/opt/cloudera/bin:$PATH            
  1. Install the tool on the host by running the following command:

    By default, the binary and script files will be installed in the $HOME/bin folder. You can change the location by replacing the $HOME/bin folder to the target folder in the following command.

    $ sed -i "s#CLOUDERA_BIN=/opt/cloudera/bin#CLOUDERA_BIN=$HOME/bin#g" && ./ enable-spark-submit-proxy -f private
    $ sed -i '' "s#CLOUDERA_BIN=/opt/cloudera/bin#CLOUDERA_BIN=$HOME/bin#g" && ./ enable-spark-submit-proxy -f private
  2. Update PATH to give access to those binary and script files.
    $ export PATH=$HOME/bin:$PATH